An automatic identification method and device for a pointer instrument

Abstract

The application provides an automatic identification method of a pointer instrument, which comprises the following steps: obtaining a template picture and prior information of an instrument in the template picture; performing feature matching on a to-be-tested picture and the template picture based on a preset matching algorithm to obtain a target template picture, and determining a first mapping relationship between the to-be-tested picture and the target template picture, wherein the first mapping relationship is used for representing a relationship between angles of the instruments in the to-be-tested picture and the target template picture; performing mapping transformation on the to-be-tested picture based on the first mapping relationship to obtain a transformed picture; determining position information of a pointer in the transformed picture based on a preset detection algorithm; determining a second mapping relationship between an instrument number and the position information of the pointer in the transformed picture based on the prior information of the instrument in the target template picture; and determining the instrument number in the transformed picture based on the second mapping relationship and the position information of the pointer in the transformed picture. The instrument identification method has high accuracy and is not easily affected by light and a shooting angle.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and more specifically to an automatic recognition method for pointer-type instruments. Background Technology

[0002] Pointer-type instruments are widely used in industries such as power systems and petrochemicals due to their simple structure, convenient installation and maintenance, and high reliability. Taking substation systems as an example, the operating status of many devices, such as pressure gauges and thermometers, needs to be monitored through instruments. However, in the daily maintenance of substation systems, traditional manual inspections suffer from drawbacks such as low safety, low efficiency, and poor reliability due to the high pressure and high temperature of the working environment and the subjective factors of the workers. With the continuous advancement of robotics and computer vision technology, using inspection robots equipped with cameras for visual inspection has greatly improved the efficiency and reliability of inspection work.

[0003] In existing technologies, when performing image recognition in complex scenes, factors such as lighting and shooting angle can easily lead to low recognition accuracy and poor recognition results. Summary of the Invention

[0004] In view of this, it is necessary to provide an automatic recognition method for pointer-type instruments to solve the technical problems existing in the prior art, which are easily affected by factors such as lighting and shooting angle in complex scenes, resulting in low recognition accuracy and poor recognition effect.

[0005] To address the aforementioned problems, this invention provides an automatic identification method for pointer-type instruments, comprising:

[0006] Obtain the template image and the prior information of the instrument in the template image;

[0007] Based on a preset matching algorithm, feature matching is performed between the image to be tested and the template image to obtain a target template image, and a first mapping relationship between the image to be tested and the target template image is determined. The first mapping relationship is used to characterize the relationship between the instrument angles in the image to be tested and the target template image.

[0008] Based on the first mapping relationship, the image to be tested is transformed to obtain a transformed image;

[0009] Based on a preset detection algorithm, the position information of the pointer in the transformed image is determined;

[0010] Based on the prior information of the instrument in the target template image, a second mapping relationship between the instrument readings and the pointer position information in the transformed image is determined. Based on the second mapping relationship and the pointer position information in the transformed image, the instrument readings in the transformed image are determined.

[0011] In some possible implementations, the position information of the pointer in the transformed image is determined based on a preset detection algorithm, including:

[0012] Determine the position information of the rotation center of the pointer in the transformed image;

[0013] Based on the rotation center position information of the pointer in the transformed image and the preset Hough line detection algorithm, the position information of the pointer in the transformed image is determined.

[0014] In some possible implementations, when the prior information includes the rotation center position information of the pointer and the radius of the dial, determining the rotation center position information of the pointer in the transformed image includes:

[0015] Based on the radius of the dial in the target template image, Hough circle detection is performed on the transformed image to obtain multiple Hough circles in the dial area of ​​the transformed image, and the center position information of the Hough circles is determined.

[0016] Determine the deviation value between the center of the Hough circle and the rotation center position of the pointer in the target template image, and obtain the target Hough circle with the deviation value within a preset range;

[0017] The transformed image is cropped based on the target Hough circle, and the center of the target Hough circle is used as the rotation center of the pointer in the transformed image.

[0018] In some possible implementations, the prior information includes the maximum and minimum values ​​of the dial, the angle between the minimum and maximum values, and the position information of the zero mark.

[0019] Based on the prior information of the instrument in the target template image, a second mapping relationship is determined between the instrument readings and the pointer position information in the transformed image, including:

[0020] Based on the maximum and minimum values ​​of the dial in the target template image, and the angle between the minimum and maximum values, a third mapping relationship is determined between the instrument readings in the transformed image and the deflection angle between the pointer and the zero mark of the dial.

[0021] Based on the position information of the zero mark line on the dial in the target template image, a fourth mapping relationship is determined between the deflection angle of the pointer and the zero mark line on the dial in the transformed image and the position information of the pointer.

[0022] Based on the third and fourth mapping relationships, a second mapping relationship is determined between the instrument readings and the pointer position information in the transformed image.

[0023] In some possible implementations, based on the position information of the zero mark on the dial in the target template image, a fourth mapping relationship is determined between the deflection angle of the pointer relative to the zero mark on the dial in the transformed image and the position information of the pointer, including:

[0024] Based on the position information of the pointer in the transformed image, a rectangular coordinate system is established with the starting point of the pointer as the origin, and the angle between the pointer and the vertical axis in the transformed image is determined.

[0025] Based on the position information of the zero mark line on the dial in the target template image, the angle between the vertical axis and the zero mark line on the dial in the transformed image is determined;

[0026] Based on the angle between the pointer and the vertical axis in the transformed image, and the angle between the vertical axis and the zero mark on the dial, the deflection angle between the pointer and the zero mark on the dial in the transformed image is determined.

[0027] In some possible implementations, based on the position information of the pointer in the transformed image, a Cartesian coordinate system is established with the starting point of the pointer as the origin, and the angle between the pointer and the vertical axis in the transformed image is determined, including:

[0028] Based on the position information of the pointer in the transformed image, a rectangular coordinate system is established with the starting point of the pointer as the origin. The arctangent value of the ratio of the difference between the horizontal coordinates of the starting point and the ending point of the pointer to the difference between the vertical coordinates is obtained, thereby determining the radian between the pointer and the vertical axis in the transformed image.

[0029] The angle between the pointer and the vertical axis in the transformed image is determined based on the radian between the pointer and the vertical axis in the transformed image.

[0030] In some possible implementations, based on a preset matching algorithm, feature matching is performed between the image to be tested and the template image to obtain a target template image, and a first mapping relationship between the image to be tested and the target template image is determined. This first mapping relationship characterizes the relationship between the instrument angles in the image to be tested and the target template image, including:

[0031] Based on a preset feature extraction algorithm, feature points are extracted from the image to be tested and the template image to obtain their corresponding feature points. The template image includes images of various types of instruments.

[0032] Based on a preset feature matching algorithm, feature points of the image to be tested and the template image are matched to obtain a target template image, and feature point matching pairs between the image to be tested and the target template image are determined.

[0033] Based on the feature point matching pairs, a first mapping relationship is determined between the image to be tested and the target template image. The first mapping relationship is used to characterize the relationship between the instrument angles in the image to be tested and the target template image.

[0034] In some possible implementations, based on a preset feature matching algorithm, feature points of the image to be tested and the template image are matched to obtain a target template image, and feature point matching pairs between the image to be tested and the target template image are determined, including:

[0035] Based on a preset feature matching algorithm, feature points of the image to be tested and the template image are matched to determine whether the image to be tested and the template image are successfully matched.

[0036] If the image to be tested does not match the template image, then a new image to be tested is obtained.

[0037] If the image to be tested successfully matches the template image, the target template image is obtained, and the feature point matching pair between the image to be tested and the target template image is determined.

[0038] In some possible implementations, the preset feature extraction algorithm is the SuperPoint algorithm, and the preset feature matching algorithm is the SuperGlue algorithm.

[0039] On the other hand, the present invention also provides an automatic identification device for pointer-type instruments, including: a template library unit for acquiring template images and prior information of instruments in the template images;

[0040] The feature matching unit is used to perform feature matching between the image to be tested and the template image to obtain the target template image, and to determine a first mapping relationship between the image to be tested and the target template image. The first mapping relationship is used to characterize the relationship between the instrument angles in the image to be tested and the target template image.

[0041] The mapping transformation unit is used to perform mapping transformation on the image to be tested based on the first mapping relationship, convert the image to be tested into a transformed image with the same instrument angle as the target template image, and determine the position information of the pointer in the transformed image;

[0042] The reading calculation unit is used to determine a second mapping relationship between the pointer position information and the instrument reading based on the prior information of the instrument in the target template image, and to determine the instrument reading in the transformed image based on the second mapping relationship and the pointer position information in the transformed image.

[0043] The beneficial effects of the above embodiments are as follows: The automatic identification method for pointer-type instruments provided by the present invention first obtains a template image and prior information of the instrument in the template image, then performs feature matching on the template image and the image to be tested, converts the image to be tested into a transformed image with the same angle as the instrument in the template image, detects the pointer from the transformed image, determines the pointer information of the transformed image, and then determines the instrument reading of the transformed image based on the relationship between the pointer position and the instrument reading obtained from the prior information of the template image, thereby realizing the identification of the instrument reading in the image to be tested. The instrument identification accuracy is high, and it is not easily affected by lighting or shooting angle, resulting in good identification effect. Attached Figure Description

[0044] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0045] Figure 1 A schematic flowchart of an embodiment of the automatic identification method for pointer-type instruments provided by the present invention;

[0046] Figure 2 For the present invention Figure 1 A schematic diagram of an embodiment of step S102;

[0047] Figure 3 The feature matching results provided by this invention;

[0048] Figure 4 For invention Figure 2 A schematic flowchart of an embodiment of step S202;

[0049] Figure 5 For the present invention Figure 1 A schematic flowchart of an embodiment of step S104;

[0050] Figure 6 For the present invention Figure 5 A schematic diagram of an embodiment of step S501;

[0051] Figure 7 Transformed images of multiple target Hough circles provided for this invention;

[0052] Figure 8 The image provided by this invention is a transformed image after being cropped using a target Hough circle.

[0053] Figure 9 For the present invention Figure 5 A schematic flowchart of an embodiment of step S502;

[0054] Figure 10 For the present invention Figure 1 A schematic flowchart of an embodiment of step S105;

[0055] Figure 11 For the present invention Figure 10 A schematic diagram of an embodiment of step S1001;

[0056] Figure 12 For the present invention Figure 11 A schematic diagram of an embodiment of step S1101;

[0057] Figure 13 This is a schematic diagram of an embodiment of the pointer-type instrument automatic identification device provided by the present invention. Detailed Implementation

[0058] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0059] It should be understood that the illustrative drawings are not drawn to scale. The flowcharts used in this invention illustrate operations implemented according to some embodiments of the invention. It should be understood that the operations in the flowcharts may be implemented out of order, and steps without logical contextual relationships may be reversed or performed simultaneously. Furthermore, those skilled in the art, guided by the content of this invention, may add one or more other operations to the flowcharts, or remove one or more operations from the flowcharts. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor systems and / or microcontroller systems.

[0060] The terms "first," "second," etc., used in the embodiments of this invention are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a technical feature defined with "first" or "second" may explicitly or implicitly include at least one of those features. "And / or" describes the relationship between related objects, indicating that three relationships may exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone.

[0061] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0062] This invention provides an automatic identification method for pointer-type instruments, which will be described in detail below.

[0063] Figure 1 A schematic flowchart of an embodiment of the automatic identification method for pointer-type instruments provided by the present invention is shown below. Figure 1 As shown, the automatic identification method for pointer-type instruments includes:

[0064] S101. Obtain the template image and the prior information of the instrument in the template image;

[0065] It should be noted that when acquiring template images, it is advisable to collect template images from multiple instruments. When collecting template images, the camera and the dial should be perpendicular to each other in a straight line, and the camera should be facing the instrument directly to prevent lens distortion from affecting the template image. The template images are then processed to obtain the prior information of the instrument, such as the center of rotation of the pointer, the radius of the dial, the maximum value, the minimum value, etc., which will be explained in detail later.

[0066] S102. Based on the preset matching algorithm, feature matching is performed on the image to be tested and the template image to obtain the target template image, and the first mapping relationship between the image to be tested and the target template image is determined. The first mapping relationship is used to characterize the relationship between the instrument angles in the image to be tested and the target template image.

[0067] S103. Based on the first mapping relationship, perform a mapping transformation on the image to be tested to obtain a transformed image;

[0068] S104. Based on a preset detection algorithm, determine the position information of the pointer in the transformed image;

[0069] S105. Based on the prior information of the instrument in the target template image, determine the second mapping relationship between the instrument reading and the pointer position information in the transformed image. Based on the second mapping relationship and the pointer position information in the transformed image, determine the instrument reading in the transformed image.

[0070] Compared with existing technologies, this invention first obtains a template image and prior information about the instrument in the template image. Then, it performs feature matching on the template image and the image to be tested, converting the image to be tested into a transformed image with the same instrument angle as the template image. The pointer is detected from the transformed image, and the pointer information of the transformed image is determined. Then, based on the relationship between the pointer position and the instrument reading obtained from the prior information of the template image, the instrument reading of the transformed image is determined, thus realizing the recognition of the instrument reading in the image to be tested. The instrument recognition accuracy is high, and it is not easily affected by lighting or shooting angle, resulting in good recognition effect.

[0071] Furthermore, compared to the dial scales pointed to by the pointer, which are dense and small and difficult to identify, pointer identification is easy. This invention calculates the instrument readings by identifying the pointer position, resulting in good identification performance.

[0072] To obtain a better first mapping relationship, in some embodiments, such as Figure 2 As shown, step S102 specifically includes:

[0073] S201. Based on the preset feature extraction algorithm, feature extraction is performed on the image to be tested and the template image to obtain their corresponding feature points. The template image includes images of various types of instruments.

[0074] S202. Based on the preset feature matching algorithm, feature points of the test image and the template image are matched to obtain the target template image, and feature point matching pairs between the test image and the target template image are determined.

[0075] S203. Based on feature point matching pairs, determine the first mapping relationship between the image to be tested and the target template image. The first mapping relationship is used to characterize the relationship between the instrument angles in the image to be tested and the target template image.

[0076] It should be noted that in this embodiment, the preset feature extraction algorithm is the SuperPoint algorithm, and the preset feature matching algorithm is the SuperGlue algorithm. Compared with traditional feature extraction and matching algorithms such as orb (Oriented FAST and Rotated BRIEF), sift (Scale-Invariant Feature Transform), and surf (Speeded Up Robust Features), the SuperPoint and SuperGlue algorithms can provide better homography estimation results on the HPatches dataset, which is beneficial for subsequent pointer detection. Specifically, SuperPoint is trained on the MS-COCO general image dataset using homography adaptation, and can repeatedly detect a richer set of feature points than the initial pre-adapted deep model and any other traditional corner detector. The SuperGlue architecture uses two types of attention: self-attention and cross-attention. Self-attention enhances the receptive field of local descriptors, while cross-attention enables cross-image communication, inspired by the way humans look back and forth when matching images.

[0077] The main implementation methods of attention mechanisms are as follows:

[0078]

[0079] In the formula, m ε→i It is mainly calculated using the weighted average, α ij It is the attention weight, a ij =Softmax j (q j T k j ). Here q j k j and v j It is obtained through linear feature mapping of GNN (Graph Neural Network), and the specific formula is as follows:

[0080] q j =W1 (l) x i Q +b1

[0081]

[0082] It should also be noted that, for the pointer-type instrument image recognition of this invention, in order to obtain good mapping transformation results, the SuperPoint algorithm training process is as follows: First, a dataset is constructed. The images in the dataset need to contain simple geometric shapes and have clear feature points. Using these feature points, the MagicPoint feature point detector is trained using the VGG (Visual Geometry Group) network. Then, unlabeled images are obtained through data augmentation and homography transformation (normalization) is performed. The trained detector is used to detect the transformed images. Finally, the feature points and descriptive factors are combined for training to obtain the final SuperPoint detector. The SuperGlue network training process is as follows: The SUN scene image dataset is used, which contains images with diverse scenes and changing viewpoints. The Adam (Adaptive Moment Estimation) optimizer is used to finally obtain the SuperGlue neural network model.

[0083] Based on the above feature matching algorithm, the following is obtained: Figure 3 The feature matching results shown are in Figure 3 In the image, the left side represents the target template image, and the right side represents the image to be tested. The line between the target template image and the image to be tested connects the feature point matching pairs. It should be noted that... Figure 3 This is an illustrative diagram to demonstrate the feature point matching process. In reality, the shooting angle of the instrument in the image to be tested will differ from the shooting angle of the target template image.

[0084] It should also be noted that, to eliminate the rotation error introduced by the rotation of the instrument panel, the first mapping relationship is obtained by solving the homography matrix of the original feature point matching pairs using the RANSAC algorithm (Random Sample Consensus). Then, the erroneous matching pairs are removed by iteratively solving the coordinate correspondence relationship. Based on the homography matrix obtained above, the image to be tested is transformed by first converting the two-dimensional image into three-dimensional space through matrix multiplication, and then mapping it to another two-dimensional space. A 3×3 matrix is ​​used. The first two rows of the matrix are the same as the affine matrix, which also realizes linear transformation and translation. The third row is used to realize the mapping transformation. The specific mapping transformation formula is as follows:

[0085]

[0086] In the formula, (x1, y1) are the coordinates of the target template image, (x2, y2) are the coordinates of the image to be tested, and H is the homography matrix.

[0087] Based on the above mapping and transformation rules, the image to be tested is transformed into a transformed image with the same angle as the target template image. At the same time, the approximate position of the dial is obtained through a matching algorithm, and the image is positioned and cropped to reduce the interference of the environmental area outside the dial on image recognition.

[0088] To address special situations, such as insufficient image clarity or severe occlusion, in some embodiments, such as Figure 4 As shown, step S202 specifically includes:

[0089] S401. Based on the preset feature matching algorithm, feature points of the test image and the template image are matched to determine whether the test image and the template image are successfully matched.

[0090] S402. If the image to be tested does not match the template image, then obtain the image to be tested again.

[0091] S403. If the image to be tested matches the template image successfully, the target template image is obtained, and the feature point matching pair between the image to be tested and the target template image is determined.

[0092] To more conveniently and accurately obtain the position information of the changing image pointer, in some embodiments, such as Figure 5 As shown, step S104 specifically includes:

[0093] S501. Determine the position information of the rotation center of the pointer in the transformed image;

[0094] S502. Based on the rotation center position information of the pointer in the transformed image and the preset Hough line detection algorithm, determine the position information of the pointer in the transformed image.

[0095] It should be noted that in this embodiment, the pointer is detected by the Hough transform algorithm to obtain the pointer's position information. In other embodiments, deep learning or machine learning can also be used to select a pointer template image to determine the pointer's position information.

[0096] To improve the efficiency and accuracy of pointer detection, in some embodiments, specifically when the prior information includes the pointer's rotation center position information and the dial radius, reference is made... Figure 6 Step S501 specifically includes:

[0097] S601. Based on the radius of the dial in the target template image, perform Hough circle detection on the transformed image to obtain multiple Hough circles in the dial area of ​​the transformed image, and determine the center position information of the Hough circles.

[0098] It should be noted that the Hough circle detection principle transforms the image from the original image space to a parameter space. In the parameter space, a certain parametric form that most boundary points satisfy is used as a description of the curve in the image. It accumulates the parameters by setting an accumulator, and the point corresponding to the peak value is the required information. The formula for the Hough circle detection principle is as follows:

[0099] (xa) 2 +(yb) 2 =r 2

[0100] The circle corresponds to a point in the ABR three-dimensional parameter space, where a, b, and r represent the x-coordinate, y-coordinate, and radius of the circle's center, respectively. Similarly, a point in the xy image corresponds to a conical surface in the ABR space. If there are enough points on the edge of the circle, their corresponding surfaces will intersect at the point (a, b, r). Therefore, when a non-zero point is found in the image, it needs to be accumulated in the ABR space accumulator. Finally, each peak in the ABR space represents a circle.

[0101] It should also be noted that before using the Hough circle detection algorithm to detect the transformed image, the image is first processed using a Gaussian blur function; when performing Hough circle detection, the radius difference between multiple Hough circles and the dial in the target template image is within a certain range, so as to reduce the interference of other irrelevant circles in the transformed image.

[0102] S602. Determine the deviation between the center of the Hough circle and the rotation center of the pointer in the target template image, and obtain the target Hough circle with the deviation within the preset range.

[0103] It should be noted that even if the transformed image after mapping has the same instrument angle and size as the target template image, it is not an absolute identical match; there will still be errors. Specifically, the rotation center position of the pointer in the transformed image and the target template image will have some discrepancy. Compared to directly using the rotation center of the pointer in the target template image as the rotation center of the pointer in the transformed image, constraining the center of the target Hough circle can yield a more accurate rotation center position for the pointer in the transformed image. This reduces the impact of excessive center deviation on subsequent pointer positioning and reading. Here, center constraint means that the deviation value is within a preset range. The final transformed image marked with multiple target Hough circles is shown below. Figure 7 As shown, Figure 7 In the dial, the white coil around the dial is the target Hough circle, and the white dot in the center of the dial is the center of the corresponding target Hough circle.

[0104] S603. Crop the transformed image based on the target Hough circle, and use the center of the target Hough circle as the rotation center of the pointer in the transformed image.

[0105] It should be noted that there are multiple target Hough circles that meet the center constraint. These target Hough circles are arranged sequentially according to their radius. Pointer detection begins with the target Hough circle with the largest radius. If no pointer is extracted, the next Hough circle is used for pointer detection until a pointer is extracted. Before pointer detection, the circumscribed square of each target Hough circle needs to be found based on its position and radius, and then clipped to obtain the desired result. Figure 8 The image shown is a transformed image after being cropped using the target Hough circle. This further reduces interference from irrelevant information outside the target Hough circle in the transformed image, which is beneficial for subsequent pointer extraction.

[0106] Based on the transformed image after cropping using the target Hough circle, the position information of the pointer in step S502 is determined, referring to... Figure 9 Step S502 specifically includes:

[0107] S901. Convert the transformed image to a grayscale image and use adaptive thresholding to segment the grayscale. Figure 2 Value-based;

[0108] It should be noted that converting the transformed image to a grayscale image can reduce the number of parameters and make it easier to find the pointer. In this embodiment, the cv2.cvtColor function is used to convert the transformed image to a grayscale image.

[0109] S902, Grayscale-based Figure 2 The value-enhanced results are used to perform edge detection on the grayscale image;

[0110] It should be noted that in this embodiment, the cv2.Canny function is used for edge detection. The basic principle of this function is to detect edges by calculating the gradient value of each pixel in the image. Before calculating the gradient value of the pixel, the grayscale image needs to be Gaussian filtered to smooth the image and remove noise. The specific process of calculating the grayscale value of the pixel is as follows: first, provide the convolution operator P on the x-axis and y-axis. x P y Calculate the magnitude g of each point in the image on the x-axis and y-axis respectively. x g y The formula is as follows:

[0111] g x = f(x,y)×P x

[0112] g y = f(x,y)×P y

[0113] In the formula, f(x,y) represents the pixel of the image, and the gradient can be understood as the direction of the region with the greatest change in magnitude around the same pixel. Based on the principle of force decomposition and trigonometric function relationships, the gradient magnitude corresponding to each point can be calculated. With direction The calculation formula is as follows:

[0114]

[0115]

[0116] S903. Use the Hough line detection algorithm to extract pointers from the grayscale image after edge detection and determine the position information of the pointers.

[0117] It should be noted that the specific process of pointer extraction using the Hough line detection algorithm is as follows: A threshold for line detection is set, varying from 120 to 30. At each threshold, the `cv2.HoughLinesP` function is used for line detection. The basic principle is to utilize the duality of points and lines, transforming all points on a line in the image coordinate system to the parameter coordinate system through a parametric expression, forming N lines. Since the slope and intercept values ​​of all points on a line in image space are the same, these lines converge on the same point. Then, an accumulator is used to accumulate the values ​​at each point, finding the peak value in the accumulator to extract the line. This transforms the detection problem from a global problem in the original image space to a local problem in the parameter space.

[0118] During the calculation, the equation of a straight line in a rectangular coordinate system is:

[0119] y = kx + b

[0120] In the formula, k is the slope, b is the intercept, and (x, y) are the coordinates of any point on the line. When the line is perpendicular, the range of values ​​for the slope and intercept is infinite. When the spatial parameter used is ρ-θ, it is usually called Hough space. Then, a line in the rectangular coordinate system is transformed to the polar coordinate system using the parametric equation expression. The polar coordinate equation is:

[0121] ρ = x·cosθ + y·sinθ

[0122] ρ represents the distance between the pole O and point P. θ represents the angle between the ray and the polar axis in the counterclockwise direction, also known as the polar angle. All points on a straight line in a rectangular coordinate system can be transformed into polar coordinates using the polar coordinate equation to obtain N sine curves, all of which converge at the same point.

[0123] If no straight line is detected at the current threshold, the process returns to the next threshold for detection. If a straight line is detected, the next filtering step is performed. Connect all points of a straight line and extend it to find a straight line whose extension passes near the center of rotation of the pointer, thus determining the pointer's position information.

[0124] To determine the relationship between the pointer's position information and the instrument readings in a transformed image, in some embodiments, when the prior information includes the maximum and minimum values ​​of the dial, the angle between the minimum and maximum values, and the position information of the zero mark, reference is made. Figure 10 In step S105, based on the prior information of the instrument in the target template image, a second mapping relationship is determined between the instrument readings and the pointer position information in the transformed image, specifically including:

[0125] S1001. Based on the maximum and minimum values ​​of the dial in the target template image, and the angle between the minimum and maximum values, determine the third mapping relationship between the instrument readings in the transformed image and the deflection angle between the pointer and the zero mark of the dial.

[0126] S1002. Based on the position information of the zero mark line of the dial in the target template image, determine the fourth mapping relationship between the deflection angle of the pointer and the zero mark line of the dial in the transformed image and the position information of the pointer.

[0127] It should be noted that the specific formula for the fourth mapping relationship is:

[0128]

[0129] In the formula, value is the instrument reading. max It is the maximum value on the dial. max θ is the minimum value on the dial, θ is the angle between the minimum and maximum values ​​on the dial, and Δθ is the deflection angle of the pointer relative to the zero mark on the dial.

[0130] S1003. Based on the third and fourth mapping relationships, determine the second mapping relationship between the instrument readings and the pointer position information in the transformed image.

[0131] It should be noted that the maximum and minimum values ​​of the dial in the template image can be obtained directly based on the type of instrument. However, obtaining the angle between the minimum and maximum values, as well as the position information of the zero mark on the dial, requires processing the template image. This involves drawing straight lines representing the maximum and minimum values ​​of the dial, as well as the zero mark, starting from the center of rotation.

[0132] In some embodiments, refer to Figure 11 Step S1001 specifically includes:

[0133] S1101. Based on the position information of the pointer in the transformed image, establish a rectangular coordinate system with the starting point of the pointer as the origin, and determine the angle between the pointer and the vertical axis in the transformed image.

[0134] S1102. Based on the position information of the zero mark line of the dial in the target template image, determine the angle between the vertical axis and the zero mark line of the dial in the transformed image;

[0135] S1103. Based on the angle between the pointer and the vertical axis in the transformed image, and the angle between the vertical axis and the zero mark of the dial, determine the deflection angle between the pointer and the zero mark of the dial in the transformed image.

[0136] It should be noted that the deflection angle of the pointer relative to the zero mark on the dial is equal to the difference between the angle between the pointer and the vertical axis and the angle between the vertical axis and the zero mark on the dial. The method for determining the angle between the vertical axis and the zero mark on the dial in the transformed image is similar to the method for determining the angle between the pointer and the vertical axis in the transformed image.

[0137] In some embodiments, refer to Figure 12 Step S1101 specifically includes:

[0138] S1201. Based on the position information of the pointer in the transformed image, a rectangular coordinate system is established with the starting point of the pointer as the origin. The arctangent value of the ratio of the difference between the horizontal coordinates of the starting point and the ending point of the pointer to the difference between the vertical coordinates is obtained, thereby determining the radian between the pointer and the vertical axis in the transformed image.

[0139] S1202. Based on the radian between the pointer and the vertical axis in the transformed image, determine the angle between the pointer and the vertical axis in the transformed image.

[0140] It should be noted that the formulas for calculating the radian distance between the pointer and the vertical axis, and the angle between the pointer and the vertical axis, are as follows:

[0141]

[0142]

[0143] In the formula, (x1, y1) and (x2, y2) are the starting point and ending point of the pointer, respectively. The starting point of the pointer is also the center of rotation of the pointer. rsd is the radian of the pointer relative to the vertical axis, and rotation is the angle of the pointer relative to the vertical axis.

[0144] To better implement the automatic identification method for pointer-type instruments in the embodiments of the present invention, based on the automatic identification method for pointer-type instruments, correspondingly, as follows: Figure 13 As shown, this embodiment of the invention also provides an automatic identification device 1300 for pointer-type instruments, comprising:

[0145] Template library unit 1301 is used to obtain template images and prior information of instruments in the template images;

[0146] The feature matching unit 1302 is used to perform feature matching between the image to be tested and the template image to obtain the target template image, and to determine the first mapping relationship between the image to be tested and the target template image. The first mapping relationship is used to characterize the relationship between the instrument angles in the image to be tested and the target template image.

[0147] The mapping transformation unit 1303 is used to perform mapping transformation on the image to be tested based on the first mapping relationship, convert the image to be tested into a transformed image with the same instrument angle as the target template image, and determine the position information of the pointer in the transformed image;

[0148] The reading calculation unit 1304 is used to determine the second mapping relationship between the pointer position information and the instrument reading based on the prior information of the instrument in the target template image, and to transform the reading in the image based on the second mapping relationship.

[0149] The position information of the needle is used to determine the instrument readings in the transformed image.

[0150] The pointer instrument automatic identification device provided in the above embodiments can realize the technical solutions described in the above pointer instrument automatic identification method embodiments. The specific implementation principles of each unit can be found in the corresponding content in the above pointer instrument automatic identification method embodiments, and will not be repeated here.

[0151] The above provides a detailed description of an automatic identification method for pointer-type instruments provided by the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method of automatic identification of an analog instrument, characterized in that, The method comprises the following steps: obtaining a template picture and prior information of a meter in the template picture; performing feature matching on a to-be-tested picture and the template picture based on a preset matching algorithm to obtain a target template picture, and determining a first mapping relationship between the to-be-tested picture and the target template picture, the first mapping relationship being used to represent a relationship between angles of meters in the to-be-tested picture and the target template picture; performing mapping transformation on the to-be-tested picture based on the first mapping relationship to obtain a transformed picture; determining position information of a pointer in the transformed picture based on a preset detection algorithm; determining a second mapping relationship between a meter scale and the position information of the pointer in the transformed picture based on the prior information of the meter in the target template picture, and determining the meter scale in the transformed picture based on the second mapping relationship and the position information of the pointer in the transformed picture; when the prior information comprises a maximum value, a minimum value, an angle between the minimum value and the maximum value of a dial, and position information of a zero scale line, the second mapping relationship between the meter scale and the position information of the pointer in the transformed picture is determined based on the prior information of the meter in the target template picture, and the meter scale in the transformed picture is determined based on the second mapping relationship and the position information of the pointer in the transformed picture; determining the second mapping relationship between the meter scale and the position information of the pointer in the transformed picture based on the prior information of the meter in the target template picture comprises: determining a third mapping relationship between the meter scale, a deflection angle of the pointer and the zero scale line of the dial in the transformed picture based on the maximum value, the minimum value and the angle between the minimum value and the maximum value of the dial in the target template picture; determining a fourth mapping relationship between the deflection angle of the pointer and the zero scale line of the dial and the position information of the pointer in the transformed picture based on the position information of the zero scale line of the dial in the target template picture comprises: establishing a rectangular coordinate system with a starting endpoint of the pointer as a coordinate origin based on the position information of the pointer in the transformed picture, determining an angle between the pointer and a vertical axis in the transformed picture, determining an angle between the vertical axis and the zero scale line of the dial in the transformed picture based on the position information of the zero scale line of the dial in the target template picture, determining an arc radian between the pointer and the vertical axis in the transformed picture by calculating an inverse tangent value of a ratio of a horizontal coordinate difference value and a vertical coordinate difference value of the starting endpoint and an ending endpoint of the pointer based on the position information of the pointer in the transformed picture, determining the deflection angle of the pointer and the zero scale line of the dial in the transformed picture based on the arc radian between the pointer and the vertical axis in the transformed picture, and determining the deflection angle of the pointer and the zero scale line of the dial in the transformed picture based on the angle between the pointer and the vertical axis in the transformed picture and the angle between the vertical axis and the zero scale line of the dial; determining the second mapping relationship between the meter scale and the position information of the pointer in the transformed picture based on the third mapping relationship and the fourth mapping relationship.

2. The automatic identification method of a pointer-type instrument according to claim 1, characterized in that, determining the position information of the pointer in the transformed picture based on the preset detection algorithm comprises: determining position information of a rotation center of the pointer in the transformed picture; determining the position information of the pointer in the transformed picture based on the position information of the rotation center of the pointer in the transformed picture and a preset Hough straight line detection algorithm.

3. The automatic identification method of a pointer-type instrument according to claim 2, characterized in that, when the prior information comprises position information of a rotation center of the pointer and a radius of the dial, the position information of the rotation center of the pointer in the transformed picture is determined by: perform Hough circle detection on the transformed picture based on a radius of the dial in the target template picture to obtain a plurality of Hough circles of the dial region in the transformed picture, and determine a center position information of the Hough circles; determine a deviation value of the center of the Hough circle from a rotating center position of the pointer in the target template picture to obtain a target Hough circle with the deviation value within a preset range; perform cropping on the transformed picture based on the target Hough circle, and take the center of the target Hough circle as the rotating center of the pointer in the transformed picture.

4. The automatic identification method of the pointer instrument according to claim 1, wherein based on a preset matching algorithm, performing feature matching on the to-be-tested picture and the template picture to obtain a target template picture, and determining a first mapping relationship between the to-be-tested picture and the target template picture, the first mapping relationship being used to represent a relationship between instrument angles in the to-be-tested picture and the target template picture, including: based on a preset feature extraction algorithm, performing feature extraction on the to-be-tested picture and the template picture to obtain corresponding feature points of the to-be-tested picture and the template picture, the template picture including pictures of multiple types of instruments; based on a preset feature matching algorithm, performing feature matching on the feature points of the to-be-tested picture and the template picture to obtain a target template picture, and determining a feature point matching pair between the to-be-tested picture and the target template picture; based on the feature point matching pair, determining a first mapping relationship between the to-be-tested picture and the target template picture, the first mapping relationship being used to represent a relationship between instrument angles in the to-be-tested picture and the target template picture.

5. The automatic identification method of a pointer-type instrument according to claim 4, characterized in that, based on a preset feature matching algorithm, performing feature matching on the feature points of the to-be-tested picture and the template picture to obtain a target template picture, and determining a feature point matching pair between the to-be-tested picture and the target template picture, including: based on a preset feature matching algorithm, performing feature matching on the feature points of the to-be-tested picture and the template picture, and determining a feature point matching pair between the to-be-tested picture and the target template picture. the preset feature extraction algorithm is a SuperPoint algorithm, and the preset feature matching algorithm is a SuperGlue algorithm. including:

6. The automatic identification method of a pointer-type instrument according to claim 4, characterized by, a template library unit configured to obtain a template picture and prior information of an instrument in the template picture; 7. An automatic pointer instrument identification device, characterized by, a feature matching unit configured to perform feature matching on a to-be-tested picture and the template picture to obtain a target template picture, and determine a first mapping relationship between the to-be-tested picture and the target template picture, the first mapping relationship being used to represent a relationship between instrument angles in the to-be-tested picture and the target template picture; a mapping and transforming unit configured to perform mapping and transforming on the to-be-tested picture based on the first mapping relationship, convert the to-be-tested picture into a transformed picture with the same instrument angle as that in the target template picture, and determine position information of a pointer in the transformed picture; ​ ​ The number calculation unit is configured to determine a second mapping relationship between the position information of the pointer and the instrument number in the target template picture based on prior information of the instrument in the target template picture, and determine the instrument number in the transformed picture based on the second mapping relationship and the position information of the pointer in the transformed picture. When the prior information includes a maximum value, a minimum value, an angle between the minimum value and the maximum value, and position information of a zero scale line of the dial; The number calculation unit is configured to determine a second mapping relationship between the position information of the pointer and the instrument number in the target template picture based on prior information of the instrument in the target template picture, and determine the instrument number in the transformed picture based on the second mapping relationship and the position information of the pointer in the transformed picture. The number calculation unit is configured to determine a third mapping relationship between the instrument number and a deflection angle of the pointer and the zero scale line of the dial in the transformed picture based on the maximum value, the minimum value, and the angle between the minimum value and the maximum value of the dial in the target template picture. The number calculation unit is configured to determine a fourth mapping relationship between the deflection angle of the pointer and the zero scale line of the dial and the position information of the pointer in the transformed picture based on the position information of the zero scale line of the dial in the target template picture, including: establishing a rectangular coordinate system with a starting endpoint of the pointer as a coordinate origin based on the position information of the pointer in the transformed picture, determining an angle between the pointer and a vertical axis in the transformed picture; determining an angle between the vertical axis and the zero scale line of the dial in the transformed picture based on the position information of the zero scale line of the dial in the target template picture; calculating an inverse tangent value of a ratio of a horizontal coordinate difference and a vertical coordinate difference between the starting endpoint and an ending endpoint of the pointer to determine an radian between the pointer and the vertical axis in the transformed picture based on the rectangular coordinate system with the starting endpoint of the pointer as the coordinate origin and the position information of the pointer in the transformed picture; determining the deflection angle of the pointer and the zero scale line of the dial in the transformed picture based on the radian between the pointer and the vertical axis in the transformed picture; and determining the deflection angle of the pointer and the zero scale line of the dial in the transformed picture based on the angle between the pointer and the vertical axis in the transformed picture and the angle between the vertical axis and the zero scale line of the dial in the transformed picture. The number calculation unit is configured to determine a second mapping relationship between the position information of the pointer and the instrument number in the target template picture based on prior information of the instrument in the target template picture, and determine the instrument number in the transformed picture based on the second mapping relationship and the position information of the pointer in the transformed picture.

Images