A method for identifying readings of square pointer instruments

Abstract

The present invention relates to a method for identifying the reading of a square pointer instrument, comprising the following steps: (1) extracting a rectangular frame image of the instrument; (2) correcting the image; (3) performing binary thinning processing on the corrected image; (4) obtaining the center of the circle around which the pointer revolves, the center of mass of the upper left corner A, and the midpoint of the pointer center line segment, connecting the center of mass of the upper left corner A and the center of the circle around which the pointer revolves as a baseline, and connecting the midpoint of the pointer center line segment and the center of the circle around which the pointer revolves as a needle line; (5) obtaining the angle between the baseline and the needle line, and fitting a linear equation reflecting the reading value based on the angle; and (6) adjusting the fitting equation using error distribution. The present invention solves the problem that the center of the circle where the scale lines are located cannot be accurately found when the square instrument has no circular texture, thereby improving precision and accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method for recognizing readings of square pointer type instruments. Background Art

[0002] When identifying pointer instruments in the power industry or other industries, the readings are generally performed on circular, uniformly scaled pointer instruments. Image matching or SSD methods are first used to determine the instrument's target location. Next, image registration is used to correct the instrument image, although some corrections are not performed. Hough transforms are then used to detect circles or ellipse fitting to determine the instrument's outline and the center of the circle around the point. Hough line detection is then used to detect the instrument's pointer or scale lines. Manual methods are used to determine the starting and ending positions, and the reading is determined based on the angle. Digital recognition is also used to determine the large scale value. Finally, the instrument is expanded into a straight line, and the distance is measured to determine the instrument's reading.

[0003] Image matching of instrument panels is extremely time-consuming, making it unsuitable for real-world applications. The SSD algorithm is no longer the best detection method in the field of image deep learning. Image registration requires corner point detection, which is extremely time-consuming and impractical for real-world applications. Furthermore, without image correction, subsequent image processing is severely hindered, resulting in poor adaptability, inability to recognize instrument panels at different angles, and ultimately poor accuracy. For square instruments, the outer contour is not circular, and the instrument panel lacks circular textures (though some do). For instruments without circular contours or textures, the established Hough circle detection algorithm cannot be used. Hough circle detection is also time-consuming, and the centroid found using ellipse fitting suffers from significant errors. Direct Hough line detection for pointer detection also requires filtering out many irrelevant lines, making the pointer easily affected by noise. Deep learning methods for recognizing instrument panel numbers require extensive training with specific data, which is time-consuming and labor-intensive, and can lead to misidentifications. These misidentifications can lead to significant reading errors, making them unsatisfactory. Existing methods fail to address the significant measurement errors and circle centroids, resulting in limited accuracy improvements. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to overcome the shortcomings of the existing technology and provide a new method for reading and identifying square pointer instruments, which solves the problem that square instruments have no circular texture and the center of the circle where the scale lines are located cannot be accurately found, thereby improving precision and accuracy.

[0005] In order to solve the above technical problems, a technical solution adopted by the present invention is:

[0006] A method for identifying readings of a square pointer instrument comprises the following steps:

[0007] (1) Read the collected original image and extract the instrument rectangular frame image;

[0008] (2) Correcting the instrument rectangular frame image in step (1);

[0009] (3) processing the image corrected in step (2) to obtain a binary refined image;

[0010] (4) Obtain the center of the circle around which the pointer revolves, the center of mass of the upper left corner A and the midpoint of the pointer center line segment, connect the center of mass of the upper left corner A and the center of the circle around which the pointer revolves as the baseline, and connect the midpoint of the pointer center line segment and the center of the circle around which the pointer revolves as the pointer line;

[0011] (5) obtaining the angle between the baseline and the needle line in step (4), and fitting a linear equation reflecting the reading value based on the angle;

[0012] (6) Using the error distribution, adjust the fitting equation;

[0013] In step (2), the image is corrected using perspective transformation. The specific process is: binarize the image on the brightness V component, use the farthest distance method to find the positions of the four points, obtain a transformation matrix, and then obtain the image after perspective transformation, that is, the corrected image.

[0014] Preferably, step (1) uses the deep learning algorithm YOLOV3 to extract the instrument rectangular frame image.

[0015] Preferably, the specific process of step (4) is: first obtain the area and centroid of each pixel block, filter according to the area size to obtain the scale centroid, then perform least squares fitting to obtain the center of the circle around which the pointer revolves, and then filter according to the area to obtain the centroid of the upper left corner A, and obtain the midpoint of the pointer center line segment through binarization, refinement, Hough transform detection of straight line segments, and straight line segment screening.

[0016] Due to the adoption of the above technical solution, the present invention has the following advantages compared with the prior art:

[0017] 1. The YOLOV3 instrument target detection and positioning speed of the present invention is faster, more accurate, and has relatively high comprehensive performance;

[0018] 2. The present invention adopts the V component binary and the farthest distance method to find four points, and perspective transformation to correct the image, avoiding feature extraction, greatly improving the calculation speed, and correcting the distortion. It can adapt to a large number of instrument pictures at different angles, with a tilt of less than about 50 degrees, and has good adaptability to the environment.

[0019] 3. The present invention uses the centroid of the scale line to fit the circle center, and then uses the average center of a large number of samples to average the detected circle center, thereby improving the detection accuracy;

[0020] 4. The reference line of the present invention is determined by the stable centroid of the dial "A" and the relatively accurate center of the circle. At the same time, the reference line is also in the common position of the pointer, so the final reading accuracy is high;

[0021] 5. Due to the existence of the baseline, the present invention avoids the difficult task of determining the initial position or the final position of the pointer;

[0022] 6. The present invention uses error distribution to adjust the fitting equation, further improving accuracy;

[0023] 7. The instrument recognition method of the present invention has a speed of less than 1 second, an average accuracy error of about one-quarter of the smallest scale unit, is adaptable to shooting and sampling at various angles, and will not have false detection in normal environments, fully meeting actual needs;

[0024] 8. The present invention makes instrument inspection more efficient and intelligent, avoids fatigue caused by long-term manual inspection, and reduces enterprise labor costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 is the original image collected;

[0026] Figure 2 is the extracted instrument rectangular frame image;

[0027] Figure 3 For Figure 2 Schematic diagram after binarization on the lightness V component;

[0028] Figure 4 for Figure 3 Schematic diagram of the instrument area retained after the opening operation;

[0029] Figure 5 For attachment Figure 4 Schematic diagram of the outline;

[0030] Figure 6 A schematic diagram showing the centroid and the first point;

[0031] Figure 7 A schematic diagram showing the second point;

[0032] Figure 8 A schematic diagram showing the third point;

[0033] Figure 9 Schematic diagram of the four points found but without a confirmed position;

[0034] Figure 10 is the corrected image;

[0035] Figure 11 This is the effect diagram of fitting the center of the circle with the scale centroid;

[0036] Figure 12 A schematic diagram showing the midpoint of the pointer centerline segment and the center of mass of A. DETAILED DESCRIPTION

[0037] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0038] The present invention provides a method for identifying readings of a square pointer instrument, which specifically comprises the following steps:

[0039] (1) Read the collected original image and extract the instrument rectangular frame image;

[0040] The present invention uses the deep learning algorithm YOLOV3 to identify the instrument dial target. The final result of the algorithm is to frame the position of the instrument dial in the image. The coordinates of the upper left corner of the rectangular frame and the length and width of the rectangular frame are calculated by the YOLOV3 algorithm. The algorithm requires a large number of on-site photos taken at different angles, and uses tools to manually select and mark them. The marked files are obtained and sent to the constructed network for training. After the training is completed, the collected original images are sent to the network for detection, and the position of the instrument rectangular frame is automatically obtained, as shown in the attached figure. Figure 1 and attached Figure 2 shown.

[0041] (2) Correcting the instrument rectangular frame image in step (1);

[0042] Correction requires the use of perspective transformation in image processing technology, and perspective transformation requires finding at least four corresponding points. The difficulty lies in finding these four points, which is illustrated below.

[0043] As attached Figure 3 As shown in the figure, the extracted instrument rectangular frame image is binarized on the brightness V component, and then the small white blocks are removed. The instrument area retained after the opening operation is shown in the attached figure. Figure 4 Next, attach Figure 5 It is attached Figure 4 After contour discovery, the contour is presented. The contour is essentially an ordered list of pixel points. The average of these pixel points is calculated to obtain the contour centroid. Then, the point farthest from the centroid in the contour point list is obtained, which is the first point obtained. Figure 6 In the figure, the center point represents the centroid, and the point in the upper right corner represents the first point obtained. Then find the point farthest from the first point in the contour point list, that is, Figure 7 The point in the lower left corner of the contour is used as the second point. After obtaining the first two points, we can find the midpoint of these two points, and then find the normal line of the midpoint. The normal line will intersect at a certain point of the contour, and then find the point in the contour point list that is farthest from this intersection as the third point. Figure 8The point in the upper left corner is the intersection of the normal line and the contour, and the turquoise point in the lower right corner is the third point. After obtaining the third point, find the point in the contour point list farthest from the third point and use it as the fourth point.

[0044] At this point, although the four points have been found, the program still does not know which of the four points is the upper left corner, which is the upper right corner, which is the lower left corner, and which is the lower right corner. The program does not know the direction of the points because it has been looking for the farthest point. At the same time, it does not distinguish whether the point in the lower right corner is the lower corner point or the upper corner point. These uncertainties cannot be used as the basis for the corresponding points of the transformed image, so it is necessary to determine the position of these points. Figure 9 Shown are four points that were found but whose positions were not determined.

[0045] Next, determine the positions of the four points. First, add the row and column values of the four points to obtain four sum values. The point with the largest sum value is the point in the lower right corner, and the point with the smallest sum value is the point in the upper left corner. Among the remaining two points, the one with a small column value and a large row value is the point in the lower left corner, and the other is the point in the upper right corner. In this way, the positions of the four points, except for the upper or lower one in the lower right corner, are all determined. As for whether the lower right corner is upper or lower, it can be judged based on the slope of the straight line determined by the lower two points and the slope of the straight line determined by the upper two points among the four points. If the slopes of the two straight lines are close, it means that the point in the lower right corner is the lower point. If the slope difference is large, it means that the point in the lower right corner is the upper point. In this way, the positions of the four points are determined.

[0046] With the positions of the four points, a pixel matrix can be given accordingly. The upper left corner of the matrix corresponds to the upper left corner of the original image, the upper right corner corresponds to the upper right corner of the original image, and the lower left corner is the same. For the lower right corner, the corresponding point position can be obtained according to whether it is close to the top or the bottom, and then according to the proportion of the instrument itself. In this way, the original image Figure 4 By corresponding the four points of a given pixel matrix, we can obtain the transformation matrix required for perspective transformation. According to the transformation matrix, we can obtain the image after perspective transformation, that is, the corrected image, as shown in the attached figure. Figure 10 shown.

[0047] (3) processing the image corrected in step (2) to obtain a binary refined image;

[0048] The image processing process of the present invention includes but is not limited to binarization and image thinning.

[0049] (4) Obtain the center of the circle around which the pointer revolves, the center of mass of the upper left corner A and the midpoint of the pointer center line segment, connect the center of mass of the upper left corner A and the center of the circle around which the pointer revolves as the baseline, and connect the midpoint of the pointer center line segment and the center of the circle around which the pointer revolves as the pointer line;

[0050] On the binary thinned image obtained in step (3), the area and centroid of each pixel block are obtained (calculating the area and centroid is a mature technology). According to the size of the area, filtering is performed to obtain the centroid of the scale. After obtaining the centroid of the scale, the least squares fitting can be performed to obtain the center point of the circle around which the pointer revolves, as shown in the attached figure. Figure 11 Then, we can filter the area to get the centroid of the upper left corner A. In addition, we can detect the straight line segment through binarization, thinning, Hough transform, and filter the straight line segment to get the midpoint of the pointer center segment, as shown in the attached figure. Figure 12 As shown, this method improves the final reading accuracy better than area screening. Direct Hough transform detection of lines results in high noise and poor results. Finally, connect the centroid of the upper left corner A and the center of the circle around which the pointer orbits as the baseline. Also, connect the midpoint of the pointer centerline segment and the center of the circle around which the pointer orbits as the needle line.

[0051] (5) obtaining the angle between the baseline and the needle line in step (4), and fitting a linear equation reflecting the reading value based on the angle;

[0052] With the baseline and pointer line, we can obtain the angle between the two lines. Based on the slope, we can determine whether the pointer line is below or above the baseline. Below the baseline is negative, and above the baseline is positive. Then, within the uniform range of 100 to 800, we can fit a linear equation reflecting the reading value based on the angle. For this instrument, if the reading is uneven below 100, a warning of less than 100 will be directly output. The uneven scale accounts for a small proportion, and in the uncommon small reading range, the accuracy of the instrument itself is not very accurate in this range. For readings above 800, an over-range warning will be output.

[0053] (6) Using the error distribution, adjust the fitting equation;

[0054] Collect a certain number of images with evenly distributed scales, perform calibration readings, and obtain a fitted linear equation. Then, test a large number of samples to obtain the error between the actual value and the measured value. These errors will form a distribution. Based on this error distribution, the fitted linear equation can be offset by a certain amount to reduce the error, which will improve the measurement accuracy of the system. Alternatively, based on a large number of samples, the average value of the center of the circle where the pointer orbits after correction can be obtained. This average value can then be used to average the individual center points of each measurement, which can also improve accuracy.

[0055] The present invention is mainly used in AGV power inspection. The pan-tilt camera on the AGV is used to take pictures of the instruments at appropriate positions, and then the pictures are transmitted to the host computer. The host computer software calls the underlying instrument reading recognition method to detect the electrical instrument readings, and the output reading results are fed back to the host computer interface for display. The host computer software can automatically perform threshold judgment, alarm and other operations.

[0056] The present invention has been described in detail above. The examples are intended only to facilitate understanding of the method and core concept of the present invention. Their purpose is to enable those skilled in the art to understand the contents of the present invention and implement them accordingly. They are not intended to limit the scope of protection of the present invention. Any equivalent changes or modifications made in accordance with the spirit and essence of the present invention are intended to be covered by the scope of protection of the present invention.

Claims

1. A method for identifying readings of a square pointer instrument, characterized in that: The following steps are involved: (1) Read the acquired original image and extract the instrument rectangular frame image; (2) Correcting the instrument rectangular frame image described in step (1); (3) Processing the image corrected in step (2) to obtain a binary refined image; (4) Obtain the center of the circle around which the pointer revolves, the center of mass of the upper left corner A and the midpoint of the pointer center line segment, connect the center of mass of the upper left corner A and the center of the circle around which the pointer revolves as the baseline, and connect the midpoint of the pointer center line segment and the center of the circle around which the pointer revolves as the pointer line; (5) obtaining the angle between the baseline and the needle line in step (4), and fitting a linear equation reflecting the reading value based on the angle; (6) Using error distribution, adjust the fitting equation; In step (2), the image is corrected using perspective transformation, and the specific process is as follows: Binarize the extracted instrument rectangular frame image on the brightness V component, remove small white blocks, and open the instrument area retained after the operation; After contour discovery, the contour is presented as an ordered list of pixels. The positions of four points are found using the farthest distance method: the pixel points are averaged to obtain the contour centroid, and the point in the contour point list farthest from the centroid is the first point found. Then find the point in the contour point list that is farthest from the first point as the second point; After obtaining the first two points, find the midpoint of these two points, and then find the normal line of the midpoint. The normal line will intersect at a certain point of the contour, which is defined as the intersection point; Then find the point in the contour point list that is farthest from this intersection point as the third point; After obtaining the third point, find the point in the contour point list that is farthest from the third point and use it as the fourth point. Next, determine the positions of the four points: First, add the row and column values of the four points to obtain the four sum values. The point with the largest sum value is the point in the lower right corner, and the point with the smallest sum value is the point in the upper left corner. Among the remaining two points, the one with a smaller column value and a larger row value is the point in the lower left corner, and the other is the point in the upper right corner. In this way, the positions of the four points are determined, except for the lower right corner being closer to the top or bottom. As for whether the lower right corner is closer to the top or bottom, it can be determined based on the slopes of the straight lines determined by the lower two points and the upper two points. If the slopes of the two straight lines are close, it means that the point in the lower right corner is the lower point. If the slopes are very different, it means that the point in the lower right corner is the upper point. In this way, the positions of the four points are determined. With the positions of the four points, a corresponding pixel matrix is given: the upper left corner of the matrix corresponds to the upper left corner of the original image, the upper right corner corresponds to the upper right corner of the original image, and the same applies to the lower left corner. As for the lower right corner, the corresponding point position can be obtained according to whether it is close to the top or the bottom, and then according to the proportion of the instrument itself. In this way, the four points of the original image correspond to the four points of the given pixel matrix, and the transformation matrix required for perspective transformation can be obtained.

2. The method for identifying readings of a square pointer instrument according to claim 1, characterized in that: The step (1) uses the deep learning algorithm YOLOV3 to extract the instrument rectangular frame image.

3. The method for identifying readings of a square pointer instrument according to claim 1, characterized in that: The specific process of step (4) is as follows: first, the area and centroid of each pixel block are obtained, and filtering is performed according to the area size to obtain the scale centroid. Then, the least squares fitting is performed to obtain the center of the circle around which the pointer revolves. Then, the centroid of the upper left corner A is obtained by filtering according to the area. The midpoint of the pointer center segment is obtained by binarization, thinning, Hough transform detection of straight line segments, and straight line segment screening.

Images