A method for fitting a pointer of a circular pointer instrument and recognizing an instrument reading

Abstract

The present application relates to a kind of circular pointer type instrument pointer fitting and instrument reading identification method, belong to image recognition field.The present application simultaneously divides the pixel area of scale value and pointer in instrument panel using semantic segmentation method;Using the method of image classification to identify scale value, determine the rotation center of pointer;The skeleton thinning algorithm and least square method are used to the pointer pixel area generated by segmentation to fit the straight line equation of pointer, determine the direction of pointer;Using multiple scale values weighted calculation pointer reading.The present application can automatically fit the straight line equation of pointer, consider multiple scale values to calculate instrument reading, it is advantageous to improve the precision of pointer instrument reading identification.

Description

Technical Field

[0001] This invention belongs to the field of image recognition, specifically relating to a method for fitting the pointer of a circular pointer instrument and recognizing instrument readings. Background Technology

[0002] Pointer instruments are widely used in chemical testing, aerospace, rail transportation, shipping, substations, and other fields. They are widely adopted due to their advantages such as convenient reading, wide application range, long lifespan, minimal environmental impact, high accuracy, and simple structure. However, traditional methods of acquiring instrument readings require significant manual labor, involving periodic and irregular field data collection, and cannot provide real-time readings. The tedious and demanding workload can lead to fatigue, resulting in inaccurate readings and subjective interpretations. Furthermore, manual reading is extremely dangerous in environments involving high altitudes, high radiation, high temperatures, and high pressures. Therefore, automating pointer instrument readings is of great significance for the development of intelligent industry.

[0003] Most existing instrument reading recognition methods rely on the accuracy of fitting the linear equation of the instrument pointer. A fast, accurate, and efficient pointer fitting method can provide crucial basic information for obtaining accurate instrument readings. However, existing instrument reading methods only consider individual scale values ​​adjacent to the pointer, neglecting the important role played by multiple scale values ​​in pointer reading calculations.

[0004] Patent [CN115512343A] mainly describes a method based on target detection to complete instrument scale value recognition and image correction based on scale value. It also describes a threshold segmentation pointer fitting method. However, this method requires manual calculation of the optimal segmentation threshold for the current instrument image, which is not very practical. Summary of the Invention

[0005] (a) Technical problems to be solved

[0006] The technical problem to be solved by this invention is how to provide a method for pointer segmentation and pointer linear equation fitting, scale value recognition and reading recognition of a circular pointer instrument, so as to solve the problems of existing pointer instruments where the detection of scale values ​​and pointers are carried out in stages during the reading recognition process, which consumes a lot of time; the fitting of the pointer linear equation depends on all pointer pixels, and the accuracy is easily affected by the detection results of pointer pixels; and multiple scale value information is not fully utilized, resulting in large instrument reading errors.

[0007] (II) Technical Solution

[0008] To address the aforementioned technical problems, this invention proposes a method for fitting the pointer of a circular pointer-type instrument and recognizing the instrument reading. The method of this invention uses a semantic segmentation algorithm to simultaneously segment the instrument pointer and scale values, and further automatically completes the instrument pointer fitting based on the pointer pixels.

[0009] This invention provides a method for fitting the pointer of a circular pointer instrument and identifying the instrument reading, the method comprising the following steps:

[0010] S101. Use semantic segmentation methods to simultaneously segment the pixel regions of the scale values ​​and pointers in the dashboard;

[0011] S102. Use image classification to identify the scale value and determine the rotation center of the pointer;

[0012] S103. Use the skeleton thinning algorithm and least squares method to fit the pointer line equation to the pointer pixel region generated by the segmentation to determine the direction of the pointer;

[0013] S104. Calculate the pointer reading using a weighted average of multiple scale values.

[0014] (III) Beneficial Effects

[0015] This invention proposes a method for fitting and recognizing the pointer of a circular pointer-type instrument. This method uses semantically segmented pointer pixel regions to fit the pointer's linear equation, is applicable to different semantic segmentation networks, and considers multiple scale values ​​to calculate the instrument reading, thus improving the accuracy of pointer instrument reading recognition. Attached Figure Description

[0016] Figure 1 This is an overall flowchart of the present invention;

[0017] Figure 2 This is a flowchart of the circular pointer instrument pointer fitting and reading recognition method of the present invention;

[0018] Figure 3 A semantic segmentation result diagram of the instrument scale values ​​and pointer;

[0019] Figure 4 The result is the ellipse at the position of the scale value and the rotation center of the pointer;

[0020] Figure 5 Obtain the results for the pixel area of ​​the instrument pointer;

[0021] Figure 6 Image showing the result of pointer pixel skeleton extraction;

[0022] Figure 7 A schematic diagram of the pointer pixel skeleton branch elimination method;

[0023] Figure 8 The graph shows the fitting result of the pointer line equation;

[0024] Figure 9 Determine the direction of the pointer using an interpretive diagram;

[0025] Figure 10 An explanation diagram of the angle method;

[0026] Figure 11 This is a diagram illustrating the weighted angle method. Detailed Implementation

[0027] To make the objectives, contents, and advantages of the present invention clearer, the specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples.

[0028] This application is based on the inventor's understanding and discovery of the following problems:

[0029] The pointer pixel regions in instrument images vary in size, and the acquired images are affected by the natural environment, resulting in significant differences in pointer pixel segmentation even for the same image. Fitting a pointer line equation using all pointer pixels introduces substantial errors. Furthermore, considering only a single scale value near the pointer leads to low accuracy in calculating instrument readings, resulting in insufficient utilization of instrument information and consequently, low precision in pointer instrument readings obtained using existing recognition methods or technologies. Therefore, it is necessary to simplify the segmented pointer pixels, retaining only the unit pixel that best represents the region for fitting the pointer line equation; and to consider multiple scale values ​​in the instrument reading calculation to improve the accuracy of pointer instrument readings.

[0030] The present invention aims to at least partially solve one of the technical problems in the related art.

[0031] Therefore, one objective of this invention is to provide a method for fitting and recognizing the pointer of a circular pointer instrument, which can effectively improve the accuracy of the reading of the circular pointer instrument.

[0032] To achieve the above objectives, this invention provides a method for fitting and recognizing the pointer of a circular pointer-type instrument, comprising:

[0033] S101. Use semantic segmentation methods to simultaneously segment the pixel regions of the scale values ​​and pointers in the dashboard;

[0034] S102. Use image classification to identify the scale value and determine the rotation center of the pointer;

[0035] S103. Use the skeleton thinning algorithm and least squares method to fit the pointer line equation to the pointer pixel region generated by the segmentation to determine the direction of the pointer;

[0036] S104. Calculate the pointer reading using a weighted average of multiple scale values.

[0037] The present invention provides a method for fitting and recognizing pointers in a circular pointer instrument. This method reduces the pointer pixels after semantic segmentation to a unit pixel, fits the pointer line equation using the least squares method, and calculates the instrument reading by considering multiple scale values, which helps to improve the accuracy of pointer instrument reading recognition.

[0038] In addition, the circular pointer instrument pointer fitting and reading recognition method according to the above embodiments of the present invention may also have the following additional technical features:

[0039] Furthermore, in one embodiment of the present invention, the method further includes: using a semantic segmentation algorithm to segment the pixel regions of the numbers and the pointer on the pointer instrument scale, using the segmented pointer pixel region as the key region for pointer fitting, and using the segmented scale value pixel region as the key region for scale value recognition.

[0040] Furthermore, in one embodiment of the present invention, the method further includes: using an image classification algorithm to classify the key regions of the segmented scale values ​​to obtain the specific value of each scale value; calculating the center point of the pixel region to obtain the coordinates of the center point of the scale value; fitting the ellipse equation of the location of the scale value using the coordinates of the center point of the scale value; and using the center of the ellipse as the rotation center of the pointer.

[0041] Furthermore, in one embodiment of the present invention, the method further includes: employing an image skeleton thinning algorithm to extract the pointer skeleton of the pointer pixel region, calculating the center point of the skeleton pixel points in different rings, eliminating the branch pixel points of the pointer skeleton, fitting the pointer line equation using the least squares method, and determining the direction of the pointer by using the distance between the intersection point of the ellipse and the pointer line equation and the center of the circle.

[0042] Furthermore, in one embodiment of the present invention, the method further includes: calculating the reading using the angle between the pointer rotation center and the pointer tip and the adjacent left and right scale values. Alternatively, four sets of data are formed based on the four scale values ​​adjacent to the left and right of the pointer tip, different weights are assigned according to the distance, and then a weighted sum is performed to obtain the accurate scale value.

[0043] Example 1:

[0044] The following describes a method and apparatus for fitting and recognizing the pointer of a circular pointer instrument according to an embodiment of the present invention, with reference to the accompanying drawings. First, the method for fitting and recognizing the pointer of a circular pointer instrument according to an embodiment of the present invention will be described with reference to the accompanying drawings.

[0045] Figure 1 , 2This is a flowchart of a method for fitting and recognizing the pointer of a circular pointer instrument according to an embodiment of the present invention.

[0046] like Figure 1 As shown, the method for fitting and recognizing the pointer of a circular pointer instrument includes the following steps:

[0047] In step S101, a semantic segmentation algorithm is used to simultaneously segment the pixel regions of the scale values ​​and pointers in the dashboard.

[0048] In one embodiment of the present invention, a convolutional neural network is used, including but not limited to the Swin-Unet algorithm, to simultaneously segment the scale values ​​and pointers in the dashboard. The Swin-Unet algorithm includes an encoder, a decoder, a bottleneck layer, and a jumper structure. The Swin Transformer Block is used as the depth feature calculation block of the image. Patch Merging completes downsampling, and Patch Expanding completes upsampling. A symmetrical pure Swin Transformer image feature extraction network is designed with a U-shaped structure.

[0049] The algorithm input is the image to be detected, and the algorithm output is the category map of the scale value pixel region and the pointer pixel region in the dashboard. The detection results are as follows: Figure 3 As shown.

[0050] In step S102, the image classification method is used to identify the scale value and determine the rotation center of the pointer. This involves using a convolutional neural network, including but not limited to the Swin Transformer image classification network, to identify the scale value of the pixel region. The step of determining the rotation center of the pointer is as follows:

[0051] S201. Determine the coordinates of the center point of each scale value.

[0052] With the top left corner of the image as the origin of the coordinate system, and the right side of the origin as... The positive axis direction, below the origin of the coordinate system. Establish a Cartesian coordinate system along the positive axis; calculate the center point of all pixels by the scale value. Determine the coordinates of the center point of this scale value. .

[0053] in, The calculation formula is shown below.

[0054] (1)

[0055] in, It is the total number of pixels in a single scale value area. It is the first of all pixels in a single tick value. Each pixel.

[0056] S202. Determine the equation of the ellipse where the scale value is located.

[0057] The equation of the ellipse is as follows:

[0058] (2)

[0059] The coordinates of the center point of all scale values ​​on the instrument were determined using the least squares method. Fitting an ellipse equation to a set of parameters involves finding the set of parameters. Let the coordinates of all the scale values ​​obtained by the division be such that the error of equation (2) is minimized, which is to say, the value of equation (3) is minimized.

[0060] (3)

[0061] in This indicates the number of all tick values. Indicates the first The center point coordinates of all pixels at each tick value .

[0062] S203. Determine the center of rotation of the pointer.

[0063] The geometric center of the ellipse is obtained from the following formula.

[0064] (4)

[0065] The geometric center of the ellipse is used as the center of pointer rotation. The resulting diagram shows the ellipse at the scale value location and the pointer's rotation center. Figure 4 As shown.

[0066] In step S103, the pointer pixel region generated by semantic segmentation is fitted with the pointer line equation using a skeleton thinning algorithm and the least squares method to determine the direction of the pointer.

[0067] The steps for fitting the equation of the pointer line are as follows:

[0068] S301. Obtain the pointer pixel area

[0069] All pointer pixels are obtained from the category map of the pointer pixel region after segmentation of the instrument. The resulting image of the obtained pointer pixels is shown below. Figure 5 As shown.

[0070] S302, Execute skeleton refinement algorithm

[0071] The pointer skeleton thinning algorithm is applied to all pointer pixel regions to obtain pointer skeleton maps containing only unit pixels. The pointer pixel skeleton thinning result is shown in the image below. Figure 6 As shown.

[0072] S303. Obtain the set of pointer coordinate points needed to fit the pointer line equation.

[0073] The center point coordinates of all pointer skeleton pixels are calculated using the following formula.

[0074] (5)

[0075] in It is the number of pixels in the entire pointer skeleton. Indicates the first The coordinates of the pointer skeleton pixels.

[0076] The center point of all coordinates of the pointer skeleton is calculated using equation (5). .by Using as the center, draw concentric circles with radii following the increasing pattern of the Fibonacci sequence, stopping when no new pointer skeleton pixels are added to the current concentric circle. Let the th circle be the th... The concentric circles drawn next are To more accurately fit the pointer, and The pointer skeleton pixels between the rings are divided into the left ring skeleton pixel set. and the set of pixels of the right circular skeleton This preserves a pixel distribution range similar to the original pointer. Let the first... The concentric circles drawn next The diameter is ,remember and The pointer skeleton pixels between the rings are ,common Each pointer skeleton pixel. If and The distance between them is less than Then Record the pixel set of the left circular skeleton. Otherwise, record it in the right circular skeleton pixel set. According to equation (5), calculate pixel center point ,calculate Pixel center point Finally, the center point of the left ring pixel is calculated for each ring. and the center point of the right circular pixel The set of pixels that make up the pointer skeleton To obtain the set of pointer coordinate points needed to fit the equation of the pointer line. Among them, the pointer pixel skeleton branch elimination method is as follows: Figure 7 As shown.

[0077] S304. Fitting the pointer line equation using the least squares method.

[0078] The equation of the pointer line is shown in the following formula.

[0079] (6)

[0080] According to all The pointer skeleton point set is used to fit the pointer line equation using the least squares method. The fitting process involves finding the parameter set. Let the center point of all the pixels in the pointer skeleton ring be... The error in equation (6) is minimized, which means that equation (7) is minimized:

[0081] (7)

[0082] in This indicates the number of center points of the pixels in the pointer skeleton ring. Indicates the detected number The coordinates of the center point of each pixel in the ring.

[0083] Finally, the fitted pointer line equation is as follows: Figure 8 As shown.

[0084] In step S103, the direction of the pointer is further determined. The steps for determining the pointer direction are as follows:

[0085] S305. Calculate the intersection point of the pointer line equation and the ellipse equation of the scale value position using equations (6) and (2). and .

[0086] S306. Determine the direction of the pointer.

[0087] Calculate the center point of the pointer and and Distance, distance The nearest intersection That is, the tip of the pointer. The result of determining the pointer direction is shown in the image below. Figure 9 As shown.

[0088] In step S104, the pointer reading is calculated using the angle between the tip of the instrument pointer and the adjacent left and right scale values, such as... Figure 10 As shown. It is the angle between the line connecting two adjacent tick marks on the pointer and the center of the circle. It is the angle between the pointer's straight line and the line connecting the smaller scale value and the center of the circle. The formula for calculating the pointer logarithm is as follows:

[0089] (8)

[0090] in It is the smaller tick mark value adjacent to the pointer line. It is the larger scale value adjacent to the pointer line. This is the final reading.

[0091] In step S104, calculating the pointer reading further includes:

[0092] Considering multiple scale values, take the four scale values ​​adjacent to the left and right of the pointer line, respectively. , And the pointer is located on the line. Between them, there are four sets of data, namely , , , The pointer is weighted sequentially based on its distance from the scale reading and the pointer's straight-line distance, with the largest distance having the smallest weight. The weights for the four sets of data are 0.4, 0.25, 0.25, and 0.1, respectively. The weighted pointer reading is then calculated using the following formula.

[0093] (9)

[0094] in, , , It is the first Angle values ​​of the data pairs The weighted angle method is explained in the diagram below. Figure 11 As shown.

[0095] This invention provides a method for fitting and recognizing the pointer of a circular pointer instrument, which can transform the position of the scale value in a tilted and rotated pointer instrument from an ellipse to a perfect circle, thereby improving the accuracy of pointer instrument reading recognition.

[0096] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for fitting the pointer of a circular pointer-type instrument and identifying the instrument reading, characterized in that, The method includes the following steps: S101. Use semantic segmentation methods to simultaneously segment the pixel regions of the scale values ​​and pointers in the dashboard; S102. Use image classification to identify the scale value and determine the rotation center of the pointer; S103. Use the skeleton thinning algorithm and least squares method to fit the pointer line equation to the pointer pixel region generated by the segmentation to determine the direction of the pointer; S104. Calculate the pointer reading using a weighted average of multiple scale values; in, Step S103 includes: using an image skeleton thinning algorithm to extract the pointer skeleton of the pointer pixel region, calculating the center point of the skeleton pixel in different rings, eliminating the branch pixel of the pointer skeleton, fitting the pointer line equation using the least squares method; and determining the direction of the pointer by using the distance between the intersection of the ellipse and the pointer line equation and the center of the circle. Step S103 specifically includes: S301. Obtain the pointer pixel area Obtain all pointer pixels from the category map of the pointer pixel region after the instrument is segmented; S302, Execute skeleton refinement algorithm A pointer skeleton refinement algorithm is applied to all pointer pixel regions to obtain a pointer skeleton map containing only unit pixels. S303. Obtain the set of pointer coordinate points needed to fit the pointer line equation. The center point coordinates of all pointer skeleton pixels are calculated using the following formula. （5） in It is the number of pixels in the entire pointer skeleton. Indicates the first The coordinates of the pointer skeleton pixels; The center point of all coordinates of the pointer skeleton is calculated using equation (5). ;by Using as the center, draw concentric circles with radii following the increasing pattern of the Fibonacci sequence, stopping when no new pointer skeleton pixels are added to the current concentric circle. Let the th circle be the th... The concentric circles drawn next are To more accurately fit the pointer, and The pointer skeleton pixels between the rings are divided into the left ring skeleton pixel set. and the set of pixels of the right circular skeleton Preserve the pixel distribution range similar to the original pointer; denoted as the first... The concentric circles drawn next The diameter is ,remember and The pointer skeleton pixels between the rings are ,common One pointer skeleton pixel; if and The distance between them is less than Then Record the pixel set of the left circular skeleton. Otherwise, record it in the right circular skeleton pixel set. According to equation (5), calculate pixel center point ,calculate Pixel center point Finally, the center point of the left ring pixel is calculated for each ring. and the center point of the right circular pixel The set of pixels that make up the pointer skeleton To obtain the set of pointer coordinate points needed to fit the equation of the pointer line. .

2. The method for fitting the pointer of a circular pointer-type instrument and identifying the instrument reading as described in claim 1, characterized in that, Step S101 includes: using a semantic segmentation algorithm to segment the pixel regions of the numbers on the pointer instrument scale and the pointer itself, using the segmented pointer pixel region as the key region for pointer fitting, and using the segmented scale value pixel region as the key region for scale value recognition.

3. The method for fitting the pointer of a circular pointer-type instrument and identifying the instrument reading as described in claim 2, characterized in that, Step S102 includes: using an image classification algorithm to classify the key regions of the segmented scale values ​​to obtain the specific value of each scale value; calculating the center point of the pixel region to obtain the coordinates of the center point of the scale value; fitting the ellipse equation of the location of the scale value using the coordinates of the center point of the scale value; and using the center of the ellipse as the rotation center of the pointer.

4. The method for fitting the pointer of a circular pointer-type instrument and identifying the instrument reading as described in claim 3, characterized in that, Step S104 includes: calculating the reading using the angle between the pointer rotation center and the pointer tip and the adjacent scale values ​​on the left and right, or forming four sets of data based on the four scale values ​​adjacent to the pointer tip on the left and right, assigning different weights according to the distance, and then performing weighted summation to obtain the accurate scale value.

5. The method for fitting the pointer of a circular pointer-type instrument and identifying the instrument reading as described in any one of claims 1-4, characterized in that, Step S101 specifically includes: A convolutional neural network (CNN) is used to simultaneously segment the scale values ​​and pointers in a dashboard. The CNN is the Swin-Unet algorithm, which includes an encoder, decoder, bottleneck layer, and jumper structure. The Swin Transformer Block is used as the image depth feature calculation block. Patch Merging performs downsampling, and Patch Expanding performs upsampling. A symmetrical pure Swin Transformer image feature extraction network with a U-shaped structure is designed. The algorithm input is the image to be detected, and the algorithm output is the category map of the scale value pixel region and the pointer pixel region in the dashboard.

6. The method for fitting the pointer of a circular pointer-type instrument and identifying the instrument reading as described in claim 5, characterized in that, Step S102 specifically includes: A convolutional neural network is used to identify the numerical values ​​of the scale value pixel region. The convolutional neural network is the SwinTransformer image classification network. The steps to determine the center of rotation of the pointer are as follows: S201. Determine the coordinates of the center point of each scale value. With the top left corner of the image as the origin of the coordinate system, and the right side of the origin as... The positive axis direction, below the origin of the coordinate system. Establish a Cartesian coordinate system along the positive axis; calculate the center point of all pixels by the scale value. Determine the coordinates of the center point of this scale value. ; in, The calculation formula is shown below. （1） in, It is the total number of pixels in a single scale value area. It is the first of all pixels in a single tick value. 1 pixel; S202. Determine the equation of the ellipse where the scale value is located. The equation of the ellipse is as follows: （2） The coordinates of the center point of all scale values ​​on the instrument were determined using the least squares method. Fitting an ellipse equation to a set of parameters involves finding the set of parameters. Let the coordinates of all the scale values ​​obtained by segmentation be such that the error of equation (2) is minimized, which is to say, the value of equation (3) is minimized: （3） in This indicates the number of all tick values. Indicates the first The center point coordinates of all pixels at each tick value ; S203. Determine the center of rotation of the pointer. The geometric center of the ellipse is obtained from the following formula. （4） The geometric center of the ellipse is used as the center of rotation for the pointer.

7. The method for fitting the pointer of a circular pointer-type instrument and identifying the instrument reading as described in claim 6, characterized in that, Step S103 further includes: S304. Fitting the pointer line equation using the least squares method. The equation of the pointer line is shown in the following formula. （6） According to all The pointer skeleton point set is used to fit the pointer line equation using the least squares method; the fitting process involves finding the parameter set. Let the center point of all the pixels in the pointer skeleton ring be... The error in equation (6) is minimized, which means that equation (7) is minimized: （7） in This indicates the number of center points of the pixels in the pointer skeleton ring. Indicates the detected number The coordinates of the center point of each pixel in the ring; S305. Calculate the intersection point of the pointer line equation and the ellipse equation of the scale value position using equations (6) and (2). and ; S306. Determine the direction of the pointer. Calculate the center point of the pointer and and Distance, distance The nearest intersection It refers to the tip of the pointer.

8. The method for fitting the pointer of a circular pointer instrument and identifying the instrument reading as described in claim 7, characterized in that, Step S104 specifically includes: The pointer reading is calculated using the angle between the tip of the instrument pointer and the adjacent left and right scale values. It is the angle between the line connecting two adjacent tick marks on the pointer and the center of the circle. The angle between the pointer's straight line and the line connecting the smaller scale value and the center of the circle is given by the following formula for calculating the pointer logarithm: （8） in It is the smaller tick mark value adjacent to the pointer line. It is the larger scale value adjacent to the pointer line. This is the final reading.

9. The method for fitting the pointer of a circular pointer-type instrument and identifying the instrument reading as described in claim 7, characterized in that, Step S104 specifically includes: Considering multiple scale values, take the four scale values ​​adjacent to the left and right of the pointer line, respectively. , And the pointer is located in a straight line. Between them, there are four sets of data, namely , , , ; Weights are assigned sequentially based on the distance of the scale value from the pointer's straight line, with the largest distance having the smallest weight. The weights for the four sets of data are 0.4, 0.25, 0.25, and 0.1, respectively. The weighted pointer reading is then calculated using the following formula. （9） in, , , It is the first Angle values ​​of the data pairs .

Images