A method for identifying the degree of pointer instruments based on coordinate transformation

Abstract

The present invention relates to a method for identifying the degree of a pointer type instrument based on coordinate transformation, comprising the following steps: obtaining an initial instrument image taken by a camera, obtaining a positioning instrument image using a yolov4 algorithm, preprocessing the positioning instrument image, extracting the elliptical scale line arc of the instrument using an edge detection algorithm from the preprocessed image, obtaining the fitting center of the instrument using an elliptical fitting according to the obtained elliptical scale line arc, establishing a dial coordinate system according to the fitting center, detecting the pointer using an LSD straight line detection algorithm, establishing a correction coordinate system with the image shooting point as the center, obtaining a coordinate transformation matrix according to the correction coordinate system and the dial coordinate system, obtaining the range value and the range angle of the pointer instrument, and obtaining the final instrument reading according to the coordinate transformation matrix. Compared with the prior art, the present invention has the advantages of high accuracy and strong applicability.

Description

Technical Field

[0001] The invention relates to the technical field of instrument measurement, and in particular to a method for identifying the degree of a pointer-type instrument based on coordinate transformation. Background Art

[0002] Mechanical pointer instruments are widely used in industry for their simple structure, low price, and timely and accurate data reflection. In power systems, a large number of pointer instruments are used in equipment such as switch cabinets and station power low-voltage panels for real-time monitoring of instrument status. Therefore, pointer instruments play an important and irreplaceable role and significance in power system status detection, and their readings have always been the focus of research.

[0003] However, due to the complex environment in which the pointer instrument equipment is located, reflections, tilts, and loud noises will inevitably appear in the pictures during the process of taking pictures and identifying them, resulting in large recognition errors for the readings of these instruments, affecting the accuracy of the readings. Among them, the reading error caused by tilt is the most complex. Due to the uncertainty of the image acquisition position, the instrument readings will often be inconsistent with the actual situation due to the tilt angle. Summary of the invention

[0004] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and provide a method for identifying the degree of a pointer instrument based on coordinate transformation.

[0005] The purpose of the present invention can be achieved by the following technical solutions:

[0006] A method for identifying the degree of a pointer instrument based on coordinate transformation comprises the following steps:

[0007] S1, obtaining an initial instrument image taken by a camera;

[0008] S2, use yolov4 algorithm to obtain the positioning instrument image;

[0009] S3, preprocessing the positioning instrument image, and using an edge detection algorithm to extract the elliptical scale arc of the instrument from the preprocessed image;

[0010] S4, according to the ellipse scale arc obtained in step S3, use ellipse fitting to obtain the fitting center of the instrument, establish a dial coordinate system according to the fitting center, and use the LSD straight line detection algorithm to detect the pointer;

[0011] S5, establishing a calibration coordinate system with the image shooting point as the center, and obtaining a coordinate transformation matrix according to the calibration coordinate system and the dial coordinate system;

[0012] S6. Obtain the range value and range angle of the pointer instrument, and obtain the final instrument reading according to the coordinate transformation matrix.

[0013] Furthermore, the specific steps of obtaining the fitting center of the instrument in step S4 include:

[0014] The direct least square method is used to perform multiple ellipse fitting on the ellipse scale arc obtained by edge detection, and the center of the ellipse obtained by each fitting is calculated and found; the coordinates of all the ellipse centers obtained are counted, and the point with the highest number of occurrences is taken as the center of the ellipse.

[0015] Furthermore, the specific step of detecting the pointer in step S4 includes using LSD straight line detection to detect the instrument pointer, and taking the straight line with a length of the detected straight line segment greater than 1 / 2R0 as the instrument pointer, wherein the size of R0 is the distance from the center of the fitting circle to the arc of the ellipse scale line.

[0016] Furthermore, the step S5 specifically includes:

[0017] S501, taking the image shooting point O' as the center, establishing a calibration coordinate system x'y'z', where the x'y'z' axes are parallel to the xyz axes of the dial coordinate system respectively;

[0018] S502, calculate the distances a, b, c between the origin of the calibration coordinate system and the three planes yoz, xoz, xoy of the dial coordinate system, and obtain the coordinates of O' in the dial coordinate system (-a, b, c), the coordinates of the origin of the dial coordinate system O in the calibration coordinate system (a, -b, -c), and the coordinates of the top of the pointer on the dial collected from point O' in the calibration coordinate system (a+x1, -b, -c+z1). The coordinates of the actual top of the pointer in the calibration coordinate system are expressed according to the height difference between the pointer and the dial. The specific coordinate expression is:

[0019]

[0020]

[0021]

[0022] S503, calculate the coordinate transformation matrix T according to the coordinates obtained in S502, and the calculation expression is as follows:

[0023] Where d is the height difference between the pointer and the dial, which can be obtained from the instrument data manual.

[0024] Furthermore, the final meter reading calculation steps are as follows:

[0025] The coordinate transformation matrix T is used to correct the coordinate transformation of the coordinate (a+x1, -b, -c+z1), and the value on the y-axis is set to zero, so as to obtain its coordinate D on the disk after perspective transformation and the angle ε between the coordinate D and the x-axis. The correct reading num after correction is calculated using the angle method. The specific calculation expression is as follows:

[0026]

[0027] In the formula, θ represents the total angle of the measuring range, and L represents the measuring range value.

[0028] Furthermore, the pointer instrument scale calculation method in step S6 specifically includes:

[0029] S601, using the Hough circle detection algorithm to detect the scale arc and perform perspective transformation; then using the Hough line detection algorithm to detect the straight line contour, setting the screening conditions for screening, obtaining the scale line, and performing perspective transformation;

[0030] S602, marking the positions and corresponding scales of the first scale line and the last scale line after the transformation, and calculating the scale difference L between them as the range value and the angle θ as the range angle.

[0031] Furthermore, the screening conditions of the scale lines are as follows:

[0032] The position of the straight line profile lies within the scale arc;

[0033] The aspect ratio of the straight line profile is greater than 1:4;

[0034] The direction of the minimum circumscribed rectangle of the straight contour points to the center of the circle.

[0035] Furthermore, the step S2 specifically includes the following steps:

[0036] S201, using the backbone feature extraction network Backbone of yolov4 to extract backbone features;

[0037] S202, constructing a feature pyramid using the SPP structure for the extracted features;

[0038] S203, using the K-means algorithm to adjust the parameters of the feature pyramid, and using the yolohead algorithm to perform prediction to obtain the instrument preset frame;

[0039] S204, retaining the image of the initial instrument image in the instrument preset frame as the positioning instrument image.

[0040] Furthermore, the preprocessing step in step S3 includes:

[0041] S301, smoothing the image using an adaptive nonlinear median filtering method to remove image noise;

[0042] S302, performing adaptive histogram equalization on the instrument image;

[0043] S303: using the maximum inter-class variance method, the image is divided into two parts: foreground and background.

[0044] Furthermore, the specific steps of extracting the instrument edge information in step S3 include: using the Canny edge detection algorithm to detect the preprocessed image to obtain an edge image, performing a closing operation on the edge image to eliminate small holes, bridge narrow discontinuities and gullies, and fill broken contours, smooth the detected contour, and obtain an elliptical scale line arc.

[0045] Compared with the prior art, the present invention has the following advantages:

[0046] 1. After collecting the images of pointer instruments at various angles, the present invention performs preprocessing and circle center extraction in sequence, and then establishes a dial coordinate system and a correction coordinate system respectively to realize the pointer coordinate transformation under the tilt of the image, so as to calculate the correct pointer instrument reading. The method has strong applicability and can obtain readings even when image acquisition is difficult. At the same time, the accuracy of the readings can be verified by measuring different angles, and the method is flexible.

[0047] 2. The present invention performs line screening when obtaining the instrument pointer and the straight line profile, thereby ensuring the accuracy of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 It is a schematic diagram of the process of the present invention.

[0049] Figure 2 It is a schematic diagram of the corresponding conversion between the dial coordinate system and the correction coordinate system of the present invention.

[0050] Figure 3 This is a schematic diagram of angle calculation of the present invention. DETAILED DESCRIPTION

[0051] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0052] This embodiment provides a method for identifying the degree of a pointer instrument based on coordinate transformation, the process is as follows: Figure 1 As shown, the specific steps include:

[0053] Step S1, obtaining an initial instrument image, setting up a camera around the instrument to collect the image, and uploading it to a computer at the same time;

[0054] Step S2: Use the yolov4 algorithm to locate the instrument in the initial instrument image to obtain a located instrument image. Considering the complex environment in the actual application of the power system, it may be impossible to accurately shoot the pointer instrument. The specific process includes the following steps:

[0055] Step S201, use the backbone feature extraction network Backbone of yolov4 to extract backbone features. The backbone feature extraction network uses CSPDarkNet53 of the CSPnet series, and the activation function uses the Mish activation function. The formula of the Mish function is as follows:

[0056] Mish = x × tanh (ln (1 + e x ))

[0057] In the formula, x represents the extracted features. The CSPnet structure splits the stack of residual blocks. The main part continues the stack of the original residual blocks, and the other part is directly connected to the end after a small amount of processing like a residual edge.

[0058] Step S202: construct a feature pyramid based on the extracted features using the SPP structure. After performing three DarknetConv2D_BN_Leaky convolutions on the last feature layer of CSPdarknet53, four maximum poolings of different scales are used for processing, so that the receptive field is increased and the most significant context features are separated, which is conducive to finding the edge of the instrument.

[0059] Step S203: Use the K-means_anchor algorithm to adjust parameters, and use distance as the similarity evaluation index, that is, the closer the distance between two objects, the greater their similarity. And use the yolohead algorithm to predict the obtained features to obtain the instrument preset frame.

[0060] Step S204: draw an instrument preset frame on the image, and only retain the image portion within the preset frame, eliminate the complex background outside the frame that interferes with instrument recognition, and obtain the positioning instrument image P1 at the angle.

[0061] Step S3: pre-process the image by filtering, and extract the inner edge information of the dial by using Canny edge detection. The specific process includes the following steps:

[0062] Step S301: Use the adaptive nonlinear median filtering method to smooth the positioning instrument image P1 to remove image noise. Salt and pepper noise is one of the most common noises in images. A typical application of median filtering is to eliminate salt and pepper noise. The specific steps are as follows:

[0063] First, the obtained positioning instrument image P1 is scanned line by line. When processing each pixel, it is determined whether the pixel is the maximum or minimum value of the neighboring pixels covered by the filter window. If it is, it means that the pixel is a noise point, and the normal median filter is used to process the pixel; if not, it means that it is an image edge point, and it is not processed, and the first processed image P2 is obtained.

[0064] Step S302: Based on the median filter, P2 is subjected to contrast-limited adaptive histogram equalization, the local histogram of P2 is calculated, and then the brightness is redistributed to change the image contrast, improve the situation where the overall image contrast is high but there are local areas that are overexposed or too dark, and enhance the image. However, in the calculation process, contrast limiting must be used for each small area to obtain the second processed image P3.

[0065] Step S303: using the maximum inter-class variance method, the image is divided into two parts, the foreground and the background. The larger the inter-class variance between the background and the foreground, the greater the difference between the two parts constituting the image. The specific steps are as follows:

[0066] Use the exhaustive method to traverse every pixel point of P3, select the initialization threshold t, t divides the image into two categories, calculate the mean of the two categories of pixel sets and the inter-class variance of the two categories under the threshold, cycle t from 0 to 255, calculate the inter-class variance under each threshold, and the corresponding t when the inter-class variance is the largest is the required threshold. The binary image at this time is recorded as the third processed image P4.

[0067] g=ω1×ω2×(μ1-μ2) 2

[0068] Where ω1 represents the ratio of foreground pixels to the whole image, ω2 represents the ratio of background pixels to the whole image, μ1 represents the average grayscale of the foreground, and μ2 represents the average grayscale of the background.

[0069] Step S304: Use Canny edge detection to perform edge detection on P4 to obtain the instrument edge image P5.

[0070] Step S305, perform a closing operation on P5, that is, first dilate and then erode, eliminate small holes, bridge narrow discontinuities and gullies, and fill broken contours, smooth the detected contour, and obtain an elliptical scale line arc.

[0071] Step S4, using ellipse fitting to find the center of the fitting circle, using LSD straight line detection to detect the pointer, and establishing the dial coordinate system. The specific process includes the following steps:

[0072] Step S401: perform multiple ellipse fitting on the ellipse scale arc obtained by edge detection using the direct least square method, and calculate and find the center of the ellipse obtained by each fitting.

[0073] Step S402, statistics are performed on the obtained coordinates of the center of the ellipse, and the point with the highest number of occurrences is used as the center of the ellipse on the dial. The shortest distance R0 from the center to the arc of the ellipse scale line is measured, and a dial coordinate system is established with the center as the origin, with the y-axis perpendicular to the dial and pointing outward, and the z-axis parallel to the dial and pointing directly upward on the dial plane.

[0074] Step S403, use LSD straight line detection to detect the instrument pointer, take the line segment with a length greater than 1 / 2R0 as the instrument pointer, ignore other detected straight lines, and record the coordinates of the top of the instrument pointer at this time as (x1, y1, z1). It is easy to deduce that y1=0 at this time.

[0075] Step S5: Obtain the coordinate transformation matrix using the established dial coordinate system. The specific process includes the following steps:

[0076] Step S501: With the image shooting point O' as the center, establish a calibration coordinate system x'y'z', where the x'y'z' axes are parallel to the xyz axes of the dial coordinate system, such as Figure 2 shown.

[0077] Step S502: Calculate the distances of the origin of the calibration coordinate system relative to the three planes yoz, xoz, and xoy of the dial coordinate system as a, b, and c, respectively. Then, relative to the dial, the coordinates of O' on the dial coordinate system are (-a, b, c), and the coordinates of the origin of the dial coordinate system O on the calibration coordinate system are (a, -b, -c). When the image is taken from point O', the coordinates of the point on the dial indicating the top of the pointer in the calibration coordinate system are expressed as (a+x1, -b, -c+z1), which are equivalent to the coordinates of the point projected from O' to the xoz plane of the dial coordinate system.

[0078] Use a vernier caliper to measure or obtain the height difference d between the pointer and the dial from the pointer data manual. Project point O' through the pointer vertex to the xoz plane to obtain point (x1, y1, z1). According to the Pythagorean theorem and similar triangles, the coordinates of the actual vertex of the pointer in the calibration coordinate system are:

[0079]

[0080]

[0081]

[0082] The coordinates obtained in steps S503 and S502 can be used in any case to calculate the coordinate transformation matrix T between the actual pointer position and the observation position under the tilt condition based on the known a, b, c, d, x1, y1, z1. The calculation expression is as follows:

[0083]

[0084] Then we can get:

[0085]

[0086] By using the above formula to obtain the transformation matrix T, the transformation between the observed position and the actual position of the pointer can be realized, and the error caused by the height difference d between the pointer and the dial can be eliminated.

[0087] Step S6, calculating the range value and the range angle of the pointer meter reading, and using coordinate transformation to obtain the actual pointer meter value. The specific process includes the following steps:

[0088] Step S601: Use Hough circle detection to detect the scale contour of the dial. Since there is only one arc in this example, the detected contour is the scale contour. Use perspective transformation to transform the shape of the elliptical arc contour into an arc. Then use Hough line detection to fit the detected straight line contour to detect the scale straight line. According to the characteristics of the dial, the screening conditions need to be set as follows:

[0089] The position of the straight line profile lies within the scale arc;

[0090] The aspect ratio of the straight line profile is greater than 1:4;

[0091] The direction of the minimum circumscribed rectangle of the straight contour points to the center of the circle.

[0092] The straight line contour that meets the above conditions is the scale line. The points in the scale line are transformed in perspective so that they are distributed on the arc after the above transformation.

[0093] Step S602, mark the position of the first scale line after transformation, record it as the starting scale 0 degree; mark the position of the last scale line after transformation, record it as the ending scale 450 degrees, represented by L, and calculate the angle between them, recorded as θ.

[0094] Step S603: For the coordinate point (a+x1, -b, -c+z1), coordinate transformation correction is performed using the coordinate transformation matrix T, and the value on the y-axis is set to zero, so that it is equivalent to projecting directly onto the dial, and its coordinate D on the disk after perspective transformation and the angle ε between the coordinate D and the x-axis are obtained, as shown in FIG. Figure 3 As shown, the angle method is used to calculate the correct reading num after correction. The calculation formula is as follows:

[0095]

[0096] It is worth noting that the present embodiment provides a method for identifying the degree of a pointer instrument based on coordinate transformation. When capturing images, it is possible to capture images of pointer instruments at different angles of the same instrument, and perform the above steps on the images at different angles respectively, to ensure the accuracy of the readings obtained through comparison.

[0097] The preferred specific embodiments of the present invention are described in detail above. It should be understood that a person skilled in the art can make many modifications and changes based on the concept of the present invention without creative work. Therefore, any technical solution that can be obtained by a person skilled in the art through logical analysis, reasoning or limited experiments based on the concept of the present invention on the basis of the prior art should be within the scope of protection determined by the claims.

Claims

1. A method for identifying the degree of a pointer instrument based on coordinate transformation, characterized in that: The following steps are involved: S1, obtaining an initial instrument image taken by a camera; S2, use yolov4 algorithm to obtain the positioning instrument image; S3, preprocessing the positioning instrument image, and using an edge detection algorithm to extract the elliptical scale arc of the instrument from the preprocessed image; S4, according to the ellipse scale arc obtained in step S3, use ellipse fitting to obtain the fitting center of the instrument, establish a dial coordinate system according to the fitting center, and use the LSD straight line detection algorithm to detect the pointer; S5, establishing a calibration coordinate system with the image shooting point as the center, and obtaining a coordinate transformation matrix according to the calibration coordinate system and the dial coordinate system; S6, obtaining the range value and range angle of the pointer instrument, and obtaining the final instrument reading according to the coordinate transformation matrix; The step S5 specifically includes: S501, taking the image shooting point O' as the center, establishing a calibration coordinate system x'y'z', where the x'y'z' axes are parallel to the xyz axes of the dial coordinate system respectively; S502, calculate the distances a, b, c between the origin of the calibration coordinate system and the three planes yoz, xoz, xoy of the dial coordinate system, and obtain the coordinates of O' in the dial coordinate system (-a, b, c), the coordinates of the origin of the dial coordinate system O in the calibration coordinate system (a, -b, -c), and the coordinates of the top of the pointer on the dial collected from point O' in the calibration coordinate system (a+x1, -b, -c+z1). The coordinates of the actual top of the pointer in the calibration coordinate system are expressed according to the height difference between the pointer and the dial. The specific coordinate expression is: S503, calculate the coordinate transformation matrix T according to the coordinates obtained in S502, and the calculation expression is as follows: Where d is the height difference between the pointer and the dial, which can be obtained from the instrument data manual; The step S2 specifically includes the following steps: S201, using the backbone feature extraction network Backbone of yolov4 to extract backbone features; S202, constructing a feature pyramid using the SPP structure for the extracted features; S203, using the K-means algorithm to adjust the parameters of the feature pyramid, and using the yolohead algorithm to perform prediction to obtain the instrument preset frame; S204, retaining the image of the initial instrument image in the instrument preset frame as the positioning instrument image.

2. The method for identifying the degree of a pointer instrument based on coordinate transformation according to claim 1, characterized in that: The specific steps of obtaining the fitting center of the instrument in step S4 include: performing multiple ellipse fitting on the ellipse scale arc obtained by edge detection using the direct least squares method, and calculating and finding the center of the ellipse obtained by each fitting; and performing statistics on the coordinates of all the ellipse centers obtained, and taking the point with the highest number of occurrences as the center of the ellipse.

3. The method for identifying the degree of a pointer instrument based on coordinate transformation according to claim 1, characterized in that: The specific step of detecting the pointer in step S4 includes using LSD straight line detection to detect the instrument pointer, and taking the straight line with a length of the detected straight line segment greater than 1 / 2R0 as the instrument pointer, wherein the size of R0 is the distance from the center of the fitting circle to the arc of the ellipse scale line.

4. The method for identifying the degree of a pointer instrument based on coordinate transformation according to claim 1, characterized in that: The final meter reading calculation steps are as follows: The coordinate transformation matrix T is used to correct the coordinate transformation of the coordinate (a+x1, -b, -c+z1), and the value on the y-axis is set to zero, so as to obtain its coordinate D on the disk after perspective transformation and the angle ε between the coordinate D and the x-axis. The correct reading num after correction is calculated using the angle method. The specific calculation expression is as follows: In the formula, θ represents the total angle of the measuring range, and L represents the measuring range value.

5. The method for identifying the degree of a pointer instrument based on coordinate transformation according to claim 1, characterized in that: The method for calculating the scale of the pointer instrument in step S6 specifically includes: S601, using the Hough circle detection algorithm to detect the scale arc and perform perspective transformation; then using the Hough line detection algorithm to detect the straight line contour, setting the screening conditions for screening, obtaining the scale line, and performing perspective transformation; S602, marking the positions and corresponding scales of the first scale line and the last scale line after the transformation, and calculating the scale difference L between them as the range value and the angle θ as the range angle.

6. The method for identifying the degree of a pointer-type instrument based on coordinate transformation according to claim 5 is characterized in that: The screening conditions for the tick marks are as follows: The position of the straight line profile lies within the scale arc; The aspect ratio of the straight line profile is greater than 1:4; The direction of the minimum circumscribed rectangle of the straight contour points to the center of the circle.

7. The method for identifying the degree of a pointer instrument based on coordinate transformation according to claim 1, characterized in that: The pre-processing step in step S3 includes: S301, smoothing the image using an adaptive nonlinear median filtering method to remove image noise; S302, performing adaptive histogram equalization on the instrument image; S303: using the maximum inter-class variance method, the image is divided into two parts: foreground and background.

8. The method for identifying the degree of a pointer instrument based on coordinate transformation according to claim 1, characterized in that: The specific steps of extracting the instrument edge information in step S3 include: The Canny edge detection algorithm is used to detect the preprocessed image to obtain the edge image, and the edge image is closed to eliminate small holes, bridge narrow discontinuities and gullies, and fill broken contours, smooth the detected contours, and obtain the elliptical scale line arc.

Images