A universal pointer instrument panel reading method

Abstract

The present invention relates to the technical field of pointer instrument image processing recognition reading method, specifically, a universal pointer instrument panel reading method is disclosed, wherein the improved Yolov8 instance segmentation network model obtained by replacing the C2f module in the yolov8n‑seg network model with the ConvFormer module and the TransFormer module in the MetaFormer of the Transformer architecture performs instance segmentation to obtain the scale starting point, scale end point and centroid point; calculates the angle corresponding to the maximum range; determines the number of pointers and the needle handle; obtains the actual pointer and its pointing point; obtains the angle and reading corresponding to the actual pointer. This method not only simplifies the marking process of the instrument scale area and reduces the dependence on network accuracy, but also can effectively cope with the reading use of double-pointer and three-pointer instruments, and improves the accuracy and applicability of instrument reading.

Description

Technical Field

[0001] The invention relates to the technical field of pointer instrument image processing recognition reading method. Background Art

[0002] The traditional substation inspection method is to manually read the values ​​of the recording instrument. The inspection efficiency is low, and it is impossible to perceive the operating status of the power equipment in real time. In some strong electric field environments, it is easy to cause electric shock accidents to the inspectors. With the advancement of science and technology, mobile inspection robots have gradually replaced manual inspections, and visual inspections are used to carry out daily inspections of substations, which helps the development of digital technology for power inspections. However, the internal forms of the instrument are diverse, and how to intelligently and accurately identify the instrument readings is still a hot research topic. At present, the public methods for visual image processing and reading recognition for pointer instruments have problems such as troublesome labeling of precisely marked scale areas, high requirements for network accuracy, and no substantive and general solutions for single, double, and triple pointers.

[0003] For example, Chinese Patent Application No. 202410749519.3 discloses a method for identifying the reading of a pointer instrument based on deep learning. MMD-Net is used to segment the center, scale, and pointer in the pointer instrument. The mask image is first obtained through the network, and then the dimension reduction ruler distance method is used to convert the annular scale area into a straight rectangular area. The width in the rectangular area and the position pointed by the pointer are used for reading. For another example, Chinese Patent Application No. 202410760568.7 discloses an industrial pointer instrument detection and reading recognition method, device, equipment, and medium. The improved YOLOv5s model is used to detect the target in the instrument area, and then the instrument tilt correction is performed on the detected instrument. The corrected instrument is segmented by the SegFormer semantic segmentation model. The segmented instrument panel image is filtered out with less noise through corrosion and expansion operations, and then the annular instrument panel outline is expanded into a rectangular image. Finally, the instrument reading is calculated by locating the relative scale position of the pointer and according to the dial range.

[0004] In the above disclosed method, the semantic segmentation model is not generally applicable to double pointers and triple pointers, where pointers of the same color may appear in the three pointers. The semantic segmentation model cannot distinguish the pointers of the same color, and it is necessary to add connected domain analysis to distinguish the pointers of the same color. Summary of the invention

[0005] The object of the present invention is to provide a universal pointer instrument panel reading method which can improve the accuracy of instrument reading and the applicability of instrument readings with different pointer numbers.

[0006] To achieve the above object, the technical solution of the present invention is: a universal pointer instrument panel reading method, characterized in that the reading method steps are as follows:

[0007] S1. The dial area, pointer area and center area of ​​the affine-corrected instrument image are segmented by the improved Yolov8 instance segmentation network model, and the scale start point, scale end point and centroid point of the dial area and the center area are calculated; the improved Yolov8 instance segmentation network model is improved based on the yolov8n-seg network model, and the ConvFormer module and TransFormer module in the MetaFormer of the Transformer architecture are used to replace the C2f module in the yolov8n-seg network model to improve the network's ability to extract global features;

[0008] S2. Calculate the angle between the scale start point, the scale end point and the centroid of the central area to obtain the angle corresponding to the maximum range of the instrument;

[0009] S3, determining the number of pointers and needle handles according to the instance segmentation result;

[0010] S4, calculating the pointer and the point it points to according to the number of pointers;

[0011] S5. Calculate the angle and reading corresponding to the pointer according to the actual pointer pointing point obtained in step S4.

[0012] The step S1 includes steps S1.1 and S1.2 before the step S1;

[0013] S1.1, the instrument area in the instrument image is identified by introducing the Star Block module based on the Yolov8 object detection network model, and the image of the area of ​​interest is cut out;

[0014] S2.2, perform SIFT feature point recognition on the image of the region of interest and the preset instrument template image, and perform affine transformation correction to obtain an affine-corrected instrument image, which is the instrument image in step S1.

[0015] Furthermore, the lightweight Yolov8 target detection network model obtained by improving the Yolov8 target detection network model in step S1.1 is as follows: specifically, the Yolov8 target detection network is divided into three parts: a backbone network, a neck and a head. The backbone network is used to extract features in the image, the neck is used to fuse features of different levels, and the head is divided into three detection heads for detecting large, medium and small objects respectively. Each detection head outputs the x-coordinate, y-coordinate, width, height and category of the center point corresponding to each object respectively. The C2f module in the backbone network is changed to the Star Block module. The Star Block module consists of a depthwise separable convolution and a fully connected layer, wherein the depthwise separable convolution is divided into a separable convolution and a pointwise convolution. The fully connected layer is used to increase the number of feature layers. The Star Block module fuses the features of two linear transformations by element-wise multiplication.

[0016] Further, the calculation method for calculating the angle in step S2 is as follows:

[0017] Calculates the angle between two vectors , the formula is , where represents a directed line segment from the centroid to the starting point of the scale. represents a directed line segment from the centroid to the end point of the scale.

[0018] The radian Convert to Angle , the conversion formula is ,

[0019] If the angle measured from the starting point of the scale When the angle exceeds 180 degrees, To convert, the conversion formula is , where , Indicates the coordinates of the farthest point from the center point. Represents the slope of the line connecting the scale starting point and the centroid. represents the intercept of the line connecting the starting point of the scale and the center of mass,

[0020] Slope The calculation formula is , where , Indicates the coordinates of the starting point of the scale. , represents the coordinates of the centroid,

[0021] intercept The calculation formula is ;

[0022] Furthermore, the needle handle determined in step S3 is calculated by fitting the minimum rotation frame operator to calculate the aspect ratio of the respective rotation frames of all the pointers segmented by the instance of step S3, and the needle handle with the largest aspect ratio is the needle handle;

[0023] Furthermore, the point pointed by the pointer calculated in step S4 is obtained by calculating the point farthest from the centroid as the pointed point;

[0024] Furthermore, the angle and reading corresponding to the pointer calculated in step S5 can be obtained by calculating the angle formed by the line connecting the starting point of the scale and the center of mass point and the line connecting the center of mass point and the point at which the pointer is farthest from the center of mass point, so as to obtain the angle value pointed by the pointer. The reading of the pointer can be obtained by calculating the angle value pointed by the pointer, dividing it by the angle value corresponding to the maximum range of the instrument and then multiplying it by the maximum range, so as to obtain the reading indicated by the pointer.

[0025] Further, in step S4, the pointer and the point it points to are calculated based on the number of pointers, including as follows: if the number of pointers is single, the pointer is the actual pointer; if the number of pointers is double, the point farthest from the centroid of each pointer is calculated, and the pointer corresponding to the shortest distance obtained by the distance from the farthest point of each pointer to the line connecting the center of the needle handle and the centroid is the actual pointer, and its point farthest from the centroid is used as the pointing point; in step S7, the angle and reading corresponding to the second pointer other than the actual pointer are also calculated, and the angle formed by calculating the line connecting the starting point of the scale and the centroid and the line connecting the centroid and the point farthest from the centroid of the second pointer from the centroid can be obtained, and the reading of the second pointer can be obtained by calculating the angle value pointed to by the second pointer, dividing it by the angle value corresponding to the maximum range of the instrument and then multiplying it by the maximum range.

[0026] Further, it also includes the following: if there are three pointers, a preset pointer is selected by obtaining color information, and its point farthest from the center of mass is calculated as the pointing point, and the points farthest from the center of mass of the other two pointers are calculated. The pointer corresponding to the shortest distance obtained by the distance from the farthest point of each of the other two pointers to the line connecting the center of the needle handle and the center of mass is the actual pointer, and the point farthest from the center of mass of each of the other two pointers is used as the pointing point; one of the other two pointers is the second pointer, and the other is the third pointer. In the step S5, the angle and reading corresponding to the third pointer are also calculated. The angle formed by calculating the line connecting the starting point of the scale and the center of mass and the line connecting the center of mass and the point farthest from the center of mass of the third pointer can be obtained. The reading of the third pointer can be obtained by calculating the angle value pointed to by the third pointer, dividing it by the angle value corresponding to the maximum range of the instrument and then multiplying it by the maximum range.

[0027] Furthermore, the method of obtaining color information to select a pointer of a preset color is as follows: S4.1, firstly, a pure black image is created, and a contour internal filling operator is used to fill the pure black image with white to generate a mask image; S4.2, the mask image is divided by 255 and multiplied with the affine-corrected instrument image to generate a pointer image with only the pointer; S4.3, the mean function in the numpy library is used to obtain the R, G, B average values ​​of the color inside the pointer in the pointer image, and the color average value of each pointer is saved;

[0028] S4.4. According to the R, G or B value of the preset color, determine that the pointer with the largest value corresponding to the preset color among the average values ​​of R, G, B among all the pointers is the preset pointer.

[0029] Furthermore, there are three pointers, namely the actual pointer and the upper limit pointer and the lower limit pointer. The pointer of the preset color is the upper limit pointer or the lower limit pointer. The upper limit pointer and the lower limit pointer are used to indicate the upper and lower limit values ​​of the current instrument settings. When there are three pointers in step S4, step S7 is followed by step S6. Step S6 determines whether the actual pointer reading is within the limit range of the upper limit pointer and the lower limit pointer based on the angle corresponding to the maximum range of the instrument obtained in step S2 and the angles corresponding to the actual pointer, the lower limit pointer and the upper limit pointer.

[0030] Furthermore, the shortest distance is obtained by the distance from the farthest point of each pointer to the line connecting the center of the needle handle and the center of mass, which is denoted as d. The calculation expression is: , where x 0 ,y 0 Indicates the distance between each pointer and the centroid C 1 Coordinates of the farthest point, x 1 ,y 1 Represents the centroid point C 1 The coordinates, x 2 ,y 2 Indicates needle handle B 4 The coordinates of the center.

[0031] Furthermore, the readings indicated by the actual pointer, the second pointer, and the third pointer are recorded as , the calculation expression is , where Indicates the angle value pointed by each pointer. Indicates the angle value corresponding to the maximum range of the instrument. Indicates the maximum measuring range of the instrument.

[0032] By adopting the above technical scheme, the beneficial effects of the present invention are as follows: the present invention proposes an instrument reading scheme based on the combination of Yolov8 target detection network model and Yolov8 instance segmentation network model, which can be used for common single, double and triple pointer instrument reading recognition. The method of the present invention does not need to mark the scale area, reducing the more cumbersome process of marking the scale area, and only needs to mark the approximate area of ​​the dial, and can ensure that the position of the scale starting point and the scale end point is at the boundary. In addition, the instance segmentation adopted can represent the three pointers with different instances respectively, and there will be no situation where different objects in semantic segmentation belong to the same category. The method can further judge which color pointer it is by the color of the pointer area as above, and can judge which one is the actual pointing pointer according to the position of the pointer and the needle handle, so as to more accurately and reliably perform multi-pointer instrument reading. Therefore, the method can not only simplify the marking process of the instrument scale area, reduce the dependence on network accuracy, but also effectively deal with the reading use of multi-pointer instruments, and improve the accuracy and applicability of instrument reading. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 The invention discloses a flow chart of a universal pointer instrument panel reading method.

[0034] Figure 2 It is a structural schematic diagram of the lightweight Yolov8 target detection network model involved in the present invention.

[0035] Figure 3 It is a structural schematic diagram of the Star Block module involved in the present invention.

[0036] Figure 4 It is a structural schematic diagram of the improved Yolov8 instance segmentation network model involved in the present invention.

[0037] Figure 5 It is a structural diagram of the ConvFormer and Transformer modules involved in the present invention.

[0038] Figure 6 This is an original instrument image in the example test according to the present invention.

[0039] Figure 7 This is a region of interest (ROI) image in an example experiment according to the present invention.

[0040] Figure 8 This is an instrument image subjected to affine correction in an example test according to the present invention.

[0041] Fig. 9 This is a template instrument image used in the example test according to the present invention.

[0042] Fig.10 It is a schematic diagram of an instrument for calculating the aspect ratio of a fitting rotating frame in an embodiment test of the present invention.

[0043] Fig.11 This is a schematic diagram of the mask processing according to the present invention.

[0044] Fig.12 It is a schematic image of the meter readings involved in the present invention.

[0045] Fig.13 It is a schematic diagram of calculating the distance d involved in the present invention. DETAILED DESCRIPTION

[0046] In order to further explain the technical solution of the present invention, the present invention is described in detail below through specific embodiments.

[0047] This embodiment discloses an embodiment of a universal pointer instrument panel reading method of the present invention. The process steps of the method of this embodiment are as follows: Figure 1 As shown, the following steps are included:

[0048] S1. Collecting an instrument image, that is, an image of the instrument whose reading is to be identified, a visible light image.

[0049] S2. The instrument area in the instrument image is identified through the lightweight Yolov8 target detection network model, the instrument information is obtained, and the region of interest (ROI) image is cut out from the instrument area through the instrument information.

[0050] The lightweight Yolov8 target detection network model is improved based on the Yolov8 target detection network model. Figure 2 As shown, the Yolov8 target detection network is divided into three parts: backbone network, neck and head. The backbone network is used to extract features in the image, the neck is used to fuse features at different levels to facilitate better acquisition of objects in the future, and the head is divided into three detection heads to detect large, medium and small objects respectively. Each detection head outputs the x-coordinate, y-coordinate, width, height and category of the center point corresponding to each object. In order to reduce the parameters of the model and make the Yolov8 target detection network more suitable for deployment on resource-constrained devices (such as embedded devices or edge computing devices), the present invention changes the C2f module in the backbone network to the Star Block module, such as Figure 3As shown, the Star Block module consists of a depth-separable convolution (DW-Conv) and a fully connected layer (FC), wherein the depth-separable convolution is divided into a separation convolution and a point-by-point convolution. Such a convolution module significantly reduces the number of parameters and the amount of computation. The fully connected layer is used to increase the number of feature layers to compensate for the reduced accuracy caused by the depth-separable convolution. The U part in the figure further fuses the features of the two linear transformations by element multiplication, thereby further enhancing the ability of feature extraction. In this way, an image is input through the above model, which can correspond to the output of the center point coordinates and width and height information of the object, and the module can significantly increase the reasoning speed while ensuring a certain accuracy. In the experiment of the present invention, the above model is trained and verified through multiple (9000 images collected in the experiment) instrument images containing instruments and processed by annotation, classification, etc., wherein the mAP@0.5 of the verification set can reach 100%, which can prove that the instrument is easy to identify in the application of the above model.

[0051] S3, performing SIFT feature point recognition, feature point matching, affine transformation and image correction processing on the region of interest image and the preset instrument template image to obtain an affine-corrected instrument image.

[0052] As in the experiment of this embodiment, Figure 6 The original instrument image input shown in the figure is used to identify and cut out the region of interest (ROI) through the lightweight Yolov8 target detection network model. Figure 7 The instrument image shown is compared with the preset instrument template image (such as Fig. 9 ) to perform SIFT feature point recognition and feature point matching, select the most representative 4 pairs of feature points, calculate the affine transformation matrix, and perform geometric correction on the current instrument image based on the matrix to eliminate deformation such as rotation, translation or scaling to obtain the instrument image after affine transformation, as shown in Figure 8 shown.

[0053] S4. Use the improved Yolov8 instance segmentation network model to perform instance segmentation of the dial area A, pointer area B, and center area C on the affine-corrected instrument image, and calculate the scale starting point A of the dial area A. 1 , scale end point A 2 and the centroid C of the central area C 1 In the application field, there may be single-pointer, double-pointer, and triple-pointer instruments, among which the three-pointer instrument has the highest complexity. In the embodiment of the present invention, the pointer area B may include a lower limit pointer B 1 、Actual pointer B 2 and upper limit pointer B 3 , and needle handle B 4, to adapt to the reading application of multi-pointer instruments, the lower limit pointer B 1 and upper limit pointer B 3 Used to indicate the upper and lower limit values ​​of the current instrument settings.

[0054] The improved Yolov8 instance segmentation network model is improved based on the yolov8n-seg network model, such as Figure 4 As shown in FIG. 1 , the ConvFormer module and the Transformer module in the MetaFormer of the Transformer architecture are used to replace the C2f module in the yolov8n-seg network model, which can improve the network's ability to extract global features and thus improve the accuracy of identifying the dial and pointer. The structural diagram of the ConvFormer Block module and the Transformer Block module is shown in FIG. Figure 5 As shown in the figure, since the Tranformer architecture adopts the self-attention mechanism, it can realize long-range dependency, that is, extract the global features of the visual scene, and solve the problem that the dial area A and the pointer area B often occupy the entire instrument space, and the original yolov8n-seg network model can only capture local information but cannot establish long-distance connections of the global image.

[0055] S5. Calculate the scale starting point A based on the dial position information of step S4 1 , scale end point A 2 and the centroid C of the central area C 1 The angle between them can be used to know the angle corresponding to the maximum range of the instrument. , the angle is calculated using the following method:

[0056] Calculate the angle between two vectors , the calculation formula is , where Represents the point C from the center of mass 1 Point to the starting point A 1 A directed line segment of Represents the point C from the center of mass 1 Point to the end point A 2 A directed line segment of

[0057] The radian Convert to Angle , the conversion formula is ,

[0058] Since the range of the arccosine value is [0,180], if we start from the scale starting point A 1 Start angle When it exceeds 180 degrees (that is, the actual pointer B1 The point is below the connecting line), and the angle Make a conversion, the conversion formula is , where , Indicates the coordinates of the point farthest from the center point of the actual pointer. Indicates the scale starting point A 1 With the centroid C 1 The slope of the line connecting Indicates the scale starting point A 1 With the centroid C 1 The intercept of the line connecting

[0059] Slope The calculation formula is , where , Indicates the scale starting point A 1 The coordinates of , Represents the centroid point C 1 The coordinates of

[0060] intercept The calculation formula is .

[0061] S6. Determine the number of pointers and needle handles B based on the instance segmentation results of step S4. 4 .

[0062] Determine the needle handle B 4 The purpose is to determine which pointer is the current pointer by the needle handle, because usually needle handle B 4 The pointer part is shorter and thicker, so the needle handle B 4 The aspect ratio is closer to 1 (such as Fig.10 Middle needle handle B 4 The aspect ratio of the pointer is closer to 0 (such as Fig.10 The aspect ratios of the remaining pointer regions B are 0.1, 0.1, and 0.09 respectively). Therefore, the aspect ratios of the rotation frames of all the pointer regions B segmented by the instance of step S4 are calculated by fitting the minimum rotation frame operator cv2.minAreaRect. The one with the largest aspect ratio is the handle B. 4 The number of pointers can be calculated by subtracting one pointer handle B from the number of all pointer areas B that are segmented. 4 Area, the number of remaining pointer areas B is the number of pointers.

[0063] S7, calculate the actual pointer B according to the number of pointers determined in step S6 2 Pointing point.

[0064] If it is determined that the number of pointers is single, then except for handle B 4 Pointer area B outside is the actual pointer B 2 , calculate the actual pointer B 2 From the center of mass C 1 The farthest point, as the actual pointer B 2 Point to point.

[0065] If it is determined that the number of pointers is double, then through the needle handle B 4 Center and centroid C 1 Connect the lines to get a straight line to calculate the difference between needle handle B 4 The pointer area B outside the center of mass C 1 The distance d between the farthest point and the line (e.g. Fig.13 As shown in the figure), the pointer area B corresponding to the shortest distance d is the actual pointer B 2 , the other is the lower limit pointer B 1 Or upper limit pointer B 3 , the actual pointer B 2 From the center of mass C 1 The farthest point is taken as the actual pointer B 2 Pointing point, lower limit pointer B 1 Or upper limit pointer B 3 From the center of mass C 1 The farthest point is used as the lower limit pointer B 1 Or upper limit pointer B 3 Point to point.

[0066] If it is determined that there are three pointers, remove the pointer handle B. 4 In all pointer areas B except for the above, the method of obtaining color information is used to determine whether the pointer B is the same as the known lower limit pointer B. 1 Or upper limit pointer B 3 The pointer corresponding to the pointer color is the corresponding lower limit pointer B 1 Or upper limit pointer B 3 , here it is customized as the pre-selected pointer (such as the known upper limit pointer B 3 It is red, and the preselected pointer here is the upper limit pointer B 3 ), the rest of the pointer area B is the actual pointer B 2 Or lower limit pointer B different from preselected pointer 1 Or upper limit pointer B 3 , here the lower limit pointer B is different from the preselected pointer 1 Or upper limit pointer B 3 Customized as the remainder pointer (as known above, it is the upper limit pointer B 3 , where the remainder is the lower limit pointer B 1 ), through the needle handle B 4 Center and centroid C 1Connect the lines to get a straight line to calculate the difference between needle handle B 4 The remaining pointer areas B are located at the centroid C except for the pre-selected pointer. 1 The distance d between the farthest point and the line (e.g. Fig.13 As shown, the farthest points of the remaining two pointers in area B are away from the centroid point C 1 The distances d are 230.23 pixels and 10.25 pixels respectively, where pixel is the unit pixel). The actual pointer B has the shortest distance d. 2 , the actual pointer B 2 From the center of mass C 1 The farthest point is taken as the actual pointer B 2 Points to point, the other is the remainder pointer, the remainder pointer is away from the center of mass point C 1 The farthest point is used as the remaining pointer to point, and the distance from the pre-selected pointer to the centroid point C is calculated. 1 The farthest point is used as the preselected pointer point.

[0067] The calculation formula of the distance d is: , where x 0 ,y 0 Indicates the distance between each pointer and the centroid C 1 Coordinates of the farthest point, x 1 ,y 1 Represents the centroid point C 1 The coordinates, x 2 ,y 2 Indicates needle handle B 4 The coordinates of the center.

[0068] As in the three pointer readings of the test of the present invention, the actual pointer B 2 The black pointer, upper limit pointer B 3 The red pointer, the lower limit pointer B 1 The black pointer is used to judge the red upper limit pointer B. 3 Set as the above pre-selected pointer, the method of obtaining color information is as follows:

[0069] S7.1. First, create a pure black image, and use the contour internal filling operator cv2.fillPoly to fill the pure black image (RGB value (0,0,0)) with white (RGB value (255,255,255)) to generate a mask image.

[0070] S7.2, divide the mask image by 255 and multiply it with the affine-corrected instrument image to generate a pointer image with only the pointer, such as Fig.11 As shown;

[0071] S7.3. Use the mean function in the numpy library to obtain the R, G, and B average values ​​of the colors inside the pointer in the pointer image, and save the color average value of each pointer;

[0072] S7.4. The red pointer is determined based on the maximum R value of the average value of R, G, and B of the color in the area, which is the upper limit pointer B. 3 .

[0073] S8, the actual pointer B obtained according to step S7 2 , lower limit pointer B 1 and upper limit pointer B 3 The actual pointer B is calculated from the point pointed to 2 , lower limit pointer B 1 and upper limit pointer B 3 The corresponding angle and actual pointer B 2 , lower limit pointer B 1 and upper limit pointer B 3 The reading is based on the angle corresponding to the maximum range of the instrument obtained in step S5 and the actual pointer B 2 , lower limit pointer B 1 and upper limit pointer B 3 The corresponding angle determines the actual pointer B 2 Check whether the reading exceeds the maximum range of the instrument. If it exceeds, it is abnormal; if it does not exceed, it is normal.

[0074] The actual pointer B in this step 2 , lower limit pointer B 1 and / or upper limit pointer B 3 The corresponding angle is calculated by the scale starting point A 1 With the centroid C 1 Connect the line and the centroid point C 1 The distance between each pointer and the center of mass point C 1 The angle formed by the line connecting the farthest points can be used to obtain the angle value pointed by each pointer. The reading of each pointer is calculated by calculating the angle value pointed by each pointer Divide by the angle value corresponding to the maximum range of the instrument Multiply by the maximum range , you can get the readings pointed by each pointer , that is, the formula is .like Fig.12 The diagram shows the reading of a universal pointer instrument. The dot in the lower left corner indicates the starting point A of the scale. 1 ; The dot in the lower right corner indicates the end point A 2 ; The point in the center represents the centroid C 1 ; The lower trapezoidal block in the middle represents the needle handle B 4 ; The left pointer among the three pointers on the upper left, middle and right indicates the lower limit pointer B1 ; The middle pointer represents the actual pointer B 2 ; The pointer on the right indicates the upper limit pointer B 3 , the actual pointer B obtained from the instrument in the figure 2 The reading is 1.23, the upper limit pointer B 3 The reading is 1.34, the lower limit pointer B 1 The reading is 0.53, so the meter is within the limit and the report is normal.

[0075] The above embodiments and drawings do not limit the product form and style of the present invention. Any appropriate changes or modifications made by ordinary technicians in the relevant technical field should be deemed to be within the patent scope of the present invention.

Claims

1. A universal pointer instrument panel reading method, characterized in that: The reading method steps are as follows: S1. Perform instance segmentation of the dial area, pointer area and center area of ​​the instrument image through the improved Yolov8 instance segmentation network model, and calculate the scale starting point, scale end point and centroid point of the dial area and the center area; the improved Yolov8 instance segmentation network model is improved based on the yolov8n-seg network model, and the ConvFormer module in the MetaFormer of the Transformer architecture is used to replace the first two layers of C2f modules of the backbone network in the yolov8n-seg network model, and the TransFormer module in the MetaFormer of the Transformer architecture is used to replace the C2f modules of the last two layers of the backbone network in the yolov8n-seg network model to improve the network's ability to extract global features; S2. Calculate the angle between the scale start point, the scale end point and the centroid of the central area to obtain the angle corresponding to the maximum range of the instrument; S3, determining the number of pointers and needle handles according to the instance segmentation result; S4, calculating the pointer and the point it points to according to the number of pointers; If the number of pointers is single, the pointer is the actual pointer, and its point farthest from the centroid is the pointing point; If the number of pointers is double, calculate the farthest point of each pointer from the centroid, and the pointer corresponding to the shortest distance from the farthest point of each pointer to the line connecting the center of the needle handle and the centroid is the actual pointer, and its farthest point from the centroid is the pointing point; If there are three pointers, a pointer of a preset color is selected by obtaining color information, and its point farthest from the center of mass is calculated as the pointing point, and the points farthest from the center of mass of the other two pointers are calculated. The pointer corresponding to the shortest distance obtained by the distance from the farthest point of each of the other two pointers to the line connecting the center of the needle handle and the center of mass is the actual pointer, and the points farthest from the center of mass of each of the other two pointers are used as the pointing points; S5. Calculate the angle and reading corresponding to each pointer according to the pointer pointing point obtained in step S4.

2. A universal pointer instrument panel reading method as claimed in claim 1, characterized in that: The step S1 includes steps S1.1 and S1.2 before the step S1; Step S1.1, identifying the instrument area in the instrument image by introducing the Star Block module based on the Yolov8 object detection network model to obtain a lightweight Yolov8 object detection network model, and cutting out the image of the area of ​​interest; S1.

2. Perform SIFT feature point recognition on the image of the region of interest and the preset instrument template image, and perform affine transformation correction to obtain an affine-corrected instrument image, which is the instrument image in step S1.

3. A universal pointer instrument panel reading method as claimed in claim 2, characterized in that: The lightweight Yolov8 target detection network model obtained by improving the Yolov8 target detection network model in step S1.1 is as follows: specifically, the Yolov8 target detection network is divided into three parts: a backbone network, a neck and a head. The backbone network is used to extract features in the image, the neck is used to fuse features at different levels, and the head is divided into three detection heads for detecting large, medium and small objects respectively. Each detection head outputs the x-coordinate, y-coordinate, width, height and category of the center point corresponding to each object respectively. The C2f module in the backbone network is changed to the Star Block module. The Star Block module consists of a depthwise separable convolution and a fully connected layer, wherein the depthwise separable convolution is divided into a separable convolution and a pointwise convolution. The fully connected layer is used to increase the number of feature layers. The features of two linear transformations are fused by element-wise multiplication in the Star Block module.

4. A universal pointer instrument panel reading method as claimed in claim 1, 2 or 3, characterized in that: The calculation method for calculating the angle in step S2 is as follows: Calculates the angle between two vectors , the formula is , where represents a directed line segment from the centroid to the starting point of the scale. The directed line segment from the centroid to the end point of the scale is shown. Convert to Angle , the conversion formula is , if the angle from the starting point of the scale is When the angle exceeds 180 degrees, To convert, the conversion formula is , where , Indicates the coordinates of the point farthest from the center of mass of the pointer. Represents the slope of the line connecting the scale starting point and the centroid. Indicates the intercept and slope of the line connecting the starting point of the scale and the center of mass The calculation formula is , where , Indicates the coordinates of the starting point of the scale. , Represents the coordinates of the centroid, the intercept The calculation formula is ; In step S3, the needle handle is determined by fitting the minimum rotation frame operator to calculate the aspect ratio of the rotation frames of all the pointers segmented in step S1, and the needle handle with the largest aspect ratio is the needle handle; The angle and reading corresponding to the pointer calculated in step S5 are obtained by calculating the angle formed by the line connecting the starting point of the scale and the center of mass point and the line connecting the center of mass point and the point at which the pointer is farthest from the center of mass point, so as to obtain the angle value pointed by the pointer. The reading of the pointer is obtained by dividing the angle value pointed by the pointer by the angle value corresponding to the maximum range of the instrument and then multiplying it by the maximum range to obtain the reading indicated by the pointer.

5. A universal pointer instrument panel reading method according to any one of claims 1 to 3, characterized in that: The specific method of obtaining color information to filter out a pointer of a preset color is as follows: S4.

1. First, create a pure black image, and use the contour interior filling operator to fill the pure black image with white to generate a mask image; S4.2, dividing the mask image by 255 and multiplying the mask image with the affine-corrected instrument image to generate a pointer image with only the pointer; S4.

3. Use the mean function in the numpy library to obtain the R, G, and B average values ​​of the color inside the pointer in the pointer image, and save the color average value of each pointer; S4.

4. According to the R, G or B value of the preset color, determine that the pointer with the largest value corresponding to the preset color among the average values ​​of R, G, B among all the pointers is the preset pointer.

6. A universal pointer instrument panel reading method as claimed in claim 4, characterized in that: There are three pointers, namely the actual pointer, the upper limit pointer and the lower limit pointer. The pointer of the preset color is the upper limit pointer or the lower limit pointer. The upper limit pointer and the lower limit pointer are used to indicate the upper and lower limit values ​​of the current instrument settings. When the number of pointers in step S4 is three, step S5 is followed by step S6. Step S6 determines whether the actual pointer reading is within the limit range of the upper limit pointer and the lower limit pointer based on the angle corresponding to the maximum range of the instrument obtained in step S2 and the angles corresponding to the actual pointer, the lower limit pointer and the upper limit pointer.

7. A universal pointer instrument panel reading method according to any one of claims 1 to 3, characterized in that: The shortest distance is obtained by the distance from the farthest point of each pointer to the line connecting the center of the needle handle and the center of mass, which is recorded as d. The calculation expression is: , where x0 and y0 represent the coordinates of the points of each pointer farthest from the center of mass point C1, x1 and y1 represent the coordinates of the center of mass point C1, and x2 and y2 represent the coordinates of the center of needle handle B4.

8. A universal pointer instrument panel reading method according to any one of claims 1 to 3, characterized in that: The readings indicated by the actual pointer, the second pointer, and the third pointer are recorded as , where Indicates the angle value pointed by each pointer. Indicates the angle value corresponding to the maximum range of the instrument. Indicates the maximum measuring range of the instrument.

Images