Pointer type instrument reading identification method and device and terminal equipment

Abstract

The invention is suitable for the technical field of instrument identification, and provides a pointer type instrument reading identification method and device and terminal equipment. The method comprises the following steps: segmenting scales and a pointer of the pointer instrument according to an image of the pointer instrument to obtain a plurality of scale lines and a pointer; performing text recognition on the pointer type instrument image to obtain a plurality of scale values; according to the feature information of the scale values and the feature information of the scale lines, the scale values and the scale lines are matched, and the scale values corresponding to the scale lines are determined; fitting the scale line and the pointer respectively to obtain a scale fitting circle and a pointer fitting straight line; according to the scale fitting circle, the pointer fitting straight line and the scale value corresponding to each scale, determining an intersection point of the pointer fitting straight line and the scale fitting circle and a scale interval where the pointer is located; and determining the reading of the pointer type instrument according to the scale interval of the pointer and the intersection point. According to the invention, accurate segmentation of the pointer and the scale can be realized, and the reading of the instrument can be accurately calculated.

Description

Technical Field

[0001] The present application relates to the technical field of meter identification, and in particular to a method, device and terminal equipment for identifying readings of pointer-type meters. Background Art

[0002] As an important tool in the field of measurement and control, instruments are widely used in different fields. Common instruments in power systems include pressure gauges, oil temperature gauges, ammeters, etc. The traditional way is to rely on people to read the readings, which is time-consuming and laborious, so it is necessary to design a way to read the readings automatically. Domestic and foreign scholars have conducted a lot of research on the automatic reading of digital instrument scales, and most of the research is based on traditional image processing methods, such as canny detection of instrument contour information, HoughTransform detection of circular instruments, and least squares fitting of pointers and scales. The traditional algorithm detects the pointer and scale through the threshold, and then reads the pointer reading through the arc. However, this algorithm has poor generalization ability and is easily affected by angles, lighting, etc., resulting in unreliable recognition results. Summary of the invention

[0003] In view of this, the embodiments of the present application provide a pointer-type instrument reading recognition method, device and terminal device, which can achieve precise segmentation of the pointer and scale, and can accurately calculate the instrument reading, and show good robustness and generalization ability in the recognition of different types of instrument readings.

[0004] In order to achieve the above objectives, this application adopts the following technical solutions:

[0005] In a first aspect, an embodiment of the present application provides a method for identifying a pointer meter reading, comprising:

[0006] Segmenting the scale and the pointer of the pointer instrument according to the pointer instrument image to obtain a plurality of scale lines and a pointer;

[0007] Performing text recognition on the pointer instrument image to obtain multiple scale values;

[0008] According to the characteristic information of the scale values ​​and the characteristic information of the scale lines, the multiple scale values ​​are matched with the multiple scale lines to determine the scale values ​​corresponding to the respective scale lines; wherein the characteristic information of the scale values ​​includes the position, angle and direction of the scale values, and the characteristic information of the scale lines includes the position, angle and direction of the scale lines;

[0009] Fitting the multiple scale lines and the one pointer respectively to obtain a scale fitting circle and a pointer fitting straight line;

[0010] According to the scale fitting circle, the pointer fitting straight line and the scale values ​​corresponding to each scale, determining the intersection point of the pointer fitting straight line and the scale fitting circle and the scale interval where the pointer is located;

[0011] The reading of the pointer instrument is determined according to the scale interval where the pointer is located and the intersection point.

[0012] Based on the first aspect, in some embodiments, matching the multiple scale values ​​with the multiple scale lines according to the characteristic information of the scale values ​​and the characteristic information of the scale lines to determine the scale values ​​corresponding to the respective scale lines includes:

[0013] Determine the first position coordinate x of the i-th scale value i and i , and the first angle θ i and the first direction v i ;

[0014] Determine the second position coordinate x of the jth tick mark j and j , and the second angle θ j and the first direction v j ;

[0015] According to the formula Calculate the characteristic information c of the i-th scale value i and the characteristic information c of the jth scale line j The Euclidean distance between

[0016] According to the formula Calculate the similarity s between the i-th scale value and the j-th scale line ij , σ is c i With c j The standard deviation of

[0017] according to Calculate the weight between the i-th scale value and the j-th scale line, and obtain the weight between each scale value and all scale lines;

[0018] The scale line with the largest weight between the i-th scale value is matched, thereby obtaining a scale line matching each scale value.

[0019] Based on the first aspect, in some embodiments, fitting the multiple scale lines and the one pointer respectively to obtain a scale fitting circle and a pointer fitting straight line includes:

[0020] Determine the center point of each tick mark;

[0021] The center point of each scale line is fitted using the least square method to obtain the scale fitting circle;

[0022] Performing Hough transformation on the pointer to generate the pointer fitting straight line, wherein the pointer fitting straight line intersects with the scale fitting circle at two points.

[0023] Based on the first aspect, in some embodiments, determining the intersection of the pointer fitting line and the scale fitting circle and the scale interval where the pointer is located according to the scale fitting circle, the pointer fitting line and the scale values ​​corresponding to each scale, includes:

[0024] Determine the target point on the pointer that is farthest from the center of the scale fitting circle;

[0025] Determine two intersection points of the scale fitting circle and the pointer fitting straight line, and determine a target intersection point closest to the target point among the two intersection points;

[0026] The distance between the center point of each scale line and the target intersection is calculated, and the scale interval formed by the scale lines corresponding to the two smallest distances is used as the scale interval where the pointer is located.

[0027] Based on the first aspect, in some embodiments, determining the reading of the pointer instrument according to the scale interval where the pointer is located and the intersection point includes:

[0028] Determine the starting position and the ending position of the scale interval;

[0029] Calculating a first arc α between the starting position and the ending position, and a second arc β between the starting position and the target intersection point;

[0030] according to Determine the reading of pointer instruments;

[0031] Where value is the reading of the pointer instrument, maxvalue is the scale value corresponding to the end position of the scale interval, minvalue is the scale value corresponding to the start position of the scale interval, and x1 and y1 are the coordinates of the starting position of the scale interval, x2 and y2 are the coordinates of the ending position of the scale interval, x3 and y3 are the coordinates of the target intersection, x c and c are the coordinates of the center of the scale fitted circle.

[0032] Based on the first aspect, in some embodiments, before the step of segmenting the scale and the pointer of the pointer instrument according to the pointer instrument image, the pointer instrument reading recognition method further includes:

[0033] Acquire an original image, wherein the original image includes a pointer instrument;

[0034] Determine the position of the pointer instrument from the original image based on the improved YOLOv9 model, and extract the pointer instrument image;

[0035] Among them, the P3 feature map and the conv-reg module, conv-cls module and concat module connected to the P3 feature map are deleted in the improved YOLOv9 model;

[0036] The process of determining the ahchor parameters in the conv-reg module in the P4 feature map and the P5 feature map of the improved YOLOv9 model is as follows: obtaining the annotated meter sizes in multiple pointer meter images, calculating the multiple meter sizes by the mean algorithm, and obtaining the ahchor box that can represent the pointer meter.

[0037] Based on the first aspect, in some embodiments, segmenting the scale and the pointer of the pointer instrument according to the pointer instrument image to obtain a plurality of scale lines and a pointer includes:

[0038] The pointer instrument image is segmented by the U2Net network algorithm to obtain multiple scale lines and a pointer;

[0039] Among them, only the En_1 layer, En_3 layer, En_5 layer and En_6 layer of the U2Net network algorithm are used to extract the scale line features and pointer features in the pointer instrument image.

[0040] Based on the first aspect, in some embodiments, performing text recognition on the pointer instrument image to obtain multiple scale values ​​includes:

[0041] Extracting features of the pointer meter image through convolution and sampling operations to obtain a feature map;

[0042] Performing a deconvolution operation on the feature map to obtain a probability map;

[0043] Processing the feature map to determine a pixel threshold corresponding to each pixel in the feature map;

[0044] According to the pixel threshold corresponding to each pixel point in the feature map, a differentiable binarization operation is performed on the probability map to obtain a binarized map;

[0045] Based on the binarized image, the pointer instrument image is segmented into text regions to obtain a text image region containing text content;

[0046] Text recognition is performed on each text image area to obtain multiple scale values.

[0047] In a second aspect, an embodiment of the present application provides a pointer-type meter reading recognition device, comprising:

[0048] A segmentation module is used to segment the scale and pointer of the pointer instrument according to the pointer instrument image to obtain multiple scale lines and a pointer;

[0049] A text recognition module, used for performing text recognition on the pointer instrument image to obtain multiple scale values;

[0050] A matching module, used for matching the multiple scale values ​​with the multiple scale lines according to the characteristic information of the scale values ​​and the characteristic information of the scale lines, and determining the scale values ​​corresponding to the respective scale lines; wherein the characteristic information of the scale values ​​includes the position, angle and direction of the scale values, and the characteristic information of the scale lines includes the position, angle and direction of the scale lines;

[0051] A fitting module, used for fitting the multiple scale lines and the one pointer respectively to obtain a scale fitting circle and a pointer fitting straight line;

[0052] A scale interval determination module, used to determine the intersection of the pointer fitting line and the scale fitting circle and the scale interval where the pointer is located according to the scale fitting circle, the pointer fitting line and the scale values ​​corresponding to each scale;

[0053] The reading determination module is used to determine the reading of the pointer instrument according to the scale interval where the pointer is located and the intersection point.

[0054] In a third aspect, an embodiment of the present application provides a terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the pointer meter reading identification method as described in any one of the above-mentioned first aspects are implemented.

[0055] The beneficial effects of the embodiments of the present application compared with the prior art are as follows:

[0056] The above-mentioned method for identifying the reading of a pointer-type instrument firstly segments the scale and pointer of the pointer-type instrument according to the image of the pointer-type instrument to obtain a plurality of scale lines and a pointer, and performs text recognition on the image of the pointer-type instrument to obtain a plurality of scale values. Afterwards, the scale values ​​and scale lines are matched according to the characteristic information of the scale values ​​and the characteristic information of the scale lines. Next, the scale lines and the pointer are fitted respectively to obtain a scale fitting circle and a pointer fitting line. According to the scale fitting circle, the pointer fitting line and the scale values ​​corresponding to each scale, the intersection of the pointer fitting line and the scale fitting circle and the scale interval where the pointer is located are determined. Finally, according to the scale interval and the intersection where the pointer is located, the reading of the pointer-type instrument is determined. The embodiment of the present application can realize the precise segmentation of the pointer and the scale lines, and can accurately calculate the reading of the instrument, and has good robustness and generalization ability. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0058] Figure 1 It is a flow chart of a method for identifying a pointer meter reading provided in an embodiment of the present application;

[0059] Figure 2 is a schematic diagram of the divided scale lines and pointers provided in an embodiment of the present application;

[0060] Figure 3 is a schematic diagram of the matching effect of scale lines and scale values ​​provided in an embodiment of the present application;

[0061] Figure 4 It is a schematic diagram of a scale fitting circle and a pointer fitting straight line provided in an embodiment of the present application;

[0062] Figure 5 It is a schematic diagram of a scale fitting circle and a pointer fitting straight line presented on a pointer-type instrument provided in an embodiment of the present application;

[0063] Figure 6 It is a schematic diagram of the process of calculating the reading of the pointer meter provided in the embodiment of the present application;

[0064] Figure 7 It is a schematic diagram of the pressure gauge reading recognition effect provided by the embodiment of the present application;

[0065] Figure 8 It is a schematic diagram of the oil level gauge reading recognition effect provided by the embodiment of the present application;

[0066] Fig. 9 It is a schematic diagram of the voltage meter reading recognition effect provided by the embodiment of the present application;

[0067] Fig.10 It is a structural schematic diagram of a pointer-type meter reading recognition device provided in an embodiment of the present application;

[0068] Fig.11 It is a schematic diagram of a terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0069] The present application is described more clearly below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the effects of the present application, but are not intended to limit the present application in any form. It should be noted that, for those of ordinary skill in the art, several variations and improvements may be made without departing from the concept of the present application. These all fall within the scope of protection of the present application.

[0070] In order to make the purpose, technical solutions and advantages of the present application clearer, specific embodiments will be described below in conjunction with the accompanying drawings.

[0071] See also Figure 1 The pointer meter reading recognition method provided in the embodiment of the present application may include the following steps:

[0072] Step 101 , segmenting the scale and pointer of the pointer instrument according to the pointer instrument image to obtain a plurality of scale lines and a pointer.

[0073] In some embodiments, step 101 may include: segmenting the pointer instrument image through the U2Net network algorithm to obtain multiple scale lines and a pointer; wherein only the En_1 layer, En_3 layer, En_5 layer and En_6 layer of the U2Net network algorithm are used to extract scale line features and pointer features in the pointer instrument image.

[0074] Specifically, the U2Net network algorithm uses the RSU module to extract image features. The RSU module can capture more contextual information, integrate features of different scales, and combine residual connections and U-Net structure to effectively improve feature extraction capabilities. Figure 2 The multiple scale lines and one pointer obtained by segmentation. However, the original U2Net network structure has 6 layers, which is relatively redundant and prone to overfitting. In this application, only the En_1, En_3, En_5, and En_6 layers of the U2Net network are used to extract instrument features. On the basis of maintaining recognition accuracy, the reasoning speed is accelerated, making it more efficient in extracting salient areas of the image. By segmenting the pointer instrument image, the precise position of each scale line and pointer on the pointer instrument can be obtained, laying the foundation for subsequent reading recognition.

[0075] Step 102: Perform text recognition on the pointer instrument image to obtain a plurality of scale values.

[0076] In some embodiments, step 102 may include: performing feature extraction on the pointer instrument image through convolution and sampling operations to obtain a feature map; performing a deconvolution operation on the feature map to obtain a probability map, wherein the probability map represents the probability that each pixel point in the feature map is text; processing the feature map to determine a pixel threshold corresponding to each pixel point in the feature map; performing a differentiable binarization operation on the probability map according to the pixel threshold corresponding to each pixel point in the feature map to obtain a binarized map; performing text area segmentation on the pointer instrument image based on the binarized map to obtain a text image area containing text content; performing text recognition on each text image area to obtain multiple scale values.

[0077] Specifically, the process of extracting features from the pointer meter image through convolution and sampling operations to obtain a feature map may include:

[0078] Step A1, performing convolution down-sampling on the pointer instrument image to obtain a first feature layer, a second feature layer, a third feature layer and a fourth feature layer, wherein the size of the fourth feature layer is one quarter of the pointer instrument image, the size of the third feature layer is one eighth of the pointer instrument image, the size of the second feature layer is one sixteenth of the pointer instrument image, and the size of the first feature layer is one thirty-second of the pointer instrument image.

[0079] Step A2: upsample the first feature layer by two times, and then add the pixel values ​​corresponding to the second feature layer to obtain the sixth feature layer. Upsample the sixth feature layer by two times, and then add the pixel values ​​corresponding to the third feature layer to obtain the seventh feature layer. Upsample the seventh feature layer by two times, and then add the pixel values ​​corresponding to the fourth feature layer to obtain the eighth feature layer.

[0080] Step A3, the fifth to eighth feature layers are processed as follows: 3×3 convolution is performed on the eighth feature layer, 3×3 convolution is performed on the seventh feature layer and then upsampled by two times, 3×3 convolution is performed on the sixth feature layer and then upsampled by four times, and 3×3 convolution is performed on the fifth feature layer and then upsampled by eight times. The processed fifth feature layer, sixth feature layer, seventh feature layer and eighth feature layer are connected together through the concat function to generate the above feature map.

[0081] In some embodiments, the above-mentioned process of performing a deconvolution operation on the feature map to obtain a probability map is as follows: step B1, the feature map is first subjected to a convolution layer to compress the channel to 1 / 4 of the input, and then subjected to BN and relu to obtain a feature map whose size is one-fourth of the pointer meter image; step B2, the obtained feature map is subjected to a 2×2 deconvolution operation, and the size of the obtained feature map is one-half of the pointer meter image; step B3, the feature map is subjected to a 2×2 deconvolution operation, the output feature map channel is 1, and the size of the obtained feature map is the same as the pointer meter image; step B4, the feature map obtained in step B3 is processed by a sigmoid function to obtain a probability map.

[0082] In some embodiments, the above-mentioned differentiable binarization operation is performed on the probability map according to the pixel threshold corresponding to each pixel point in the feature map to obtain a binarized map, including: Get the binary image, B i,j is the pixel value of the pixel in the i-th row and j-th column of the binary image, P i,j is the pixel value of the pixel in the i-th row and j-th column of the probability map, T i,j is the pixel threshold corresponding to the pixel point in the i-th row and j-th column in the feature map, and k is the magnification factor, for example, k=50.

[0083] The CRNN+CTC text recognition algorithm can be used to perform text recognition on each text image area. The algorithm can be trained end-to-end, that is, the final text sequence is directly predicted from the original image without the need for preprocessing steps such as manual feature extraction or character segmentation. This simplifies the model training process and improves training efficiency. It can process text sequences of any length and is not limited by the number of characters in the input image. It is suitable for recognizing text in various scenarios, such as text in natural scenes, table text, long text, etc.

[0084] Step 103: Match the multiple scale values ​​with the multiple scale lines according to the characteristic information of the scale values ​​and the characteristic information of the scale lines, and determine the scale values ​​corresponding to the respective scale lines.

[0085] The characteristic information of the scale value includes the position, angle and direction of the scale value, and the characteristic information of the scale line includes the position, angle and direction of the scale line. The direction of the scale value and the scale line is the direction from the inside of the instrument to the outside. The angle of the scale value can be the angle corresponding to the line connecting the center of the scale value and the center of the circle in the coordinate system with the center of the circle fitted by the scale value as the origin, such as the angle between the line connecting the center of the scale value and the center of the circle and the coordinate axis of the coordinate system. The angle of the scale line can be the angle corresponding to the line connecting the center of the scale line and the center of the circle in the coordinate system with the center of the circle fitted by the scale line as the origin, such as the angle between the line connecting the center of the scale line and the center of the circle and the coordinate axis of the coordinate system. The circle fitted by the scale value and the circle fitted by the scale line have the same center.

[0086] In some embodiments, step 103 may specifically include:

[0087] Determine the first position coordinate x of the i-th scale value i and i , and the first angle θ i and the first direction v i ;

[0088] Determine the second position coordinate x of the jth tick mark j and j , and the second angle θ j and the first direction v j ;

[0089] According to the formula Calculate the characteristic information c of the i-th scale value i and the characteristic information c of the jth scale line j The Euclidean distance between

[0090] According to the formula Calculate the similarity s between the i-th scale value and the j-th scale line ij , σ is c i With c j The standard deviation of is used to measure c i and c j The degree of dispersion;

[0091] according to Calculate the weight between the i-th scale value and the j-th scale line, and obtain the weight between each scale value and all scale lines;

[0092] The scale line with the largest weight between the i-th scale value is matched, thereby obtaining a scale line matching each scale value.

[0093] The position of the scale value may be the position of the center point of the scale value, and the position of the scale line may be the position of the center point of the scale line. In this step, the scale line and the scale value are matched by the position, angle and direction of the scale value and the position, angle and direction of the scale line, so as to improve the matching accuracy between the scale and the pointer. The matching result is as follows: Figure 3 shown.

[0094] Step 104 , fitting the multiple scale lines and the one pointer respectively to obtain a scale fitting circle and a pointer fitting straight line.

[0095] In some embodiments, step 104 may include: determining the center point of each scale line; fitting the center point of each scale line using the least squares method to obtain the scale fitting circle; performing a Hough transform on the pointer to generate the pointer fitting straight line, wherein the pointer fitting straight line intersects the scale fitting circle at two points.

[0096] In this step, for each scale line of the segmented instrument, the center point of the scale line mask is taken, and the least squares method is used to determine a best fitting circle, that is, Figure 4 The scale fitting circle shown in the figure can ensure that the sum of the squares of the distances between the scale fitting circle and all the center points is minimized, thereby obtaining an accurate circle to represent the scale of the instrument. For the pointer part obtained by segmentation, Hough transform is used to perform straight line fitting, as shown in Figure 4 The Hough transform can effectively detect straight lines from images and maintain high accuracy even in the presence of noise. In this way, a straight line can be determined to represent the pointer of the instrument. The scale fitting circle and the pointer fitting straight line obtained by fitting are presented on the pointer instrument as shown in the following figure. Figure 5 shown.

[0097] Step 105 , determining the intersection of the pointer fitting line and the scale fitting circle and the scale interval where the pointer is located according to the scale fitting circle, the pointer fitting line and the scale values ​​corresponding to each scale.

[0098] In some embodiments, step 105 may include: determining the target point on the pointer that is farthest from the center of the scale fitting circle; determining two intersection points of the scale fitting circle and the pointer fitting straight line, and determining the target intersection point closest to the target point among the two intersection points; calculating the distance between the center point of each scale line and the target intersection point, and taking the scale interval formed by the scale lines corresponding to the two smallest distances as the scale interval where the pointer is located.

[0099] Step 106, determining the reading of the pointer instrument according to the scale interval where the pointer is located and the intersection point.

[0100] In some embodiments, step 106 may include: determining a starting position and an ending position of the scale interval; calculating a first arc α between the starting position and the ending position, and a second arc β between the starting position and the target intersection point; Determine the reading of the pointer meter.

[0101] Where value is the reading of the pointer instrument, maxvalue is the scale value corresponding to the end position of the scale interval, minvalue is the scale value corresponding to the start position of the scale interval, and x1 and y1 are the coordinates of the starting position of the scale interval, x2 and y2 are the coordinates of the ending position of the scale interval, x3 and y3 are the coordinates of the target intersection, x c and c are the coordinates of the center of the scale fitted circle.

[0102] like Figure 6 As shown in the figure, the pointer fitting line and the scale fitting circle have two intersection points, so we first need to confirm the quadrant (also known as the scale interval) where the pointer is located. c ,y c )The point on the pointer mask farthest from the pointer is the quadrant where the pointer is located. The intersection of the pointer fitting line and the scale fitting circle is formed by substituting the equation of the pointer fitting line into the equation of the scale fitting circle to form a quadratic equation. Solving this equation can obtain the coordinates of the intersection p3 (x3, y3); calculate the angle value of the center point of each scale line, and thus determine the two scales adjacent to p3, namely the starting position p1 (x1, y1) and the ending position p2 (x2, y2) of the scale area where the pointer is located. Calculate the arc α from p1 to p2, and the arc β from p1 to p3, and finally calculate the pointer reading based on the ratio of α and β and the starting and ending values ​​of the scale interval where the pointer is located.

[0103] Specifically, according to Determine the reading of the pointer instrument. Where value is the reading of the pointer instrument, maxvalue is the scale value corresponding to the end position p2 of the scale interval, minvalue is the scale value corresponding to the start position p1 of the scale interval, and x1 and y1 are the coordinates of the starting position p1 of the scale interval, x2 and y2 are the coordinates of the ending position p2 of the scale interval, x3 and y3 are the coordinates of the target intersection point p3, x c and c is the coordinate of the center c of the scale fitting circle.

[0104] In some embodiments, before step 101, the pointer meter reading recognition method may further include: obtaining an original image, wherein the original image contains a pointer meter; determining the position of the pointer meter from the original image based on the improved YOLOv9 model, and extracting the pointer meter image. In which, the P3 feature map and the conv-reg module, conv-cls module, and concat module connected to the P3 feature map are deleted from the improved YOLOv9 model; the ahchor parameter determination process in the conv-reg module in the P4 feature map and the P5 feature map of the improved YOLOv9 model is: obtaining the annotated meter sizes in multiple pointer meter images, calculating the multiple meter sizes by the mean algorithm, and obtaining the ahchor box that can represent the pointer meter.

[0105] The YOLOv9 model introduces groundbreaking technologies such as programmable gradient information (PGI) and generalized efficient layer aggregation network (GELAN), marking a major advancement in the field of real-time target detection. Since the target to be identified is a meter rather than a small target, the ahchor parameter in the bounding box regression module conv-reg in the original YOLOv9 model is replaced by the small target anchor in it with the meter size annotated by analyzing the data set (including multiple pointer meter images). The K-mean algorithm can be used to calculate an anchor box that best represents the pointer meter, thereby improving the meter detection accuracy. Since the size of the meter is relatively fixed, the P3 feature map in the original YOLOv9 model is used to detect smaller objects. Therefore, the P3 feature map in the original YOLOv9 model and the conv-reg module, conv-cls module, and concat module connected to the P3 feature map are deleted, thereby improving its detection speed. The improved YOLOV9 model is applied to the detection meter, and the detection speed and detection quality are better guaranteed.

[0106] See also Figures 7 to 9 ,The application of this application can show good robustness and generalization ability when applied to different types of ,instrument images such as pressure gauge, oil level gauge and voltmeter.

[0107] The above-mentioned method for identifying the reading of a pointer-type instrument firstly segments the scale and pointer of the pointer-type instrument according to the image of the pointer-type instrument to obtain a plurality of scale lines and a pointer, and performs text recognition on the image of the pointer-type instrument to obtain a plurality of scale values. Afterwards, the scale values ​​and scale lines are matched according to the characteristic information of the scale values ​​and the characteristic information of the scale lines. Next, the scale lines and the pointer are fitted respectively to obtain a scale fitting circle and a pointer fitting line. According to the scale fitting circle, the pointer fitting line and the scale values ​​corresponding to each scale, the intersection of the pointer fitting line and the scale fitting circle and the scale interval where the pointer is located are determined. Finally, according to the scale interval and the intersection where the pointer is located, the reading of the pointer-type instrument is determined. The embodiment of the present application can realize the precise segmentation of the pointer and the scale lines, and can accurately calculate the reading of the instrument, and has good robustness and generalization ability.

[0108] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0109] Corresponding to the pointer meter reading recognition method described in the above embodiment, Fig.10 A structural block diagram of a pointer meter reading recognition device provided in an embodiment of the present application is shown. For ease of explanation, only the parts related to the embodiment of the present application are shown.

[0110] See also Fig.10 The embodiment of the present application provides a pointer meter reading recognition device, including a segmentation module 201, a text recognition module 202, a matching module 203, a fitting module 204, a scale interval determination module 205 and a reading determination module 206.

[0111] The segmentation module 201 is used to segment the scale and pointer of the pointer instrument according to the pointer instrument image to obtain a plurality of scale lines and a pointer.

[0112] The text recognition module 202 is used to perform text recognition on the pointer instrument image to obtain a plurality of scale values.

[0113] The matching module 203 is used to match the multiple scale values ​​with the multiple scale lines according to the characteristic information of the scale values ​​and the characteristic information of the scale lines, and determine the scale values ​​corresponding to each scale line. The characteristic information of the scale values ​​includes the position, angle and direction of the scale values, and the characteristic information of the scale lines includes the position, angle and direction of the scale lines.

[0114] The fitting module 204 is used to fit the multiple scale lines and the one pointer respectively to obtain a scale fitting circle and a pointer fitting straight line.

[0115] The scale interval determination module 205 is used to determine the intersection of the pointer fitting line and the scale fitting circle and the scale interval where the pointer is located according to the scale fitting circle, the pointer fitting line and the scale values ​​corresponding to each scale.

[0116] The reading determination module 206 is used to determine the reading of the pointer instrument according to the scale interval where the pointer is located and the intersection point.

[0117] Optionally, the matching module 203 is specifically used for:

[0118] Determine the first position coordinate x of the i-th scale value i and i , and the first angle θ i and the first direction v i ;

[0119] Determine the second position coordinate x of the jth tick mark j and j , and the second angle θ j and the first direction v j ;

[0120] According to the formula Calculate the characteristic information c of the i-th scale value i and the characteristic information c of the jth scale line j The Euclidean distance between

[0121] According to the formula Calculate the similarity s between the i-th scale value and the j-th scale line ij , σ is c i With c j The standard deviation of

[0122] according to Calculate the weight between the i-th scale value and the j-th scale line, and obtain the weight between each scale value and all scale lines;

[0123] The scale line with the largest weight between the i-th scale value is matched, thereby obtaining a scale line matching each scale value.

[0124] Optionally, the fitting module 204 is specifically used for:

[0125] Determine the center point of each tick mark;

[0126] The center point of each scale line is fitted using the least square method to obtain the scale fitting circle;

[0127] Performing Hough transformation on the pointer to generate the pointer fitting straight line, wherein the pointer fitting straight line intersects with the scale fitting circle at two points.

[0128] Optionally, the scale interval determination module 205 is specifically used for:

[0129] Determine the target point on the pointer that is farthest from the center of the scale fitting circle;

[0130] Determine two intersection points of the scale fitting circle and the pointer fitting straight line, and determine a target intersection point closest to the target point among the two intersection points;

[0131] The distance between the center point of each scale line and the target intersection is calculated, and the scale interval formed by the scale lines corresponding to the two smallest distances is used as the scale interval where the pointer is located.

[0132] Optionally, the reading determination module 206 is specifically configured to:

[0133] Determine the starting position and the ending position of the scale interval;

[0134] Calculating a first arc α between the starting position and the ending position, and a second arc β between the starting position and the target intersection point;

[0135] according to Determine the reading of pointer instruments;

[0136] Where value is the reading of the pointer instrument, maxvalue is the scale value corresponding to the end position of the scale interval, minvalue is the scale value corresponding to the start position of the scale interval, and x1 and y1 are the coordinates of the starting position of the scale interval, x2 and y2 are the coordinates of the ending position of the scale interval, x3 and y3 are the coordinates of the target intersection, x c and c are the coordinates of the center of the scale fitted circle.

[0137] Optionally, the pointer meter reading recognition device further includes an image extraction module, which is used to:

[0138] Acquire an original image, wherein the original image includes a pointer instrument;

[0139] Determine the position of the pointer instrument from the original image based on the improved YOLOv9 model, and extract the pointer instrument image;

[0140] Among them, the P3 feature map and the conv-reg module, conv-cls module and concat module connected to the P3 feature map are deleted in the improved YOLOv9 model;

[0141] The process of determining the ahchor parameters in the conv-reg module in the P4 feature map and the P5 feature map of the improved YOLOv9 model is as follows: obtaining the annotated meter sizes in multiple pointer meter images, calculating the multiple meter sizes by the mean algorithm, and obtaining the ahchor box that can represent the pointer meter.

[0142] After the image extraction module extracts the pointer instrument image, the segmentation module 201 segments the scale and pointer of the pointer instrument according to the pointer instrument image to obtain a plurality of scale lines and a pointer.

[0143] Optionally, the segmentation module 201 is specifically used for:

[0144] The pointer instrument image is segmented by the U2Net network algorithm to obtain multiple scale lines and a pointer;

[0145] Among them, only the En_1 layer, En_3 layer, En_5 layer and En_6 layer of the U2Net network algorithm are used to extract the scale line features and pointer features in the pointer instrument image.

[0146] Optionally, the text recognition module 202 is specifically used for:

[0147] Extracting features of the pointer meter image through convolution and sampling operations to obtain a feature map;

[0148] Performing a deconvolution operation on the feature map to obtain a probability map;

[0149] Processing the feature map to obtain an adaptive threshold map;

[0150] Perform a differentiable binarization operation on the probability map and the adaptive threshold map to obtain a binarized map;

[0151] Based on the binarized image, the pointer instrument image is segmented into text regions to obtain a text image region containing text content;

[0152] Text recognition is performed on each text image area to obtain multiple scale values.

[0153] Fig.11 Schematic diagram of a terminal device provided by an embodiment of the present invention. Fig.11 As shown, the terminal device 300 of this embodiment includes a processor 310 and a memory 320. The memory 320 stores a computer program that can be run on the processor 310, such as a pointer meter reading recognition program. When the processor 310 executes the computer program, the steps in the above-mentioned pointer meter reading recognition method embodiment are implemented, such as Figure 1Alternatively, when the processor 310 executes the computer program, the functions of each module / unit in the above-mentioned device embodiments are realized, for example, Fig.10 The functions of the segmentation module 201 to the reading determination module 206 are shown.

[0154] Exemplarily, the computer program may be divided into one or more modules / units, which are stored in the memory 320 and executed by the processor 310 to complete the present invention. The one or more modules / units may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program in the terminal device 300. For example, the computer program may be divided into a segmentation module, a text recognition module, a matching module, a fitting module, a scale interval determination module, and a reading determination module.

[0155] The terminal device 300 may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The terminal device may include, but is not limited to, a processor 310 and a memory 320. Those skilled in the art will appreciate that Fig.11 It is only an example of the terminal device 300 and does not constitute a limitation of the terminal device 300. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the terminal device may also include input and output devices, network access devices, buses, etc.

[0156] The processor 320 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0157] The memory 320 may be an internal storage unit of the terminal device 300, such as a hard disk or memory of the terminal device 300. The memory 320 may also be an external storage device of the terminal device 300, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal device 300. Further, the memory 320 may also include both an internal storage unit of the terminal device 300 and an external storage device. The memory 320 is used to store the computer program and other programs and data required by the terminal device. The memory 320 may also be used to temporarily store data that has been output or is to be output.

[0158] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0159] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A method for identifying readings of a pointer instrument, characterized in that: include: Segmenting the scale and the pointer of the pointer instrument according to the pointer instrument image to obtain a plurality of scale lines and a pointer; Performing text recognition on the pointer instrument image to obtain multiple scale values; According to the characteristic information of the scale values ​​and the characteristic information of the scale lines, the multiple scale values ​​are matched with the multiple scale lines to determine the scale values ​​corresponding to the respective scale lines; wherein the characteristic information of the scale values ​​includes the position, angle and direction of the scale values, and the characteristic information of the scale lines includes the position, angle and direction of the scale lines; Fitting the multiple scale lines and the one pointer respectively to obtain a scale fitting circle and a pointer fitting straight line; According to the scale fitting circle, the pointer fitting straight line and the scale values ​​corresponding to each scale, determining the intersection point of the pointer fitting straight line and the scale fitting circle and the scale interval where the pointer is located; The reading of the pointer instrument is determined according to the scale interval where the pointer is located and the intersection point.

2. The method for identifying pointer meter readings according to claim 1, characterized in that: The matching of the multiple scale values ​​with the multiple scale lines according to the characteristic information of the scale values ​​and the characteristic information of the scale lines to determine the scale values ​​corresponding to the respective scale lines includes: Determine the first position coordinate x of the i-th scale value i and i , and the first angle θ i and the first direction v i ; Determine the second position coordinate x of the jth tick mark j and j , and the second angle θ j and the first direction v j ; According to the formula Calculate the characteristic information c of the i-th scale value i and the characteristic information c of the jth scale line j The Euclidean distance between According to the formula Calculate the similarity s between the i-th scale value and the j-th scale line ij , σ is c i With c j The standard deviation of according to Calculate the weight between the i-th scale value and the j-th scale line, and obtain the weight between each scale value and all scale lines; The scale line with the largest weight between the i-th scale value is matched, thereby obtaining a scale line matching each scale value.

3. The method for identifying pointer meter readings according to claim 1, characterized in that: The step of fitting the multiple scale lines and the one pointer respectively to obtain a scale fitting circle and a pointer fitting straight line comprises: Determine the center point of each tick mark; The center point of each scale line is fitted using the least square method to obtain the scale fitting circle; Performing Hough transformation on the pointer to generate the pointer fitting straight line, wherein the pointer fitting straight line intersects with the scale fitting circle at two points.

4. The method for identifying pointer meter readings according to claim 1, characterized in that: The step of determining the intersection of the pointer fitting line and the scale fitting circle and the scale interval where the pointer is located according to the scale fitting circle, the pointer fitting line and the scale values ​​corresponding to each scale, comprises: Determine the target point on the pointer that is farthest from the center of the scale fitting circle; Determine two intersection points of the scale fitting circle and the pointer fitting straight line, and determine a target intersection point closest to the target point among the two intersection points; The distance between the center point of each scale line and the target intersection is calculated, and the scale interval formed by the scale lines corresponding to the two smallest distances is used as the scale interval where the pointer is located.

5. The method for identifying pointer meter readings according to claim 4, characterized in that: Determining the reading of the pointer instrument according to the scale interval where the pointer is located and the intersection point includes: Determine the starting position and the ending position of the scale interval; Calculating a first arc α between the starting position and the ending position, and a second arc β between the starting position and the target intersection point; according to Determine the reading of pointer instruments; Where value is the reading of the pointer instrument, maxvalue is the scale value corresponding to the end position of the scale interval, minvalue is the scale value corresponding to the start position of the scale interval, and x1 and y1 are the coordinates of the starting position of the scale interval, x2 and y2 are the coordinates of the ending position of the scale interval, x3 and y3 are the coordinates of the target intersection, x c and c are the coordinates of the center of the scale fitted circle.

6. The method for identifying pointer meter readings according to claim 1, characterized in that: Before the step of segmenting the scale and the pointer of the pointer instrument according to the pointer instrument image, the pointer instrument reading recognition method further includes: Acquire an original image, wherein the original image includes a pointer instrument; Determine the position of the pointer instrument from the original image based on the improved YOLOv9 model, and extract the pointer instrument image; Among them, the P3 feature map and the conv-reg module, conv-cls module and concat module connected to the P3 feature map are deleted in the improved YOLOv9 model; The process of determining the ahchor parameters in the conv-reg module in the P4 feature map and the P5 feature map of the improved YOLOv9 model is as follows: obtaining the annotated meter sizes in multiple pointer meter images, calculating the multiple meter sizes by the mean algorithm, and obtaining the ahchor box that can represent the pointer meter.

7. The method for identifying pointer meter readings according to claim 1, characterized in that: The step of segmenting the scale and the pointer of the pointer instrument according to the pointer instrument image to obtain a plurality of scale lines and a pointer includes: The pointer instrument image is segmented by the U2Net network algorithm to obtain multiple scale lines and a pointer; Among them, only the En_1 layer, En_3 layer, En_5 layer and En_6 layer of the U2Net network algorithm are used to extract the scale line features and pointer features in the pointer instrument image.

8. The method for identifying pointer meter readings according to claim 1, characterized in that: The text recognition is performed on the pointer instrument image to obtain multiple scale values, including: Extracting features of the pointer meter image through convolution and sampling operations to obtain a feature map; Performing a deconvolution operation on the feature map to obtain a probability map; Processing the feature map to determine a pixel threshold corresponding to each pixel in the feature map; According to the pixel threshold corresponding to each pixel point in the feature map, a differentiable binarization operation is performed on the probability map to obtain a binarized map; Based on the binarized image, the pointer instrument image is segmented into text regions to obtain a text image region containing text content; Text recognition is performed on each text image area to obtain multiple scale values.

9. A pointer-type instrument reading recognition device, characterized in that: include: A segmentation module is used to segment the scale and pointer of the pointer instrument according to the pointer instrument image to obtain multiple scale lines and a pointer; A text recognition module, used for performing text recognition on the pointer instrument image to obtain multiple scale values; A matching module, used for matching the multiple scale values ​​with the multiple scale lines according to the characteristic information of the scale values ​​and the characteristic information of the scale lines, and determining the scale values ​​corresponding to the respective scale lines; wherein the characteristic information of the scale values ​​includes the position, angle and direction of the scale values, and the characteristic information of the scale lines includes the position, angle and direction of the scale lines; A fitting module, used for fitting the multiple scale lines and the one pointer respectively to obtain a scale fitting circle and a pointer fitting straight line; A scale interval determination module, used to determine the intersection of the pointer fitting line and the scale fitting circle and the scale interval where the pointer is located according to the scale fitting circle, the pointer fitting line and the scale values ​​corresponding to each scale; The reading determination module is used to determine the reading of the pointer instrument according to the scale interval where the pointer is located and the intersection point.

10. A terminal device, comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, characterized in that: When the processor executes the computer program, the steps of the pointer meter reading recognition method as described in any one of claims 1 to 8 are implemented.

Images