Multi-pointer meter reading identification method based on traditional algorithm and deep learning

Abstract

The invention discloses a multi-pointer meter reading identification method based on a traditional algorithm and deep learning, and relates to the technical field of industrial monitoring, and the method comprises the steps: 1, setting the configuration of an image collection device, collecting a meter image according to the configuration of the image collection device, and carrying out the calibration, and obtaining calibration information and a reference image; step 2, acquiring a to-be-identified meter image, and performing search matching on the to-be-identified meter image according to the reference image by adopting a template matching strategy to obtain a target meter ROI (Region of Interest); 3, identifying all pointer positions in the ROI of the target meter by adopting a Canny edge detection algorithm, a Hough straight line detection algorithm, a straight line filtering method and a deep learning algorithm according to the calibration information, and obtaining a final effective straight line set; and step 4, calculating the meter reading according to the calibration information and the final effective straight line set. According to the method, accurate identification of the pointer meter can be realized, and the robustness problem and the performance problem in a current meter identification algorithm are optimized.

Description

Technical Field

[0001] The present invention relates to the field of industrial monitoring technology, and more specifically to a multi-pointer meter reading recognition method based on traditional algorithms and deep learning. Background Art

[0002] Traditional pointer meters (devices with no electrical signals and purely mechanical mechanisms) are still widely used in many fields (such as the industrial field) to measure various values ​​due to their considerable reliability. However, the readings of such pointer meters (hereinafter referred to as meters) usually require manual observation and recording. In the industrial field, meters are often installed at a long distance and in poor on-site environmental conditions, which results in low efficiency of manual meter reading. At the same time, as the number of meters that need to be read on the same day increases, there is also the problem of easy errors in meter reading. Although there are patents and methods for meter reading recognition, they have certain limitations. The existing technical methods mainly include:

[0003] The publication number is: CN117132756A, and the name is: A patent for a meter needle reading recognition method, device and equipment. It proposes to integrate traditional image processing algorithms (such as median filtering, binarization processing, Hough transform, etc.) and hardware equipment (camera, processor, etc.) to realize automatic meter reading recognition and data upload. Due to the recognition based on pure traditional image processing algorithm, if there are interfering non-pointer straight lines or the number of pointers is greater than one, it will cause the pointer to be misidentified / missed. Therefore, the robustness and generalization of this recognition method still have room for improvement; the publication number is: CN116844027A, and the name is: A patent for a meter reading recognition method for a smart distribution station based on deep learning. It proposes to first use yolov3 to detect the meter ROI, and then perform an ROI (Region of The pointer position is obtained by a two-stage method of segmenting the region of interest (region of interest) through u-net, and finally the pointer reading recognition is realized by using Hough transform and calibration information. This method uses two deep learning models. In terms of computing power, it consumes more resources. In terms of data, it is necessary to prepare and annotate the data of the two models, and there is a certain quantity requirement on the amount of data. This method is costly and does not support multi-pointer meter reading recognition; the publication number is: CN116503757A, and the name is: Patent for substation switch and meter reading method, device and computer equipment, which proposes to obtain the meter ROI image by the aerial image through the image area recognition model, and then pass it through DEEP LAB The V3 neural network segments the meter pointer to obtain the pointer position, flattens the meter's effective reading area into a rectangular area through information such as the meter's starting and ending scale positions, and finally calculates the ratio of the distance between the pointer and the starting point of the scale in the rectangular area to the distance from the starting point to the end point to calculate the pointer reading using a two-stage method. This method uses a more advanced semantic segmentation model architecture with larger parameters, making pointer recognition more accurate but also increasing computing resource costs. At the same time, the method uses the geometric center of the pointer image to determine the center point of the circle, which places strict requirements on the accuracy of the pointer image area recognition model.

[0004] Therefore, how to effectively improve the reading efficiency and accuracy of pointer meters while saving computing resources is an urgent problem that technical personnel in this field need to solve. Summary of the invention

[0005] In view of this, the present invention provides a multi-pointer meter reading recognition method based on traditional algorithms and deep learning. By using calibration information and combining traditional algorithms and deep learning algorithms, based on any camera gun or pan / tilt head as a shooting device, a multi-pointer meter reading recognition method is implemented, which effectively optimizes the robustness and performance problems in the current meter recognition algorithm, and supports multi-pointer meter reading recognition.

[0006] In order to achieve the above object, the present invention adopts the following technical solution:

[0007] A multi-pointer meter reading recognition method based on traditional algorithms and deep learning includes the following steps:

[0008] Step 1: Set the image acquisition device configuration, collect meter images according to the image acquisition device configuration and calibrate them to obtain calibration information and reference images;

[0009] Step 2: Collect the image of the meter to be identified, use the template matching strategy to search and match the image of the meter to be identified according to the reference image, and obtain the target meter ROI;

[0010] Step 3: Based on the calibration information, the Canny edge detection algorithm, Hough line detection algorithm, line filtering method and deep learning algorithm are used to identify all pointer positions in the target meter ROI to obtain the final valid line set;

[0011] Step 4: Calculate the meter reading based on the calibration information and the final valid straight line set.

[0012] Preferably, in step 1, the meter ROI frame, scale and range in the meter image are calibrated, and the calibration information obtained mainly includes the meter's ROI frame coordinates, starting scale point coordinates, center scale point coordinates, ending scale point coordinates, starting range and ending range; the camera configuration includes focal length, preset position, etc., which are used to instruct the camera to return to the position of photographing the meter.

[0013] Preferably, step 1 comprises the following steps:

[0014] Step 11: Place the image acquisition device according to the image acquisition device configuration, and acquire the meter image as a reference image;

[0015] Step 12: Calibrate the reference image to obtain calibration information.

[0016] Preferably, in step 2, a local-first-global template matching strategy is used to perform search and matching, including the following steps:

[0017] Step 21: Taking the short side of the meter ROI frame as a reference, perform a local search in the meter image to be identified within the pixel range of ±20% of the ROI frame coordinates in the calibration information ± the short side length of the meter ROI frame, and compare the similarity of pixel information. Take the search area in the meter image to be identified that is similar to the meter ROI frame as the target meter ROI, and obtain the target meter ROI frame coordinates. If the target meter ROI frame coordinates are not found, proceed to step 22. If the calculated similarity score of the pixel information of the search area in the meter image to be identified and the pixel information of the meter ROI frame in the reference image falls within the similarity threshold, then the two are judged to be similar.

[0018] Step 22: Taking the short side of the meter ROI frame as a reference, perform a global search in the entire image pixel range using external expansion calibration, compare pixel information, and use the search area in the meter image to be identified that is similar to the meter ROI frame as the target meter ROI to obtain the target meter ROI frame coordinates; if the target meter ROI is not found, the recognition fails.

[0019] The technical effect of the above technical solution is that the purpose of the search and matching operation is to solve the problem of image position offset when calling the pan-tilt preset position due to minor machine failure or improper maintenance when using a pan-tilt camera, and a correction mechanism is designed to ensure that the actual meter is accurately located through the calibrated reference map to obtain the actual target meter ROI; the local-first-global template matching strategy is based on the fact that in most cases, although the pan-tilt camera preset position image is offset, the overall offset is not very large. In order to reduce the calculation cost, the short edge of the picture is first used as the reference, and a local search is performed using the ±edge×20% pixel range of the meter position during external calibration. If the local search fails, a full-image search is performed.

[0020] Preferably, step 3 comprises the following steps:

[0021] Step 31: Use the Resize function to adjust the target meter ROI into a regular rectangle to obtain a corrected ROI image;

[0022] Step 32: Using the Canny edge detection algorithm to convert the corrected ROI image into a feature map;

[0023] Step 33: using the Hough transform line detection method to detect all lines in the feature map, and using the meter center point neighborhood line filtering method to filter out invalid lines, to obtain a candidate line set;

[0024] Step 34: Calculate the target meter center point coordinates, the target meter start scale point coordinates, the target meter end scale point coordinates, the target meter start range and the target meter end range through the target meter ROI frame coordinates and calibration information; take the target meter center point as the center of the circle and the start scale point coordinates and the end scale point coordinates in the marking information as the radius, and take the average of the two radii as the target radius to make a target circle; since the meter center point position is fixed relative to the meter ROI frame coordinates, the target meter center point coordinates are calculated through the target meter ROI frame coordinates; calculate the offset of the target meter center point coordinates and the center point coordinates of the ROI frame in the reference diagram, and calculate the target meter start scale point coordinates, the target meter end scale point coordinates, the target meter start range and the target meter end range respectively according to the offset and the start scale point coordinates, the end scale point coordinates, the start range and the end range in the calibration information;

[0025] Step 35: Calculate the two intersection points P1 and P2 of all candidate lines in the candidate line set and the target circle, connect the two intersection points P1 and P2 with the meter center point, and obtain two corrected opposite pointer lines corresponding to each candidate line to form a corrected line set;

[0026] Step 36: The center point of the meter is connected to the coordinates of the starting scale point of the target meter and the coordinates of the ending scale point of the target meter to obtain the starting straight line and the ending straight line respectively, and the angle value θ between all the correction lines in the correction line set and the starting straight line is calculated. i , and the angle values ​​θ of the starting and ending lines, obtain the meter angle range [0, θ], and filter out the angle value θ i The correction lines within the meter angle range constitute a line subset;

[0027] Step 37: Set the angle values ​​θ corresponding to all the corrected lines in the line subset i The angle difference is obtained by subtracting each other, and the correction lines with angle differences less than the set threshold are judged as repeated lines. A repeated line subset is constructed, and the angle value θ of each correction line is calculated in the repeated line subset. i Arrange in ascending order, select the correction line corresponding to the median angle value as the representative line, and the representative line and the correction line whose angle difference is greater than or equal to the set threshold constitute a valid line set;

[0028] Step 38: Use the trained classification model to perform binary classification of the pointer and background on the valid straight line set to obtain the final valid straight line set.

[0029] Preferably, in step 4, the angle values ​​θ corresponding to all the correction lines in the final valid straight line set are calculated. i and the meter angle range [0, θ] to calculate the meter reading, the expression is:

[0030]

[0031] Among them, S0 represents the starting range of the target meter; S1 represents the ending range of the target meter.

[0032] Preferably, the pixel setting range of the side length w in the corrected ROI image is:

[0033] w ∈ [80, the shortest side length of the target meter ROI], w ≥ 80

[0034] The lower bound of the minimum value 80 is to ensure that the adjusted image resolution will not be too low and sufficient effective image details can be retained; the upper bound of the maximum value is to ensure that the image will not be enlarged too much and distorted and cause excessive calculation; the larger the side length w is set, the higher the reading accuracy is improved and the calculation cost increases; the smaller the side length is set, the lower the reading accuracy is and the calculation cost decreases. When the side length is greater than 80, the error loss of the general meter reading accuracy is usually within 1%.

[0035] Preferably, the expression of the Hough transform line detection method is: ρ = x cosθ + y sinθ, where (x, y) represents the line coordinates; θ represents the line angle.

[0036] Preferably, the expression of the line filtering method in the neighborhood of the meter center point is:

[0037]

[0038] Among them, (x0, y0) represents the coordinates of the target meter center point; r represents the neighborhood of the center point; d represents the distance between the candidate line and the target meter center point; the candidate lines with d < r are screened out.

[0039] Preferably, a boundary error is set to determine the meter angle range. The boundary error can take a value of ±1°, then the meter angle range is [θ - 1, θ + 1]. Keep the calibration lines with θ - 1 ≤ θ i ≤ θ + 1 to obtain a line subset and filter out the invalid line subset outside the range.

[0040] Preferably, in step 38, the ResNet18 neural network is used as the backbone network to construct a classification model. In the ResNet18 neural network, the input of each residual block is added to the output through a shortcut connection. Three fully connected layers are added to the head of the backbone network. The activation function of the backbone network uses the sigmoid function. The Adam optimizer and the FocalLoss loss function are used to train the classification model.

[0041] Preferably, a data set is constructed to train a classification model, image data of meters in actual scenes are collected, and the meter pointer area (pointer) in the image and the non-meter pointer area (background) about the center point at any angle of the pointer are annotated to construct a data set; when annotating the meter pointer area in the image, first annotate the line segment from the center point of the meter to the pointer endpoint, and then perform area selection based on the line segment to determine the meter pointer area; when annotating the non-meter pointer area, use an estimation method to determine the line segment length to mark the area without the pointer, and then perform area selection based on the line segment to determine the non-meter pointer area; the area selection method is to use the image area of ​​8 pixels expanded to the left and right by the line segment as the meter pointer area or the non-meter pointer area; if it is a line segment about the pointer endpoint in the meter pointer area, the label is pointer, and if it is a line segment arbitrarily annotated about the background in the non-meter pointer area, the label is background.

[0042] Through the above technical solutions, it can be known that compared with the prior art, the present invention discloses a multi-pointer meter reading recognition method based on traditional algorithms and deep learning. For multi-pointer meter reading recognition, data calibration is used as prior auxiliary information and traditional algorithms and deep learning algorithms such as template matching strategy, Canny edge detection algorithm, Hough line detection algorithm, and line filtering method are used to effectively optimize computing resources and the accuracy of meter reading recognition, improve the scope of application of the method, and be able to cope with more complex meter reading scenarios. Among them, based on the relatively fixed relationship between the image acquisition device (camera gun, pan / tilt) and the meter, a template matching strategy of first local and then global is proposed to accelerate the search for the meter ROI image position and improve the performance and robustness of the meter recognition method; a meter center point neighborhood line filtering method, correction of line set pointing, and use of the effective angle range to filter the line set are proposed to effectively filter the invalid and redundant pointer lines appearing in the pointer recognition algorithm, bringing effective gains to computing performance and recognition accuracy. The implementation method technology of the present invention relates to the fields of image processing, deep learning, and computer vision technology, and can be applied to the fields of fixed camera guns, pan-tilt automatic intelligent monitoring, safe production intelligent monitoring, etc. The entire implementation process is clear and simple. In addition to the camera, no other additional sensing equipment is required. The deployment cost is low and the detection effect is good. It not only makes up for the problem of excessive computing power expenditure in pure deep learning methods, but also makes up for the problems of insufficient accuracy and robustness in pure traditional algorithms. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0044] Figure 1 A schematic diagram of a flow chart of a multi-pointer meter reading recognition method based on traditional algorithms and deep learning provided by the present invention;

[0045] Figure 2 This is a schematic diagram of image search and matching provided by the present invention. DETAILED DESCRIPTION

[0046] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0047] The embodiment of the present invention discloses a multi-pointer meter reading recognition method based on traditional algorithms and deep learning, such as Figure 1 As shown, the following steps are included:

[0048] S1: Setting the image acquisition device configuration, acquiring the meter image according to the image acquisition device configuration and calibrating it, and obtaining the calibration information and the reference image;

[0049] S2: Collect the image of the meter to be identified, use the template matching strategy to search and match the image of the meter to be identified according to the reference image, and obtain the target meter ROI;

[0050] S3: According to the calibration information, the Canny edge detection algorithm, the Hough line detection algorithm, the line filtering method and the deep learning algorithm are used to identify all the pointer positions in the target meter ROI to obtain the final valid line set;

[0051] S4: Calculate the meter reading based on the calibration information and the final valid straight line set.

[0052] On the other hand, in a specific embodiment, when the image acquisition device is a camera, the multi-pointer meter reading recognition method comprises the following steps:

[0053] S1: Configure the calibration information of the meter according to the camera; perform data calibration configuration work for related scenarios before deploying the application, including: setting and saving the best focal length of the camera to observe the meter, preset position (PTZ camera) and other configuration information, which is used to read the configuration and set the camera screen when identifying the meter; after adjusting the best screen, save the screen and use it as a reference map to calibrate and save the meter screen; information that needs to be calibrated: meter ROI frame, meter ROI frame coordinates, meter start scale point coordinates, end scale point coordinates, meter center point coordinates, start range, end range;

[0054] S2: Use a template matching strategy that first localizes and then globalizes to search for the target meter ROI in the actual recognition image through the template image, and obtain the position coordinates of the meter ROI, as Figure 2 shown; specifically, read the calibration information of the corresponding meter, use the reference image of the meter as the template, first use the short side (edge) of the reference image as the benchmark, and use the pixel range of ±edge×20% of the meter ROI frame coordinates during external calibration for local search. If the local search fails, then perform a full-image search mechanism. If the full-image search fails, the algorithm recognition fails, and it is prompted that the meter reading recognition is incorrect; the search process is a similarity comparison process. If the similarity score of the pixel information in the search area of the meter image to be recognized and the pixel information of the meter ROI frame in the reference image falls within the similarity threshold range, it is determined that the two are similar, and the similar area is used as the target meter ROI, and the corresponding target meter ROI frame coordinates are obtained;

[0055] S3: Based on the calibration information, use traditional algorithms and deep learning algorithms to identify the positions of all pointers;

[0056] S31: Adjust and correct the meter ROI image: Use Resize to adjust the meter ROI image into a regular rectangle, which can be used to correct the distortion caused by the angle to a certain extent; the side length of the regular rectangle w Recommended pixel setting range:

[0057] w∈[80, the shortest side length of the target meter ROI], w≥80;

[0058] S32: Convert the meter ROI image into a feature map through the Canny edge detection algorithm;

[0059] S33: Use the Hough transform line detection method to detect all lines in the feature map, and filter out invalid lines through the line filtering method in the neighborhood of the meter center point to obtain a candidate line set; the filtering method is as follows (only keep d<r):

[0060] Hough line equation: ρ = x cosθ + y sinθ

[0061] Filtering equation:

[0062] (x0, y0) represents the coordinates of the target meter center point, r represents the neighborhood of the center point; d represents the distance between the candidate line and the target meter center point; select the candidate lines with d<r

[0063] S34: Because there is a fixed relationship between the center point position of the meter and the coordinates of the meter ROI frame, the center point coordinates of the target meter are calculated through the ROI frame coordinates of the target meter, and the offset between the center point coordinates of the target meter and the center point coordinates of the ROI frame in the reference diagram is calculated. According to the offset and the starting scale point coordinates, ending scale point coordinates, starting range and ending range in the calibration information, the starting scale point coordinates of the target meter, the ending scale point coordinates of the target meter, the starting range of the target meter and the ending range of the target meter are calculated respectively; the center point of the target meter is connected with the starting scale point coordinates of the target meter and the ending scale point coordinates of the target meter respectively and the mean of the connecting lines is calculated, and a circle is made with the center point of the meter as the center and the mean as the radius, and is marked as C. The two intersection points P1 and P2 of all the straight lines in the candidate straight line set and the circle C are calculated, and the two intersection points P1 and P2 are connected with P1 and P2 respectively through the center point of the meter to obtain two corrected opposite pointer straight lines;

[0064] S35: Calculate the angle value θ by using the straight line set obtained in S34 and the coordinates of the starting scale point of the target meter and the center point of the target meter i , calculate the angle between the straight line formed by the coordinates of the starting scale point of the target meter and the center point of the target meter and the straight line formed by the coordinates of the ending scale point of the target meter and the center point of the target meter, as the meter angle range [0, θ]; set the boundary error ±1°, and keep 0-1≤θ i The straight lines ≤θ+1 constitute a straight line subset, and the invalid straight line subsets outside the range are filtered out;

[0065] S36: The straight line angle value θ obtained in S35 i The lines with angle differences within ±2° are determined as the same repeated line, and the filtering strategy is performed. The specific strategy is: i The values ​​are sorted in ascending order, and the straight line with the median angle of the repeated straight line subset is taken as the representative straight line of this repeated straight line subset. The representative straight line and the straight line whose angle difference is not within ±2° are used to construct a valid straight line set.

[0066] S37: Use the ResNet18 neural network model to classify the pointer and background of the S36 valid straight line set and set the score threshold to filter and obtain the final valid straight line set; the ResNet18 neural network model training, reasoning method and steps are as follows:

[0067] S371: Construct a data set; collect image data sets of meters in actual scenes, use annotation tools to manually annotate meter pointer areas (pointers) and non-meter pointer areas (background) about the center point at any angle, and select areas by expanding 8 pixels to the left and right from the pointer straight line as image areas; annotate the line segments from the center point of the meter to the end point of the pointer, and expand to form the meter pointer area, with the line segment label as the pointer. Similarly, use line segments of estimated length to annotate non-meter pointers, and expand to form non-meter pointer areas, with the line segment label of the estimated length as the background; the ratio of meter pointer data to non-meter pointer data is 1:1. Then divide the collected data into training set, test set, and validation set in a ratio of 7:2:1, where the training set is used to train the model, the test set is used to evaluate the model, and the validation set is used to check the model performance during training;

[0068] S372: Model construction; ResNet18 neural network is used as the backbone network for identifying meter pointers; The main feature of ResNet18 is that it contains residual blocks. The model architecture with quick connections solves the problem of gradient vanishing and gradient exploding during deep neural network training. ResNet 18 means a network containing 18 convolutional layers. The lightweight and suitable model architecture helps to reduce computing resources while ensuring a certain accuracy; The main modules of ResNet 18 are as follows: Residual block: In each residual block, the input is directly added to the output through a quick connection. The basic form of the residual block is: y = F(x) + x, where x is the input, F(x) is a combination of some convolutional layers, batch normalization layers and activation functions, and y is the output; Quick connections: These connections enable the network to directly pass input information to deeper networks, alleviate the problem of gradient vanishing, and allow deeper networks to be built without significantly increasing training errors; Three fully connected layers are added to the head of the backbone network to classify background and pointers, and sigmoid is used as the activation function;

[0069] S373: training model; using the training set for model training and optimization, inputting the meter pointer area in the training set in S371 into the model, and converting it into a ROI image with a width of 16 and a height of 64 through perspective transformation for training; using the test set for model testing and evaluating the performance of model indicators, the model training optimization algorithm uses the Adam optimizer, the initial learning rate is set to lr = 1e-3, the number of loop iterations epoch is set to 200 times, the batch size batch_size = 64, and early_stop = 10. When the accuracy of the model training does not improve within 10 epochs, the model training is terminated;

[0070] S374: The loss function uses FocalLoss to supervise the learning of difficult samples and improve the detection effect of difficult samples. FocalLoss = -α t (1-p t )Y log(p t ), where γ is a regulating factor used to adjust the rate at which the weight of simple samples is reduced. When γ = 0, FocalLoss is equivalent to the cross entropy loss function; α t It is a balancing factor, which is used to balance the imbalance in the number of positive and negative samples.

[0071] S375: Deploy the model; after the training is completed, obtain the ResNet18 neural network model as the classification model, deploy the model on the deep learning machine platform, perform model inference and test on the image, and obtain the final valid straight line set; when performing model inference, input the valid straight line set into the trained model, output the score value, and predict the labels of all the lines in the valid straight line set according to the score value and the set classification threshold, and select all the lines labeled as pointers to construct the final valid straight line set; the classification threshold can be 0.5, and the lines with a score value greater than or equal to 0.5 are predicted as pointers, and the lines with a score value less than 0.5 are predicted as backgrounds;

[0072] S4: Calculate the final reading based on the calibration information and the pointer position carried by the final valid straight line set; the expression is:

[0073]

[0074] Among them, S0 represents the starting range of the target meter, and S1 represents the ending range of the target meter.

[0075] On the other hand, image processing is performed using edge detection models trained based on deep learning to obtain the feature map of the meter, thereby improving the generalization and precision of feature extraction.

[0076] On the other hand, the use of neural networks such as ResNet34, ResNet50, ResNeXt, and ViT to build models for straight line classification has larger model scale and more complex architecture, which can further improve the accuracy of classification.

[0077] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0078] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A multi-pointer meter reading recognition method based on traditional algorithms and deep learning, It is characterized in that The following steps are involved: Step 1: Set the image acquisition device configuration, collect meter images according to the image acquisition device configuration and calibrate them to obtain calibration information and reference images; Step 2: Collect the image of the meter to be identified, use the template matching strategy to search and match the image of the meter to be identified according to the reference image, and obtain the target meter ROI; Step 3: Based on the calibration information, the Canny edge detection algorithm, Hough line detection algorithm, line filtering method and deep learning algorithm are used to identify all pointer positions in the target meter ROI to obtain the final valid line set; Step 4: Calculate the meter reading based on the calibration information and the final valid straight line set.

2. According to claim 1, a multi-pointer meter reading recognition method based on traditional algorithm and deep learning, It is characterized in that In step 1, the meter ROI frame, scale and range in the meter image are calibrated, and the calibration information obtained includes the meter ROI frame coordinates, starting scale point coordinates, center point coordinates, ending scale point coordinates, starting range and ending range; the camera configuration includes focal length and preset position.

3. According to claim 1, a multi-pointer meter reading recognition method based on traditional algorithm and deep learning, It is characterized in that Step 1 includes the following steps: Step 11: Place the image acquisition device according to the image acquisition device configuration, and acquire the meter image as a reference image; Step 12: Calibrate the reference image to obtain calibration information.

4. According to claim 2, a multi-pointer meter reading recognition method based on traditional algorithm and deep learning, It is characterized in that In step 2, the search and matching is performed using a local-first-global template matching strategy, which includes the following steps: Step 21: In the meter image to be identified, a local search is performed in the pixel range of ROI frame coordinates ± meter ROI frame short side length × 20% by using external expansion calibration, and pixel information similarity comparison is performed. The search area similar to the meter ROI frame in the meter image to be identified is taken as the target meter ROI, and the target meter ROI frame coordinates are obtained; if the target meter ROI is not found, proceed to step 22; Step 22: In the meter image to be identified, use external expansion calibration to perform a global search within the entire image pixel range, compare pixel information, and use the search area in the meter image to be identified that is similar to the meter ROI frame as the target meter ROI to obtain the target meter ROI frame coordinates; if the target meter ROI is not found, the identification fails.

5. According to claim 2, a multi-pointer meter reading recognition method based on traditional algorithm and deep learning, It is characterized in that Step 3 includes the following steps: Step 31: Use the Resize function to adjust the target meter ROI into a regular rectangle to obtain a corrected ROI image; Step 32: Using the Canny edge detection algorithm to convert the corrected ROI image into a feature map; Step 33: using the Hough transform line detection method to detect all the lines in the feature map, and using the meter center point neighborhood line filtering method to filter out invalid lines, to obtain a candidate line set; Step 34: Calculate the target meter center point coordinates, the target meter start scale point coordinates, the target meter end scale point coordinates, the target meter start range and the target meter end range through the target meter ROI frame coordinates and calibration information; take the target meter center point as the center of the circle and the start scale point coordinates and the end scale point coordinates as the radius, and take the average of the two radii as the target radius to form a target circle; Step 35: Calculate the two intersection points P of all candidate lines in the candidate line set and the target circle 1 and P 2 , the meter center point is connected to two intersection points P 1 and P 2 , obtain the two corrected opposite pointer lines corresponding to each candidate line to form a corrected line set; Step 36: The center point of the meter is connected to the coordinates of the starting scale point of the target meter and the coordinates of the ending scale point of the target meter to obtain the starting straight line and the ending straight line respectively, and the angle value θ between all the correction lines in the correction line set and the starting straight line is calculated. i , and the angle values ​​θ of the starting and ending lines, obtain the meter angle range [0, θ], and filter out the angle value θ i The correction lines within the meter angle range constitute a line subset; Step 37: Set the angle values ​​θ corresponding to all the corrected lines in the line subset i The angle difference is obtained by subtracting each other, and the correction lines with angle differences less than the set threshold are judged as repeated lines. A repeated line subset is constructed, and the angle value θ of each correction line is calculated in the repeated line subset. i Arrange in ascending order, select the correction line corresponding to the median angle value as the representative line, and the representative line and the correction line whose angle difference is greater than or equal to the set threshold constitute a valid line set; Step 38: Use the trained classification model to perform binary classification of the pointer and background on the valid straight line set to obtain the final valid straight line set.

6. According to claim 5, a multi-pointer meter reading recognition method based on traditional algorithm and deep learning, It is characterized in that In step 4, the angle values ​​θ corresponding to all the correction lines in the final valid straight line set are calculated. i and the meter angle range [0, θ] to calculate the meter reading, the expression is: Among them, S 0 Indicates the starting range of the target meter; S 1 Indicates the end range of the target meter.

7. According to claim 5, a multi-pointer meter reading recognition method based on traditional algorithm and deep learning, It is characterized in that The pixel setting range of the side length w in the corrected ROI image is: w∈[80, target table ROI shortest side length], w≥80.

8. According to claim 5, a multi-pointer meter reading recognition method based on traditional algorithm and deep learning, It is characterized in that The expression of the Hough transform line detection method is: ρ=xcosθ+ysinθ, where (x, y) represents the line coordinates; θ represents the line angle.

9. According to claim 5, a multi-pointer meter reading recognition method based on traditional algorithm and deep learning, It is characterized in that The expression of the meter center point neighborhood straight line filtering method is: Among them, (x 0 ,y 0 ) represents the coordinates of the center point of the target meter; r represents the neighborhood of the center point; d represents the distance between the candidate straight line and the center point of the target meter; and the candidate straight line with d<r is selected.

10. According to claim 5, a multi-pointer meter reading recognition method based on traditional algorithm and deep learning, It is characterized in that The boundary error is set to determine the meter angle range. The boundary error is ±1°, so the meter angle range is [0-1, θ+1].

11. According to claim 5, a multi-pointer meter reading recognition method based on traditional algorithm and deep learning, It is characterized in that In step 38, the ResNet18 neural network is used as the backbone network to build a classification model. The input of each residual block in the ResNet18 neural network is added to the output through a shortcut connection. Three fully connected layers are added to the head of the backbone network. The activation function of the backbone network uses the sigmoid function. The Adam optimizer and FocalLoss loss function are used to train the classification model.

Images