Electric meter detection method and device based on SVTRLCNetV3

Abstract

The invention discloses an electric meter detection method and device based on SVTRLCNetV3. The method comprises the following steps: setting a plurality of robot inspection points; constructing and training a state value identification model and a numerical value identification model of the intelligent polling ammeter; making a template drawing and an algorithm configuration file of each robot inspection point; training a bidirectional LSTM sequence result analyzer; the inspection robot records an electricity meter video; according to the ammeter video, the algorithm configuration file corresponding to the robot inspection point and the template drawing closest to the current time period, executing an algorithm detection program to obtain an ammeter reading sequence; and a bidirectional LSTM sequence result analyzer is used to carry out reasoning on the ammeter reading sequence, and an ammeter value corresponding to each state value is output. According to the method, effective detection can be realized for the intelligent polling ammeter with multiple states continuously and circularly switched, an interference identification result is effectively eliminated, and the accuracy of the identification result is finally ensured.

Description

Technical Field

[0001] The present invention belongs to the technical field of smart electric meters, and in particular relates to an electric meter detection method and device based on SVTR_LCNetV3. Background Art

[0002] At present, multi-state smart patrol meters are widely applied to the field of meter counting and identification. The main feature of smart patrol meters is that they can display more than ten different state values ​​on one device in a cycle. This can greatly reduce the number of digital display meters and integrate the information that needs to be displayed by multiple meters on one device. However, there are currently few visual inspection products for smart patrol meters. Visual identification of smart patrol meters has the following difficulties:

[0003] First, unlike ordinary meter reading recognition, smart patrol meters need to cyclically display more than ten built-in status values. According to on-site surveys, smart patrol meters will continuously jump status values ​​within 1 to 2 minutes, and each state will only stay for about 3 to 5 seconds. At the same time, the meter value in that state will be displayed, and the meter value will switch with the switching of the state value. The detection of smart patrol meters needs to correctly identify each state and the corresponding meter value within a complete patrol time, and complete the correct matching of each state and value. Frequent switching of states can easily cause detection errors. Therefore, the business logic of smart patrol meters is more complex than that of ordinary meters, and higher accuracy and real-time requirements are required.

[0004] Secondly, the scope of character recognition is greatly increased, indicating that the status value of the smart patrol meter is composed of Chinese characters and Roman numerals, while the meter value area is composed of Arabic numerals and decimal points. Therefore, the conventional meter value text recognition model is not suitable for smart patrol meters. Finally, when the state of the smart patrol meter is switched, the previous state value will change from dark to light, and the new variable of the next state value will gradually deepen from the blank space. During the detection during the state switching, the previous state value variable and the next state value variable coexist, and the color is lighter, so it is very easy to cause false alarms, which brings great challenges to the overall recognition of smart patrol meters. Summary of the invention

[0005] Technical purpose: In view of the above technical problems, the present invention proposes an electric meter detection method and device based on SVTR_LCNetV3, which can realize effective detection of smart patrol electric meters that continuously switch between multiple states, effectively eliminate interference in recognition results, and ultimately ensure the accuracy of recognition results.

[0006] Technical solution: To achieve the above technical objectives, the present invention adopts the following technical solution:

[0007] An electric meter detection method based on SVTR_LCNetV3, comprising the steps of:

[0008] (1) Set up multiple robot inspection points, each of which is equipped with an intelligent patrol meter;

[0009] (2) Using SVTR_LCNetV3 as the backbone network, the state value recognition model and value recognition model of the smart patrol meter are constructed and trained;

[0010] (3) Prepare template diagrams and algorithm configuration files for each robot inspection point. The algorithm configuration files contain preliminary status values ​​and character area coordinates of numerical values;

[0011] (4) Train the bidirectional LSTM sequence result analyzer;

[0012] (5) The inspection robot moves to any robot inspection point and records the meter video;

[0013] (6) Execute the algorithm detection program according to the meter video recorded in step (5), the algorithm configuration file corresponding to the robot inspection point, and the template diagram closest to the current time period to obtain a meter reading sequence that meets the predetermined format;

[0014] (7) Use the bidirectional LSTM sequence result analyzer trained in step (4) to infer the meter reading sequence obtained in step (6), and output the meter value corresponding to each state value of the smart patrol meter corresponding to the current meter video.

[0015] Preferably, the step (2) comprises the following steps:

[0016] (2.1) The inspection robot shoots video footage of electric meters at multiple time periods at each robot inspection point;

[0017] (2.2) Extract frames, label and augment the electric meter video material to obtain a labeled training data set and an unlabeled training data set;

[0018] (2.3) Using SVTR_LCNetV3 as the backbone network, the state value recognition model and value recognition model of the smart patrol meter are constructed, and the labeled training data set is used for model training;

[0019] (2.4) Use the trained state value recognition model and value recognition model of the smart patrol meter to automatically recognize characters on the unlabeled data in the unlabeled training data set to obtain a labeled file;

[0020] (2.5) Use the labeling tool PPOCRLabel to manually proofread and correct the automatically labeled data, and then add the proofread and corrected data to the labeled training data set to obtain an updated labeled training data set;

[0021] (2.6) Use the updated labeled training data set to retrain the character recognition model of the smart patrol meter status value and the character recognition model of the meter value.

[0022] Preferably, in the step (2.2), the frame extraction graph obtained by the frame extraction process is labeled in the following manner:

[0023] Use the OCR annotation tool PPOCRLabel to perform manual OCR annotation on the frame image. The annotated character box is expanded to include all possible areas where the status value characters of the smart patrol meter may appear. Use a combination of uppercase English letters to annotate the Roman numerals displayed by the smart patrol meter.

[0024] The training dataset is augmented as follows:

[0025] Use random offset and random scaling methods to process character boxes to achieve character annotation data augmentation;

[0026] The labeled data of the status value and the labeled data of the numerical value of the smart patrol meter are extracted from the constructed training data set, and all the characters of the status value including Chinese characters and uppercase English letters are added to the dictionary of status value character recognition, and all the characters of the numerical value including Arabic numerals and decimal points are added to the dictionary of numerical value character recognition.

[0027] The step (3) comprises the following steps:

[0028] (3.1) Adjust the position and rotation angle of the inspection robot and take photos of the smart patrol meters at each robot inspection point at different time periods, including the readings on the display screen of the smart patrol meters;

[0029] (3.2) Process the photo of the smart patrol meter taken in step (3.1), draw the character area and the numerical area of ​​the smart patrol meter status value, and save the character area and the numerical area in the form of coordinates of four corner points in the algorithm configuration file.

[0030] Preferably, the step (4) comprises the following steps:

[0031] (4.1) Create training data for the bidirectional LSTM sequence result analyzer:

[0032] Obtain the electric meter video material obtained in step (2.1), and decompose it to obtain frame images within a sequence of time;

[0033] Using the character area and value area information in the algorithm configuration file of step (3.2), and the character recognition model of the state value and the character recognition model of the value trained in step (2.6) to perform reasoning, obtain the recognition result of the state value and the corresponding value corresponding to each frame image;

[0034] The recognition results of the state value of each frame image and the corresponding numerical value are stored in a sequence according to the time series as the training data of the bidirectional LSTM sequence result analyzer. The recognition results of the state value use the state value dictionary table bit sequence directly inferred by the model as input, and do not need to be converted into Chinese and English characters according to the state value dictionary table.

[0035] Through manual observation, all state values ​​that appear in the corresponding video material and the corresponding correct values ​​are written in the self-defined state value sequence as labels for the training data;

[0036] (4.2) Determine whether the length of the sequence obtained in step (4.1) meets the specified length requirement. If it is less than the specified length, use the first and last elements in the sequence to fill the missing part; if it is more than the specified length, delete the excess part directly at the end of the sequence;

[0037] (4.3) According to steps (4.1) and (4.2), prepare training data and labels that meet the training requirements, divide the prepared training data into training set and validation set, use the cross entropy loss function and Adam optimizer to train the bidirectional LSTM sequence result analyzer, and save the training weights.

[0038] Preferably, the step (5) comprises the following steps:

[0039] (5.1) The inspection robot moves to the robot inspection point, adjusts the shooting position and angle, and uses the camera to record the video data of the complete cycle of the smart patrol meter according to the cycle time of the smart patrol meter, and stores it;

[0040] (5.2) According to step (5.1), the video data of the intelligent patrol meters of all robot inspection points are recorded in sequence;

[0041] (5.3) The inspection robot executes step (5.2) once every preset time period to obtain the video data of the intelligent patrol meter at multiple time periods at each robot inspection point.

[0042] Preferably, the step (6) comprises the following steps:

[0043] (6.1) performing frame extraction processing on the electric meter video recorded in step (5) to obtain a plurality of frame extraction images;

[0044] (6.2) Calculate the correction offset based on the frame extraction image obtained in step (6.1) and the template image of the corresponding robot inspection point;

[0045] (6.3) Calculate the character area coordinates of the corrected status value and value according to the corrected offset and the algorithm configuration file of the corresponding inspection point;

[0046] (6.4) extracting multiple state value ROI regions and multiple value ROI regions from the frame extraction image obtained in step (6.1) according to the corrected state value and value character region coordinates, and obtaining a state value region stitched image and a value region stitched image after stitching.

[0047] (6.5) Using the state value recognition model trained in step (2) to recognize the state value region stitching picture, and using the value recognition model trained in step (2) to recognize the value region stitching picture, to obtain the state value and the electric meter value;

[0048] (6.6) The state value and the meter value obtained in step (6.5) are stored as a meter reading sequence in a predetermined format.

[0049] Preferably, in the step (6.2), the lightglue matching algorithm is used to calculate the offset between the template image and each abstracted image, that is, feature points and descriptors are extracted from the template image and the abstracted image, a set of matching point pairs are found by the similarity of the feature descriptors, the offsets of all matching point pairs are calculated, and then all offsets are averaged to obtain a global offset (Δx, Δy):

[0050]

[0051] Among them, N is the number of matched feature point pairs, I a ,I b Represent the template image and the frame extraction image respectively, p i Belong to I a ,q i Belong to I b , p i and q i is a pair of matching points, Δx and Δy represent the offsets in the x and y directions between the template image and the image to be registered, respectively;

[0052] Then, the mean of the offsets between the template image and each frame extraction image is calculated, and the mean is used as the corrected offset;

[0053] An electric meter detection device based on SVTR_LCNetV3, comprising:

[0054] Inspection point setting module, used to set multiple robot inspection points, each robot inspection point is equipped with an intelligent patrol meter;

[0055] The recognition model construction and training module is used to build and train the state value recognition model and value recognition model of the smart patrol meter based on SVTR_LCNetV3 as the backbone network;

[0056] The calibration module is used to create template images and algorithm configuration files for each robot inspection point. The algorithm configuration files contain preliminary status values ​​and character area coordinates of numerical values.

[0057] Analyzer training module, used to train the bidirectional LSTM sequence result analyzer;

[0058] The meter video acquisition module moves the inspection robot to any robot inspection point to record the meter video;

[0059] The algorithm detection module is used to obtain the meter video transmitted by the meter video acquisition module, the algorithm configuration file corresponding to the robot inspection point and the template diagram closest to the current time period, execute the algorithm detection program, and obtain a meter reading sequence that meets the predetermined format;

[0060] The analysis and detection result output module is used to use the trained bidirectional LSTM sequence result analyzer to infer the meter reading sequence obtained by the algorithm detection module, and output the meter value corresponding to each state value of the smart patrol meter corresponding to the current meter video.

[0061] Preferably, the algorithm detection module includes:

[0062] A video data processing unit, used for receiving and recording the electric meter video and performing frame extraction processing to obtain a plurality of frame extraction images;

[0063] An offset correction unit is used to receive the frame extraction image transmitted by the video data processing unit, obtain the template image of the corresponding robot inspection point, and calculate the correction offset;

[0064] A character frame correction unit, used to calculate the character area coordinates of the corrected status value and value according to the correction offset and the algorithm configuration file of the corresponding inspection point;

[0065] The ROI region extraction and splicing unit is used to extract multiple state value ROI regions and multiple value ROI regions from the frame extraction image transmitted by the receiving video data processing unit according to the corrected state value and value character region coordinates, and obtain a state value region splicing picture and a value region splicing picture after splicing processing; the state value region splicing picture and the value region splicing picture are respectively input into the trained state value recognition model and the value recognition model to obtain the state value and the meter value;

[0066] The sequence generation unit is used to obtain the state value and meter value output by the state value recognition model and the value recognition model of the training intelligent patrol meter, and store them as a meter reading sequence in a predetermined format.

[0067] Beneficial effects: Due to the adoption of the above technical solution, the present invention has the following beneficial effects:

[0068] The present invention detects multi-state intelligent patrol meters based on a robot fixed-point inspection method. Through fixed-point area frame calibration and LightGlue matching algorithm, the accuracy of character area positioning can be improved, character interference in non-target areas can be eliminated, and the inference time of the character detection model can be saved, thereby improving the accuracy of the multi-state detection results of the meter. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 This is a flow chart of the intelligent patrol meter detection method based on SVTR_LCNetV3 proposed by the present invention;

[0070] Figure 2 The left is an example of a dictionary table of state values. Figure 2 On the right is an example of some sequence training data for training a bidirectional LSTM model;

[0071] Figure 3 A schematic diagram showing the order of label data for training a bidirectional LSTM model according to the specified state values;

[0072] Figure 4 To complete the label display diagram of the corresponding sequential state value;

[0073] Figure 5 This is a recognition result diagram of a certain intelligent patrol meter state one;

[0074] Figure 6 This is the recognition result diagram of a certain intelligent patrol meter state 2;

[0075] Figure 7 This is the recognition result diagram of state three of a smart patrol meter. DETAILED DESCRIPTION

[0076] The embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0077] Embodiment 1

[0078] like Figure 1 As shown, the present invention proposes a smart patrol meter detection method based on SVTR_LCNetV3, comprising the steps of:

[0079] S1: Set the robot inspection point, take photos and calibrate the inspection point template;

[0080] S2: The robot camera is located at each inspection point, recording the video data of each complete cycle of the smart patrol meter;

[0081] S3: After the collected video data is processed into frames, manual data labeling is performed;

[0082] S4: Use the associated character recognition network of SVTR_LCNetV3 and Transformer to train OCR character recognition models for status values ​​and meter values ​​respectively.

[0083] S5: Use the bidirectional LSTM network as the analyzer to process the character recognition result sequence. After the training is completed, input the recognition result of each frame and output the final state value of the smart meter in this round.

[0084] S6: In the inspection task, the complete patrol video of the smart patrol meter is collected and input into the program for parsing. The character areas are extracted and the corresponding OCR recognition model is used for reasoning. The bidirectional LSTM is used as the result analyzer. All the recognition results of this sequence are input and the final value corresponding to each state of the smart patrol meter is output.

[0085] In step S1, the setting of inspection points and calibration of inspection point templates include the following steps:

[0086] S11: Set the position and rotation angle of the inspection robot's pan-tilt fixed-point photography to ensure that the front position of the smart patrol meter can be clearly photographed and the reading on the display screen is clearly visible. Then, according to the set position and camera angle, take pictures of the smart patrol meter template at the inspection point at different time periods.

[0087] The template image is the original image of the camera screen that the camera gimbal takes pictures at a fixed angle and position according to the inspection settings. The shooting angle and position must be consistent with the task being performed, and the camera resolution and camera settings must be consistent with the task being performed. The meter values ​​in the image must be clearly visible and centered, with high recognition.

[0088] S12: Based on the photos of the smart patrol meters taken at the S11 inspection point, use software tools to draw character rectangular areas of the smart patrol meter status values ​​and rectangular areas of the smart patrol meter values ​​on the pictures, and save these rectangular areas in the form of coordinates of the four corner points in the configuration file of the algorithm.

[0089] In step S2, recording video data of a complete cycle of the smart patrol meter includes the following steps:

[0090] S21: The robot moves to the inspection point set in S11 and adjusts the shooting position. According to the cycle time of the smart patrol meter, the robot camera records the video data of the complete cycle of the smart patrol meter at a fixed point and stores it.

[0091] S22: According to the method of S21, the video data of the smart patrol meters of all inspection points are recorded in sequence.

[0092] S23: In order to take into account the possible changes in ambient light at different time periods, the smart patrol meter data collection task of the entire inspection path in step S22 is performed once every 4 hours to ensure that the data contains video materials of each time period until sufficient smart patrol meter video data is collected.

[0093] In step S3, the data annotation includes the following steps:

[0094] S31: saving and storing the collected smart patrol meter video at a fixed frame rate.

[0095] S32: For the frame images collected in step S31, a portion of the frame images are uniformly sampled and manually labeled by OCR using the public Baidu OCR labeling tool PPOCRLabel. During OCR labeling, the labeled character area box needs to be expanded to include all possible areas where the status value characters of the row may appear. Use a combination of multiple uppercase English letters to label the Roman numerals displayed on the meter. For example, use three uppercase letters 'I' to represent the Roman numeral III, and use one uppercase 'I' and one uppercase 'V' to represent the Roman numeral IV.

[0096] In step S4, respectively training the character recognition models of the state value and the meter value includes the following steps:

[0097] S41: In this solution, since the robot inspection can realize fixed-point detection, the calibration frame can be used to replace the character detection module. Since there is a small deviation in the robot's fixed-point photography, a certain limit of random offset and random scaling is used for the character frame in the character annotation data to realize the augmentation of the annotation data.

[0098] S42: Extract the annotation data of the status value of the smart patrol meter from the annotated data set, and add all the characters of the status value including Chinese characters and uppercase English letters to the dictionary of status value character recognition; then extract the annotation data of the numerical value of the smart patrol meter from the annotated data set, and add all the characters of the numerical value including Arabic numerals and decimal points to the dictionary of numerical character recognition.

[0099] S43: In the PP-OCRv4 architecture, SVTR_LCNetV3 was selected as the backbone network and optimized and improved on this basis. First, SVTR_LCNetV3 was used as a feature extractor, the preprocessed image was input and the feature map was output, which was input into the Transformer encoder after the feature conversion module, and the multi-head attention mechanism was used to capture information of different scales, and finally the Transformer decoder was output and converted into the final text. This network was used to train the character recognition model of the status value of the smart patrol meter and the character recognition model of the meter value respectively. The Transformer module was embedded in the SVTR_LCNetV3 backbone network, which increased the depth of the network and could improve the accuracy of the character recognition model to a certain extent. The SVTR_LCNetV3 backbone network can also be used directly to train the character recognition model.

[0100] S44: Using the two character recognition models trained based on step S43, perform automatic character recognition on the unlabeled data divided in step S32 and output a labeling file. Use the labeling tool PPOCRLabel to manually proofread and correct the automatically labeled data.

[0101] S45: Add the data marked in step S44 to the training data set, and train the character recognition model of the smart patrol meter status value and the character recognition model of the meter value again according to the operations in steps S41, S42 and S43.

[0102] The present invention sets inspection points at all locations where intelligent patrol meters need to be inspected by inspection robots, shoots inspection point templates for multiple time periods, and uses tools to mark the coordinate information of the character area to be identified on the intelligent patrol meter. By collecting materials for multiple time periods, disassembling them into frame images, using data augmentation and adopting the advanced SVTR_LCNetV3 character recognition network, the accuracy of character recognition can be effectively improved. The SVTR_LCNetV3 character recognition network used trains the state value character recognition model composed of Chinese and uppercase letters and the meter value character recognition model composed of Arabic numerals and decimal points respectively. With the help of the model's rapid automatic annotation combined with manual correction to expand the data set, and then using the expanded data set to optimize the recognition model, the accuracy of the model is improved.

[0103] In step S5, training the bidirectional LSTM sequence result analyzer includes the following steps:

[0104] S51: First, you need to make training data for the bidirectional LSTM sequence result analyzer. The bidirectional LSTM processes sequence information from both the forward and backward directions. First, decompose each segment of video data obtained in step S31 to obtain a frame image within a sequence time, and then use step S12 to calibrate the regional information in the data, and use the character recognition model for the state value of the intelligent patrol meter and the character recognition model for the meter value trained in step S45 to perform reasoning to obtain the state value information on each frame and the corresponding result value. Then, the recognition results of the state value of each frame and the corresponding value recognition results are stored in a sequence according to the time series as the training data of the bidirectional LSTM sequence result analyzer, wherein the recognition results of the state value use the state value dictionary table bit sequence directly inferred by the model as input, and there is no need to convert it into Chinese and English characters according to the state value dictionary table. Then, manually observe and write all the state value types and corresponding correct values ​​that appear in this video in the self-defined state value order as the label of the training data. Figure 2 and Figure 3 shown.

[0105] S52: Since the cycle time of each smart patrol meter is basically the same, a fixed frame rate is used to save the frame image, and the final total number of frame images is also basically the same. In order to ensure the consistency of the input data structure size, the present invention determines the length of the input state value recognition result sequence for each result analysis. If there is an error that causes the number of frame images extracted from the video to be less than the specified length, the first element and the last element in the sequence are used to fill the missing part. If there is an error that causes the number of frame images extracted from the video to be more than the specified length, the excess part is directly deleted at the end of the sequence, such as Figure 4 shown.

[0106] S53: Prepare enough training data and labels in the same way as step S51, and divide the training set and the validation set. Use cross entropy as the loss function and use the Adam optimizer. Then train the bidirectional LSTM result analyzer and save the training weights.

[0107] In step S6, the result of identifying each state value of the smart patrol meter in the inspection task includes the following steps:

[0108] S61: The robot performs the inspection task, moves to the inspection point, and records the complete loop video of the smart patrol meter at a preset camera angle. The video data to be detected recorded at the inspection point, the algorithm configuration file of the inspection point generated in step S12, and the template image closest to the current time period obtained in step S11 are input into the algorithm detection program of the smart patrol meter.

[0109] S62: In the intelligent patrol meter algorithm detection program, 5 frames of the video data to be detected provided by the inspection point are uniformly extracted. The previously calibrated template image is used to correct the possible offset error of the fixed-point photo taken during the task execution, and the position information of the meter value calibration box in the inspection point configuration file is updated according to the calculated offset. That is, the template image I a Extraction Frame I b Calculate the offset value of the Lightglue matching algorithm. First, extract feature points and descriptors from the template image and the frame image, and find a set of matching point pairs {(p i ,q i )}, where p i Belong to I a ,q i Belong to I b , then calculate the offsets of all matching point pairs, and then average all offsets to obtain the global offset (Δx, Δy). The calculation process can be expressed as:

[0110]

[0111] Where N is the number of matched feature point pairs, and x and y represent the x and y directions of the two-dimensional plane. The template image is averaged with the five extracted frames after calculating the offset. Fixed-point region frame calibration and lightglue matching algorithm can improve the accuracy of character region positioning, eliminate character interference in non-target areas, and save the inference time of the character detection model.

[0112] S63: extract the character rectangular area coordinates marked in the algorithm configuration file, and then add the offset calculated in step S62 to generate the final character rectangular area coordinates.

[0113] When performing the task of intelligent patrol meter detection, the present invention uses the lightglue matching algorithm to calculate the offset value, and obtains the final state value area and meter value area to be identified according to the character area coordinates and offset value of the configuration file. The video to be detected is disassembled into frame images, and then the state value result and meter value result of the frame are identified using the recognition model. Finally, the recognition results of each frame are synthesized into sequence data as the input of the analyzer network, and the final result of each state value is output. This method can achieve effective detection for intelligent patrol meters that continuously switch between multiple states, and can effectively eliminate interference recognition results, ultimately ensuring the accuracy of the recognition results.

[0114] S64: Split the video of the smart patrol meter to be detected into frame images to be detected at a certain frame rate. When detecting each frame of the image, extract multiple ROI areas related to the state value of the smart patrol meter from the whole image through the final state value character area coordinate information obtained in step S63, and use the state value character recognition model generated in S45 for inference after the images are spliced ​​to obtain the state value of the smart patrol meter in this frame; extract the ROI area related to the value of the smart patrol meter from the whole image through the final meter value character area coordinate information obtained in step S63, and use the meter value character recognition model generated in S45 for inference to obtain the specific value of the smart patrol meter in this frame. Finally, store the state value and specific meter value recognized by the frame.

[0115] S65: Store all the state value recognition results and value recognition results of each frame in step S64 into the analysis sequence, and then process the length of the analysis sequence in the same way as step S52. After the processing is completed, use the bidirectional LSTM trained in step S53 as the analyzer of the sequence results to perform inference and output the meter value corresponding to each state value. Figures 5 to 7 The figure shows the recognition results of a certain intelligent patrol meter in three different states.

[0116] like Figure 5 As shown in the figure, the status values ​​are displayed in two lines: "Reverse active power level" and "Electricity". After the obtained ROI areas are spliced, the two status values ​​are displayed on one picture, and the status value OCR recognition model can be directly used for one-time inference to obtain the result. Compared with the unspliced ​​pictures, the extended pictures after splicing take very little extra time, and the model inference time can be reduced once, and the computing efficiency can be improved by about 50%.

[0117] After the detection is completed, a visualization diagram is drawn on the original image based on the results of the bidirectional LSTM output and the calibration box information of the configuration file. The LSTM output is a set of values, sorted in the order of fixed state values, such as Figure 3 For example, the first row of values ​​corresponds to the "forward active total current", and the second row of values ​​corresponds to the "forward active average power". The numerical order of each state value in the final output result is determined by the order used when S51 makes LSTM labels. All sequence labels are input with the corresponding numerical values ​​in a unified state value order.

[0118] The output result of the entire detection and prevention method obtained by the present invention can be a structured numerical result or a visual effect diagram. The LSTM output is supplemented with the corresponding state value, and finally the result information in dictionary format is output, such as Figure 4 Or output the effect diagram for customers to view visually, such as Figure 5-Figure 7As shown. In actual projects, the output dictionary format results are uploaded to the platform for viewing, and the renderings are uploaded to the detection platform to facilitate personnel to confirm whether the recognition results are correct. The structured numerical results and the visual renderings are the output of the entire detection method.

[0119] The method of the present invention trains a bidirectional LSTM sequence result analyzer, uses the results of character recognition model inference of existing videos as training data, and then manually observes all state values ​​and corresponding results in a cycle as label data to train the bidirectional LSTM sequence result analyzer. In the comprehensive judgment stage of the results, the bidirectional LSTM result analyzer directly processes all sequence recognition results, and the multi-state detection results finally output have higher accuracy.

[0120] Embodiment 2

[0121] This embodiment proposes an electric meter detection device based on SVTR_LCNetV3, including:

[0122] Inspection point setting module, used to set multiple robot inspection points, each robot inspection point is equipped with an intelligent patrol meter;

[0123] The recognition model construction and training module is used to build and train the state value recognition model and value recognition model of the smart patrol meter based on SVTR_LCNetV3 as the backbone network;

[0124] The calibration module is used to create template images and algorithm configuration files for each robot inspection point. The algorithm configuration files contain preliminary status values ​​and character area coordinates of numerical values.

[0125] Analyzer training module, used to train the bidirectional LSTM sequence result analyzer;

[0126] The meter video acquisition module moves the inspection robot to any robot inspection point to record the meter video;

[0127] The algorithm detection module is used to obtain the meter video transmitted by the meter video acquisition module, the algorithm configuration file corresponding to the robot inspection point and the template diagram closest to the current time period, execute the algorithm detection program, and obtain a meter reading sequence that meets the predetermined format;

[0128] The analysis and detection result output module is used to use the trained bidirectional LSTM sequence result analyzer to infer the meter reading sequence obtained by the algorithm detection module, and output the meter value corresponding to each state value of the smart patrol meter corresponding to the current meter video.

[0129] Among them, the algorithm detection module includes:

[0130] A video data processing unit, used for receiving and recording the electric meter video and performing frame extraction processing to obtain a plurality of frame extraction images;

[0131] An offset correction unit is used to receive the frame extraction image transmitted by the video data processing unit, obtain the template image of the corresponding robot inspection point, and calculate the correction offset;

[0132] A character frame correction unit, used to calculate the character area coordinates of the corrected status value and value according to the correction offset and the algorithm configuration file of the corresponding inspection point;

[0133] The ROI region extraction and splicing unit is used to extract multiple state value ROI regions and multiple value ROI regions from the frame extraction image transmitted by the receiving video data processing unit according to the corrected state value and value character region coordinates, and obtain a state value region splicing picture and a value region splicing picture after splicing processing; the state value region splicing picture and the value region splicing picture are respectively input into the trained state value recognition model and the value recognition model to obtain the state value and the meter value;

[0134] The sequence generation unit is used to obtain the state value and meter value output by the state value recognition model and the value recognition model of the training intelligent patrol meter, and store them as a meter reading sequence in a predetermined format.

[0135] The present invention is implemented for multi-state intelligent patrol electric meters, can be widely used in practical applications and meet actual detection needs.

[0136] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the above embodiments do not limit the present invention in any form, and any technical solution obtained by equivalent replacement or equivalent transformation falls within the protection scope of the present invention.

Claims

1. An electric meter detection method based on SVTR_LCNetV3, characterized in that: Includes steps: (1) Set up multiple robot inspection points, each of which is equipped with an intelligent patrol meter; (2) Using SVTR_LCNetV3 as the backbone network, the state value recognition model and value recognition model of the smart patrol meter are constructed and trained; (3) Prepare template diagrams and algorithm configuration files for each robot inspection point. The algorithm configuration files contain preliminary status values ​​and character area coordinates of numerical values; (4) Train the bidirectional LSTM sequence result analyzer; (5) The inspection robot moves to any robot inspection point and records the meter video; (6) Execute the algorithm detection program according to the meter video recorded in step (5), the algorithm configuration file corresponding to the robot inspection point, and the template diagram closest to the current time period to obtain a meter reading sequence that meets the predetermined format; (7) Use the bidirectional LSTM sequence result analyzer trained in step (4) to infer the meter reading sequence obtained in step (6), and output the meter value corresponding to each state value of the smart patrol meter corresponding to the current meter video.

2. The electric meter detection method based on SVTR_LCNetV3 according to claim 1 is characterized in that: The step (2) comprises the following steps: (2.1) The inspection robot shoots video footage of electric meters at multiple time periods at each robot inspection point; (2.2) Extract frames, label and augment the electric meter video material to obtain a labeled training data set and an unlabeled training data set; (2.3) Using SVTR_LCNetV3 as the backbone network, the state value recognition model and value recognition model of the smart patrol meter are constructed, and the labeled training data set is used for model training; (2.4) Use the trained state value recognition model and value recognition model of the smart patrol meter to automatically recognize characters on the unlabeled data in the unlabeled training data set to obtain a labeled file; (2.5) Use the labeling tool PPOCRLabel to manually proofread and correct the automatically labeled data, and then add the proofread and corrected data to the labeled training data set to obtain an updated labeled training data set; (2.6) Use the updated labeled training data set to retrain the character recognition model of the smart patrol meter status value and the character recognition model of the meter value.

3. The electric meter detection method based on SVTR_LCNetV3 according to claim 2 is characterized in that: In the step (2.2), the frame extraction graph obtained by the frame extraction process is labeled in the following manner: Use the OCR annotation tool PPOCRLabel to perform manual OCR annotation on the frame image. The annotated character box is expanded to include all possible areas where the status value characters of the smart patrol meter may appear. Use a combination of uppercase English letters to annotate the Roman numerals displayed by the smart patrol meter. The training dataset is augmented as follows: Use random offset and random scaling methods to process character boxes to achieve character annotation data augmentation; The labeled data of the status value and the labeled data of the numerical value of the smart patrol meter are extracted from the constructed training data set, and all the characters of the status value including Chinese characters and uppercase English letters are added to the dictionary of status value character recognition, and all the characters of the numerical value including Arabic numerals and decimal points are added to the dictionary of numerical value character recognition.

4. The electric meter detection method based on SVTR_LCNetV3 according to claim 3 is characterized in that: The step (3) comprises the following steps: (3.1) Adjust the position and rotation angle of the inspection robot and take photos of the smart patrol meters at each robot inspection point at different time periods, including the readings on the display screen of the smart patrol meters; (3.2) Process the photo of the smart patrol meter taken in step (3.1), draw the character area and the numerical area of ​​the smart patrol meter status value, and save the character area and the numerical area in the form of coordinates of four corner points in the algorithm configuration file.

5. The electric meter detection method based on SVTR_LCNetV3 according to claim 4 is characterized in that: The step (4) comprises the following steps: (4.1) Create training data for the bidirectional LSTM sequence result analyzer: Obtain the electric meter video material obtained in step (2.1), and decompose it to obtain frame images within a sequence of time; Using the character area and value area information in the algorithm configuration file of step (3.2), and the character recognition model of the state value and the character recognition model of the value trained in step (2.6) to perform reasoning, obtain the recognition result of the state value and the corresponding value corresponding to each frame image; The recognition results of the state value of each frame image and the corresponding numerical value are stored in a sequence according to the time series as the training data of the bidirectional LSTM sequence result analyzer. The recognition results of the state value use the state value dictionary table bit sequence directly inferred by the model as input, and do not need to be converted into Chinese and English characters according to the state value dictionary table. Through manual observation, all state values ​​that appear in the corresponding video material and the corresponding correct values ​​are written in the self-defined state value sequence as labels for the training data; (4.2) Determine whether the length of the sequence obtained in step (4.1) meets the specified length requirement. If it is less than the specified length, use the first and last elements in the sequence to fill the missing part; if it is more than the specified length, delete the excess part directly at the end of the sequence; (4.3) According to steps (4.1) and (4.2), prepare training data and labels that meet the training requirements, divide the prepared training data into training set and validation set, use the cross entropy loss function and Adam optimizer to train the bidirectional LSTM sequence result analyzer, and save the training weights.

6. The electric meter detection method based on SVTR_LCNetV3 according to claim 1 is characterized in that: The step (5) comprises the following steps: (5.1) The inspection robot moves to the robot inspection point, adjusts the shooting position and angle, and uses the camera to record the video data of the complete cycle of the smart patrol meter according to the cycle time of the smart patrol meter, and stores it; (5.2) According to step (5.1), the video data of the intelligent patrol meters of all robot inspection points are recorded in sequence; (5.3) The inspection robot executes step (5.2) once every preset time period to obtain the video data of the intelligent patrol meter at multiple time periods at each robot inspection point.

7. The electric meter detection method based on SVTR_LCNetV3 according to claim 1 is characterized in that: The step (6) comprises the following steps: (6.1) performing frame extraction processing on the electric meter video recorded in step (5) to obtain a plurality of frame extraction images; (6.2) Calculate the correction offset based on the frame extraction image obtained in step (6.1) and the template image of the corresponding robot inspection point; (6.3) Calculate the character area coordinates of the corrected status value and value according to the corrected offset and the algorithm configuration file of the corresponding inspection point; (6.4) extracting multiple state value ROI regions and multiple value ROI regions from the frame extraction image obtained in step (6.1) according to the corrected state value and value character region coordinates, and obtaining a state value region stitched image and a value region stitched image after stitching. (6.5) Using the state value recognition model trained in step (2) to recognize the state value region stitching picture, and using the value recognition model trained in step (2) to recognize the value region stitching picture, to obtain the state value and the electric meter value; (6.6) The state value and the meter value obtained in step (6.5) are stored as a meter reading sequence in a predetermined format.

8. The electric meter detection method based on SVTR_LCNetV3 according to claim 1, characterized in that: In the step (6.2), the lightglue matching algorithm is used to calculate the offset between the template image and each frame image, that is, feature points and descriptors are extracted from the template image and the frame image, and a set of matching point pairs p is found by the similarity of the feature descriptors. i and q i , calculate the offsets of all matching point pairs, and then average all offsets to obtain the global offset (Δx, Δy): Among them, N is the number of matched feature point pairs, I a ,I b Represent the template image and the frame extraction image respectively, p i Belong to I a ,q i Belong to I b , p i and q i is a pair of matching points, Δx and Δy represent the offsets in the x and y directions between the template image and the image to be registered, respectively; Then, the average of the offsets between the template image and each frame extraction image is calculated, and the average is used as the corrected offset.

9. An electric meter detection device based on SVTR_LCNetV3, characterized in that: include: Inspection point setting module, used to set multiple robot inspection points, each robot inspection point is equipped with an intelligent patrol meter; The recognition model construction and training module is used to build and train the state value recognition model and value recognition model of the smart patrol meter based on SVTR_LCNetV3 as the backbone network; The calibration module is used to create template images and algorithm configuration files for each robot inspection point. The algorithm configuration files contain preliminary status values ​​and character area coordinates of numerical values. Analyzer training module, used to train the bidirectional LSTM sequence result analyzer; The meter video acquisition module moves the inspection robot to any robot inspection point to record the meter video; The algorithm detection module is used to obtain the meter video transmitted by the meter video acquisition module, the algorithm configuration file corresponding to the robot inspection point and the template diagram closest to the current time period, execute the algorithm detection program, and obtain a meter reading sequence that meets the predetermined format; The analysis and detection result output module is used to use the trained bidirectional LSTM sequence result analyzer to infer the meter reading sequence obtained by the algorithm detection module, and output the meter value corresponding to each state value of the smart patrol meter corresponding to the current meter video.

10. The electric meter detection device based on SVTR_LCNetV3 according to claim 9, characterized in that: The algorithm detection module includes: A video data processing unit, used for receiving and recording the electric meter video and performing frame extraction processing to obtain a plurality of frame extraction images; An offset correction unit is used to receive the frame extraction image transmitted by the video data processing unit, obtain the template image of the corresponding robot inspection point, and calculate the correction offset; A character frame correction unit, used to calculate the character area coordinates of the corrected status value and value according to the correction offset and the algorithm configuration file of the corresponding inspection point; The ROI region extraction and splicing unit is used to extract multiple state value ROI regions and multiple value ROI regions from the frame extraction image transmitted by the receiving video data processing unit according to the corrected state value and value character region coordinates, and obtain a state value region splicing picture and a value region splicing picture after splicing processing; the state value region splicing picture and the value region splicing picture are respectively input into the trained state value recognition model and the value recognition model to obtain the state value and the meter value; The sequence generation unit is used to obtain the state value and meter value output by the state value recognition model and the value recognition model of the training intelligent patrol meter, and store them as a meter reading sequence in a predetermined format.

Images