Method and device for automatically checking electric meter defect work order based on machine vision
Through machine vision automated review of power meter defect work orders, deep learning models are used to extract and compare defect information of power meter and work order images, solving the problem of time-consuming and labor-consuming manual review and achieving efficient and accurate automated review.
Patent Information
- Application Number
- CN202510308880.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-03-17
- Publication Date
- 2025-07-04
AI Technical Summary
In the prior art, the inspection of electricity meter defects relies on manual naked eye comparison to consume a lot of manpower and time, and it is impossible to efficiently process a large amount of safety inspection work order information.
Using an automated audit method based on machine vision, the key points of work orders and meter images are extracted through deep learning models, perspective transformation and OCR recognition are performed, defect label lists are generated, and defect label consistency comparison is performed to realize automated audits.
It greatly shortens the audit time and labor costs, improves the audit efficiency, ensures the accuracy and objectivity of the audit, and can quickly process a large amount of work order information.
Smart Images
Figure CN120260052A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of electric energy metering and collection, and specifically to a method and device for automatically auditing electric meter defect work orders based on machine vision. Background Art
[0002] The work of customer electricity safety management is an important measure to ensure customer electricity safety, discover electricity safety hazards, and complete electricity defect elimination. Among them, the inspection of customer electric meter defects is a very important part of the customer electricity safety management work. Specifically, the inspection of customer electric meter defects requires inspectors to go to the site for manual inspection. According to the inspection situation, fill in the safety inspection work order form, and take pictures of the filled work order and the defective electric meter and upload them to the background server. Currently, for the background audit of safety inspection work orders, it mainly relies on manual visual comparison of the photos of the uploaded work order form and the photos of the defective electric meter to detect whether the uploaded electric meter photo is really defective, and whether the type of electric meter defect is consistent with the type filled in the work order form. Now, taking Fujian as an example, nearly 100,000 network-related safety inspection work order information can usually be collected every year. Therefore, this will consume a large amount of manpower and time. Summary of the Invention
[0003] In view of the deficiencies of the prior art, the present invention provides a method and device for automatically auditing electric meter defect work orders based on machine vision, which solves the existing problems.
[0004] To achieve the above objectives, the present invention is realized through the following technical solutions: A method for automatically auditing electric meter defect work orders based on machine vision, including the following steps:
[0005] Step S1, Work order image processing:
[0006] S11, Key point extraction: Use the YOLO series model to extract the key point coordinates after training with a semi-automatic constructed sample set;
[0007] S12, Perspective transformation: Correct the work order image to a standard image according to the key point coordinates;
[0008] S13, Text area extraction: Extract the handwritten text area of the corrected image according to the border coordinates of the standard image;
[0009] S14, OCR recognition: Use a mainstream model (such as RCNN) to construct training data by semi-automatically simulating the construction of a training sample model to recognize text;
[0010] S15, Text defect extraction: Construct a keyword dictionary tree, and use a matching algorithm to extract text defect category information to form a label list;
[0011] Step S2, Electric meter image processing:
[0012] S21. Key point extraction: Use the YOLO series of models to construct a sample set by simulating the construction of a training sample model, and extract the key point coordinates after training.
[0013] S22. Perspective transformation: Correct the electricity meter image to a standard image according to the key point coordinates.
[0014] S23. Pixel comparison: Compare the pixels of the images within the pre-set border areas of the standard image and the corrected image, and judge the defects to form a label list of the image defect category information.
[0015] Step S3. Defect label consistency comparison: Judge whether the information is consistent through the intersection of the label lists obtained from the work order and the electricity meter image, and realize the audit of the work order information for safety inspection.
[0016] Preferably, the specific steps for key point extraction of the work order image include: adopting a deep learning image key point recognition model of the YOLO series, constructing a training sample set by a semi-automatic simulation method for constructing training samples. First, prepare 10 - 100 standard work order atlases and set the key point coordinates (such as the four corners of the work order document), then prepare 10 - 100 common work order background atlases, and then randomly select standard work order images and background images. After performing a random perspective transformation on the standard work order image, embed it into the background image to form a simulated work order image, and at the same time perform the same perspective transformation on the corresponding key point coordinates as the regression label.
[0017] Preferably, the specific steps for perspective transformation of the work order image include: using the key point coordinates obtained by the work order key point recognition model, correcting the original work order image to a standard work order image through perspective transformation, so that it corresponds to the key point coordinates of the pre-determined standard work order image.
[0018] Preferably, the specific steps for extracting the handwritten text area include: according to the characteristic that the handwritten text areas of the corrected work order image and the standard work order image are the same, extract the handwritten text area of the corrected work order image according to the border coordinates of the handwritten text area of the pre-set standard work order image.
[0019] Preferably, the specific steps for OCR text recognition include: adopting a mainstream text recognition deep learning model, constructing training data by a semi-automatic simulation method for constructing training samples. First, sort out the character set involved in the handwritten content, construct 10 - 50 bitmap images of handwritten font styles for each character, prepare 10 - 100 document background sets under different illuminations, and then randomly generate simulated handwritten text area images and determine the corresponding label values.
[0020] Preferably, the specific steps for the text defect category include: pre-sort out a keyword set representing different defect categories to construct a trie tree, and adopt a keyword matching algorithm based on the trie tree to extract defect category information from the string recognized by OCR text.
[0021] Preferably, the specific steps for extracting key points of the electric meter image include: selecting a mainstream deep learning image key point recognition model, constructing a training sample set in a semi-automatic simulation way to construct training samples, preparing a standard normal electric meter diagram and setting key point coordinates (such as the four corners of the liquid crystal display frame of the electric meter), preparing 10 - 100 common electric meter background diagram sets, generating simulated electric meter diagrams through random simulation, and at the same time performing perspective transformation on the corresponding key point coordinates as the regression label.
[0022] Preferably, the specific steps for perspective transformation of the electric meter image include: with the help of the key point coordinates obtained by the electric meter key point recognition model, correcting the original electric meter diagram into a standard electric meter diagram through perspective transformation.
[0023] Preferably, the specific steps for image pixel comparison include: previously setting the area border of the concerned component in the standard normal electric meter image, and sequentially performing pixel comparison on the images within the same area border of the corrected electric meter image and the standard normal electric meter image, so as to judge whether there are defects in the component, and further obtain the image defect category information.
[0024] The present invention also discloses a device for automatically auditing electric meter defect work orders based on machine vision, including:
[0025] A key point extraction module, used for extracting the key point coordinates of the work order image and the key point coordinates of the electric meter image;
[0026] A perspective transformation module, used for correcting the work order diagram into a standard diagram according to the key point coordinates, and correcting the electric meter diagram into a standard diagram according to the key point coordinates;
[0027] A text area extraction module, used for extracting the handwritten text area of the corrected work order image for subsequent OCR recognition module, used for recognizing the handwritten text area image through a mainstream model and converting it into a text string;
[0028] A text defect extraction module, used for extracting defect category information from the text string recognized by OCR through a matching algorithm to generate a work order defect label list;
[0029] A pixel comparison module, used for comparing the image pixels within the corresponding borders of the corrected electric meter diagram and the standard diagram to judge defects and generate an electric meter defect label list;
[0030] A defect label consistency comparison module, used for comparing the intersection of the defect label lists of the work order and the electric meter image, and judging whether the information is consistent by comparing elements to audit the safety inspection work order information.
[0031] The present invention provides a method and device for automatically auditing electric meter defect work orders based on machine vision.
[0032] Compared with the prior art, it has the following beneficial effects:
[0033] 1. The method and device for automatically auditing electricity meter defect work orders based on machine vision respectively extract defect information based on the work order image and defect information based on the electricity meter image, and finally compare whether the two defect information is consistent, so as to realize the audit and verification of safety inspection work order information, make full use of the advantages of the current deep learning model in image processing, and realize the automatic audit and inspection of the above work order pictures and electricity meter pictures, which can greatly shorten the audit time and manpower, and can effectively ensure the audit accuracy.
[0034] 2. The method and device for automatically auditing electricity meter defect work orders based on machine vision automatically complete the processes of processing, analyzing the work order image and the electricity meter image, and comparing the consistency of defect labels through the coordinated cooperation of multiple modules, without manual visual comparison one by one, which can greatly improve the audit efficiency, quickly process a large amount of work order information, save labor costs, utilize advanced machine vision technologies, such as accurate key point extraction, image pixel comparison, etc., and extract and compare defect category information based on algorithms, which is more objective and accurate than manual audit, and can more accurately judge whether the electricity meter defect type is consistent with the type filled in the work order form, ensuring the accuracy of the audit of safety inspection work order information. BRIEF DESCRIPTION OF THE DRAWINGS
[0035] Figure 1 It is a schematic flowchart of the method of the present invention;
[0036] Figure 2 It is a schematic diagram of the module connection of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS
[0037] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.
[0038] Refer to Figure 1-2 , the present invention provides the following two technical solutions:
[0039] The first embodiment: A method for automatically auditing electricity meter defect work orders based on machine vision includes the following steps:
[0040] Step S1, Work order image processing:
[0041] S11. Key point extraction: Use a deep learning image key point recognition model of the YOLO series to construct a training sample set by means of a semi-automatic simulated training sample construction method. First, prepare 10 - 100 standard work order atlas and set the key point coordinates (such as the four corners of the work order document), then prepare 10 - 100 common work order background atlas. Then randomly select the standard work order map and the background map, perform a random perspective transformation on the standard work order map and embed it into the background map to form a simulated work order map, and at the same time perform the same perspective transformation on the corresponding key point coordinates as the regression label;
[0042] S12. Perspective transformation: Use the key point coordinates obtained by the work order key point recognition model to correct the original work order map into a standard work order map through perspective transformation, so that it corresponds to the key point coordinates of the standard work order map determined in advance;
[0043] S13. Handwritten text area extraction: Based on the characteristic that the handwritten text area of the corrected work order image is consistent with that of the standard work order map, extract the handwritten text area of the corrected work order image according to the border coordinates of the handwritten text area of the standard work order map set in advance;
[0044] S14. OCR recognition: Use a mainstream deep learning model for text recognition to construct training data by means of a semi-automatic simulated training sample construction method. First, sort out the character set involved in the handwritten content, construct 10 - 50 bitmap of handwritten font styles for each character, prepare 10 - 100 document background sets under different illuminations, and then randomly generate simulated handwritten text area maps and determine the corresponding label values;
[0045] S15. Text defect extraction: Sort out a keyword set representing different defect categories in advance to construct a trie tree, and use a keyword matching algorithm based on the trie tree to extract defect category information from the string recognized by OCR text;
[0046] Step S2. Electric meter image processing:
[0047] S21. Key point extraction: Select a mainstream deep learning image key point recognition model to construct a training sample set in the way of semi-automatic simulated training sample construction. Prepare a standard normal electric meter map and set the key point coordinates (such as the four corners of the electric meter liquid crystal display frame), prepare 10 - 100 common electric meter background atlas, generate a simulated electric meter map through random simulation, and at the same time perform a perspective transformation on the corresponding key point coordinates as the regression label;
[0048] S22. Perspective transformation: With the help of the key point coordinates obtained by the electric meter key point recognition model, correct the original electric meter map into a standard electric meter map through perspective transformation;
[0049] S23. Pixel comparison: Set the area border of the component of interest in the standard normal electricity meter image in advance, and sequentially perform pixel comparison on the images within the same area border of the corrected electricity meter image and the standard normal electricity meter image, so as to determine whether there are defects in the component, and further obtain the image defect category information;
[0050] Step S3. Defect label consistency comparison: Take the intersection of the defect label list obtained from the work order image and the defect label list obtained from the electricity meter image, and judge whether the label information provided by the two is consistent, so as to realize the review of the safety inspection work order information.
[0051] The second implementation method: The present invention also discloses a device for automatically auditing electricity meter defect work orders based on machine vision, including:
[0052] The key point extraction module is used to extract the key point coordinates of the work order image and the key point coordinates of the electricity meter image, aiming to use advanced deep learning image key point recognition technology to extract key coordinate information for the work order image and the electricity meter image respectively, providing basic support for subsequent operations such as image correction and defect analysis, and is the starting key link in the entire defect category extraction process.
[0053] During the extraction of the key points of the work order image: Select a mainstream deep learning model such as the YOLO series that performs well in the field of image key point recognition to execute the task of extracting the key points of the work order image. Given that the work order image has a specific generation mode, that is, it first exists in the form of a printed A4 document, and then people will fill in relevant defect information at fixed positions, and finally take a photo to form a work order image. Based on this characteristic, a semi-automatic simulated construction training sample method is adopted to construct a training sample set, so as to train the model efficiently and at low cost.
[0054] During specific operations, first prepare about 10 - 100 standard work order atlases, and clearly set the key point coordinates for these standard work order maps. For example, often set the four corners of the work order document as key points, which can well determine the overall layout and key position information of the work order image. At the same time, about 10 - 100 common work order background atlases also need to be prepared. Next, use the method of random simulation to generate training samples. Randomly select a standard work order map from the standard work order atlas, and then randomly select a background map from the background atlas. Perform a random perspective transformation on the standard work order map, and then embed the perspective-transformed standard work order map into the selected background map, thus forming the final simulated work order map. At the same time, perform the same perspective transformation operation on the key point coordinates originally corresponding to the standard work order map, and these transformed key point coordinates are used as the regression labels when training the model, so that the model can learn the corresponding relationship of the key point coordinates under different transformation situations, and thus accurately identify the key point coordinates in the work order image.
[0055] In the process of extracting key points from the electricity meter image: It also relies on the mainstream deep learning image key point recognition model. Models like the YOLO series are also applicable to the work of extracting key points from the electricity meter image. The construction idea of the training samples for the key points of the electricity meter image is highly similar to the construction method of the training samples for the key points of the work order image. The semi-automatic simulation method is also used to construct training samples to quickly produce the required training sample set, thereby reducing the large amount of time cost consumed by manually annotating training samples.
[0056] The specific construction process is as follows. First, prepare a standard normal electricity meter image. According to the structural characteristics of the electricity meter and subsequent analysis requirements, set the key point coordinates of the standard normal electricity meter image. For example, set the four corners of the liquid crystal display frame of the electricity meter as key points, which is of great significance for determining the key display area and overall layout of the electricity meter. Then prepare about 10 - 100 common electricity meter background image sets. Subsequently, use the random simulation method to generate training samples, that is, randomly select a background image from the electricity meter background image set, perform a random perspective transformation on the standard electricity meter image, and embed the perspective-transformed standard electricity meter image into this background image to form a simulated electricity meter image. Correspondingly, perform the same perspective transformation on the original key point coordinates corresponding to the standard electricity meter image, and use the transformed key point coordinates as the regression label for training, so that the model can learn the ability to accurately extract the key point coordinates of the electricity meter image based on this.
[0057] The perspective transformation module is used to correct the work order image into a standard image according to the key point coordinates, and to correct the electricity meter image into a standard image according to the key point coordinates. Its core function is to use the mathematical transformation means of perspective transformation based on the key point coordinate information obtained by the previous key point extraction module to correct the work order image and the electricity meter image into standard images respectively, so that various subsequent image processing and analysis can be carried out on the basis of a unified and standardized image, greatly improving the accuracy and convenience of analysis, and ensuring the coherence and stability of the entire defect category extraction process.
[0058] The specific process of perspective transformation of the work order image: After obtaining the corresponding key point coordinates through the work order image key point recognition model, by means of the principle and algorithm of perspective transformation, accurately map the key point coordinates extracted from the work order image to the key point coordinate positions of the pre-determined standard work order image. Based on such the same perspective transformation operation, the original work order image can be corrected into a standard work order image. This process is of great significance because the corrected work order image is consistent with the standard image in terms of layout, proportion, etc., especially the positions of key parts such as the handwritten text area become standardized and unified, laying a solid foundation for subsequent accurate extraction of the handwritten text area, text recognition, and defect category extraction, etc., and avoiding interference and errors caused by factors such as the image shooting angle and placement position.
[0059] Specific process of perspective transformation for electricity meter images: For electricity meter images, after their key points are accurately recognized by the key point recognition model of electricity meter images and the corresponding key point coordinates are obtained, the perspective transformation method is also used to map the key point coordinates of these electricity meter images to the key point coordinate positions of the standard electricity meter diagram according to established rules. Through the same perspective transformation process, the original electricity meter image can be corrected to a standard electricity meter diagram. In this way, the corrected electricity meter image and the standard normal electricity meter image are in a completely consistent state in terms of the position correspondence of each component, facilitating subsequent operations such as pixel comparison within the same regional border, helping to accurately judge whether there are defects in the electricity meter components, and being an indispensable link for accurately extracting defect category information based on images.
[0060] The text region extraction module is used to extract the handwritten text region of the corrected work order image for subsequent recognition. By making full use of the consistency feature of the handwritten text region between the corrected work order image and the standard work order diagram, according to the pre-set border coordinates of the handwritten text region of the standard work order diagram, the handwritten text region is accurately extracted from the corrected work order image, providing accurate image input for subsequent OCR text recognition, ensuring that the text recognition work can focus on the region that truly contains key handwritten content, and improving the recognition efficiency and accuracy.
[0061] Since the handwritten text region of the corrected work order image shows a completely consistent state with the handwritten text region of the standard work order diagram after being processed by the perspective transformation module, this is based on the characteristic that the relative position relationship of each part of the image remains unchanged during the perspective transformation process. Based on this favorable condition, according to the carefully set border coordinates of the handwritten text region of the standard work order diagram in advance, through corresponding image segmentation or extraction algorithms, the corresponding handwritten text region is completely and accurately extracted from the corrected work order image. The extracted handwritten text region images carry important text information related to defects filled in manually in the work order and will be sent to the OCR recognition module later for further conversion into a text string form for analysis.
[0062] The OCR recognition module is used to recognize the handwritten text region image through mainstream models and convert it into a text string. By using current mainstream deep learning models for text recognition, such as RCNN, etc., efficient and accurate text recognition processing is carried out on the handwritten text region images extracted from the work order image, converting the handwritten text content in image form into a clear text string form, thus realizing the key conversion from image information to text information, creating conditions for subsequent in-depth mining of defect category information contained in the text, and playing an important role in connecting the preceding with the following in the entire defect category extraction process based on work order images.
[0063] Considering the characteristics that the handwritten content in the work order mainly revolves around meter defect-related vocabulary and the overall vocabulary is relatively limited, a semi-automatic simulated construction training sample method is adopted to construct the training data for handwritten text recognition, thereby significantly reducing the workload of manually annotating training samples.
[0064] When specifically constructing the training data, first, all the characters involved in the handwritten content, including Chinese characters, letters, numbers, and punctuation marks, etc., are sorted out to form a complete character set. Then, for each character in the character set, 10 - 50 different handwritten font style bitmaps are carefully constructed in advance. Here, the bitmap is an image representation form where the area covered by the character lines takes the value of 1 and the rest of the area takes the value of 0, aiming to simulate various possible handwritten font forms. In addition, 10 - 100 document background sets under different lighting conditions are prepared to simulate different background situations that may appear in the actual work order image.
[0065] When actually generating the simulated handwritten text area map for training, first, a handwritten style is randomly selected, then a string of a specified length (for example, length 20) is randomly selected from the character set. Subsequently, the bitmaps of each character in the string corresponding to the handwritten style are sequentially stitched together in the order of columns to generate the bitmap corresponding to the string. After that, a background image is randomly selected from the document background set, and the area with a value of 0 in the string bitmap is filled with the selected background image. For the area with a value of 1 in the bitmap, a certain color pixel value (such as red, blue, black, etc.) is randomly selected for filling. In this way, the simulated handwritten text area map is successfully constructed, and at the same time, the string itself serves as the label value corresponding to this simulated handwritten text map.
[0066] After constructing the training data using the above method, a mainstream text recognition deep learning model such as RCNN is used for training to enable it to learn the mapping relationship between the handwritten text area image and the corresponding text string. When actually recognizing the handwritten text area picture extracted from the work order image, the model can accurately convert the handwritten text in the image into a text string based on the learned knowledge, providing accurate text input content for the subsequent text defect extraction module.
[0067] The text defect extraction module is used to extract defect category information from the text string recognized by OCR through a matching algorithm, generate a list of work order defect labels. By skillfully applying the keyword matching algorithm based on the trie tree, it deeply mines the defect category information contained in the text string output by the OCR recognition module, accurately extracts and organizes this information, and finally generates a list of defect labels corresponding to the work order image, providing a core basis for comprehensively and accurately judging the meter defect situation involved in the work order. It is a key link in the process of converting text information into defect category information in the entire defect category extraction process based on work order images.
[0068] Since the meter defect categories are relatively fixed in actual application scenarios, a keyword set that can represent different defect categories can be systematically sorted out in advance. On this basis, use these keywords to construct a trie tree structure. The trie tree is an efficient data structure that can quickly perform keyword matching and search operations on the input text string.
[0069] After obtaining the text string output by the OCR recognition module, input it into the constructed trie tree and process it using the keyword matching algorithm based on the trie tree. The algorithm will quickly scan and search for the parts in the text string that match the keywords pre-stored in the trie tree. Once a matching keyword is found, it means that the corresponding defect category information has been found. By traversing the entire text string and collecting, organizing, and classifying the defect category information corresponding to all the matching keywords, a complete list of defect labels corresponding to the work order image is finally generated. This list clearly shows the possible defect category situations of the meters involved in the work order, providing important data support for subsequent defect label consistency comparison and the review of the overall safety inspection work order information.
[0070] The pixel comparison module is used to compare the image pixels within the corresponding borders of the corrected meter image and the standard image, judge defects, and generate a list of meter defect labels. For the corrected meter image and the standard normal meter image, according to the pre-set border of the area of interest components, perform a pixel-by-pixel detailed comparison of the image pixels within the same area border. By analyzing the pixel differences, accurately judge whether there are defects in the components, and then generate a list of defect labels corresponding to the meter image, providing an intuitive and accurate judgment basis for the defect category extraction based on the meter image. It is the core judgment link in the entire meter image defect analysis process.
[0071] Considering that after the corrected electricity meter image and the standard normal electricity meter image are processed by the perspective transformation module, the positions corresponding to each component are completely the same, which creates favorable conditions for pixel comparison. Before actual operation, according to the structural characteristics of the electricity meter and the experience summary of the positions and types of electricity meter defects in the past, in the standard normal electricity meter image, corresponding regional borders are carefully set for the positions of components that need to be focused on and may have defects, clearly defining the image range that needs to be compared and analyzed.
[0072] Then, for the image parts within these same regional borders of the corrected electricity meter image and the standard normal electricity meter image, an image pixel comparison algorithm is used to compare the pixel attribute information such as color values and brightness values pixel by pixel. If obvious differences are found between the corresponding pixels and this difference exceeds the reasonable range caused by normal image acquisition, processing, etc., then it can be judged that there is a defect in this position of the component. By performing such pixel comparison operations on all the images within the set regional borders and recording and sorting out the defect category information corresponding to the components judged to have defects, a defect label list corresponding to the electricity meter image is finally generated. This list details the defect situations presented in the electricity meter image, providing an important reference basis for subsequent defect label consistency comparison and the review of overall safety inspection work order information.
[0073] The defect label consistency comparison module is used to compare whether the element judgment information is consistent by taking the intersection of the defect label lists of the work order and the electricity meter image, and to review the safety inspection work order information. By performing an intersection operation on the defect label lists obtained from the work order image and the electricity meter image respectively, and deeply comparing the relationship between the intersection elements and the elements of the overall list, it rigorously judges whether the label information provided by the two is highly consistent, so as to achieve a comprehensive and accurate review of the safety inspection work order information, timely discover possible inconsistencies between the work order information and the actual situation of the electricity meter, and thus provide a key basis for subsequent further inspections, verifications, and decision-making. It is the final summary and judgment link in the process of defect category extraction and review based on the work order image and the electricity meter image.
[0074] First, obtain the defect label list generated after a series of processes on the work order image, and the defect label list generated after the corresponding processing process on the electricity meter image. Then perform an intersection operation on these two defect label lists to obtain the set of common label elements. Next, carefully compare this set of intersection elements with all the elements in the two original defect label lists to check whether the intersection elements completely cover the key defect category information in the two lists, and whether there are important difference elements outside the intersection but in the original lists.
[0075] If it is found that the tag information provided by the two is exactly the same after comparison, it indicates that the work order information matches the actual condition of the electric meter, which means that the content recorded in the safety inspection work order largely conforms to the true defect situation of the electric meter, and it can be considered that the current safety inspection information is reliable. On the contrary, if differences are found, that is, the intersection elements do not fully cover the important defect category information in the two lists, or there are some key tag elements that exist in one list but are missing in the other list, then this indicates that there may be information deviation or equipment abnormality. At this time, further in-depth inspection and verification work need to be carried out, such as re-checking the content filled in the work order, conducting more detailed detection of the electric meter, etc., in order to accurately find out the reason and ensure the accuracy and effectiveness of the safety inspection work.
[0076] At the same time, the content not described in detail in this specification belongs to the prior art well-known to those skilled in the art, and the model parameters of each electrical appliance are not specifically limited, and conventional equipment can be used.
[0077] It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device.
[0078] Although the embodiments of the present invention have been shown and described, those of ordinary skill in the art can understand that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principle and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.
Claims
1. A method for automatically auditing electricity meter defect work orders based on machine vision, characterized in that, It includes the following steps: Step S1, work order image processing: S11. Key point extraction: Use the YOLO series of models to extract the key point coordinates after training with a semi - automatic constructed sample set; S12. Perspective transformation: Correct the work order image into a standard image according to the key point coordinates; S13. Handwritten text area extraction: Extract the handwritten text area of the corrected image according to the border coordinates of the standard image; S14. OCR recognition: Use the mainstream model to construct training data by semi - automatic simulation of constructing a training sample model to recognize text; S15. Text defect extraction: Construct a keyword dictionary tree and use a matching algorithm to extract text defect category information to form a label list; Step S2, electric meter image processing: S21. Key point extraction: Use the YOLO series of models to construct a sample set by simulating the construction of a training sample model, and extract the key point coordinates after training; S22. Perspective transformation: Correct the electric meter image into a standard image according to the key point coordinates; S23. Pixel comparison: Compare the image pixels within the pre - set border area of the standard image and the corrected image to judge the defect and obtain the image defect category information to form a label list; Step S3, defect label consistency comparison: Judge whether the information is consistent through the intersection of the label lists obtained from the work order and electric meter images, and realize the audit of the safety inspection work order information.
2. The method for automatically auditing electricity meter defect work orders based on machine vision according to claim 1, wherein: The specific steps of the key point extraction of the work order image include: Adopt the deep - learning image key point recognition model of the YOLO series, construct a training sample set by semi - automatic simulation of constructing a training sample method. First, prepare 10 - 100 standard work order image sets and set the key point coordinates, then prepare 10 - 100 common work order background image sets, and then randomly select standard work order images and background images. After performing random perspective transformation on the standard work order images, embed them into the background images to form simulated work order images, and at the same time perform the same perspective transformation on the corresponding key point coordinates as the regression label.
3. A method for automatically auditing electricity meter defect work orders based on machine vision according to claim 1, characterized in that: The specific steps of the perspective transformation of the work order image include: Use the key point coordinates obtained by the work order key point recognition model to correct the original work order image into a standard work order image through perspective transformation, so that it corresponds to the key point coordinates of the pre - determined standard work order image.
4. A method for automatically auditing electricity meter defect work orders based on machine vision according to claim 1, characterized in that: The specific steps of the extraction of the handwritten text area include: According to the characteristic that the handwritten text area of the corrected work order image is consistent with that of the standard work order image, extract the handwritten text area of the corrected work order image according to the border coordinates of the handwritten text area of the pre - set standard work order image.
5. A method for automatically auditing electricity meter defect work orders based on machine vision according to claim 1, characterized in that: The specific steps of the OCR text recognition include: Adopt the mainstream text recognition deep - learning model, construct training data by semi - automatic simulation of constructing a training sample method. First, sort out the character set involved in the handwritten content, construct 10 - 50 bitmap images of handwritten font styles for each character, prepare 10 - 100 document background sets under different illuminations, and then randomly generate simulated handwritten text area images and determine the corresponding label values.
6. The method for automatically auditing electricity meter defect work orders based on machine vision according to claim 1, characterized in that: The specific steps of the text defect category include: Sort out the keyword set representing different defect categories in advance to construct a dictionary tree, and adopt the keyword matching algorithm based on the dictionary tree to extract defect category information from the string recognized by OCR text.
7. A method for automatically auditing electricity meter defect work orders based on machine vision according to claim 1, characterized in that: The specific steps for extracting key points of the electricity meter image include: selecting a mainstream deep learning image key point recognition model, constructing a training sample set in a semi-automatic simulation method to construct training samples, preparing standard normal electricity meter diagrams and setting key point coordinates, preparing 10 - 100 common electricity meter background diagram sets, generating simulated electricity meter diagrams through random simulation, and at the same time performing perspective transformation on the corresponding key point coordinates as regression labels.
8. A method for automatically auditing electricity meter defect work orders based on machine vision according to claim 1, characterized in that: The specific steps for perspective transformation of the electricity meter image include: with the help of the key point coordinates obtained by the electricity meter key point recognition model, correcting the original electricity meter diagram into a standard electricity meter diagram through perspective transformation.
9. A method for automatically auditing electricity meter defect work orders based on machine vision according to claim 1, characterized in that: The specific steps for image pixel comparison include: previously setting the area border of the component to be concerned in the standard normal electricity meter image, and sequentially performing pixel comparison on the images within the same area border of the corrected electricity meter image and the standard normal electricity meter image, so as to judge whether there are defects in the component, and further obtain the image defect category information.
10. A device for automatically auditing electricity meter defect work orders based on machine vision, based on the method for automatically auditing electricity meter defect work orders based on machine vision described in claims 1-9, characterized in that , including: A key point extraction module, used to extract the key point coordinates of the work order image and the key point coordinates of the electricity meter image; A perspective transformation module, used to correct the work order diagram into a standard diagram according to the key point coordinates, and correct the electricity meter diagram into a standard diagram according to the key point coordinates; A text area extraction module, used to extract the handwritten text area of the corrected work order image for subsequent OCR recognition module, which is used to recognize the handwritten text area image through the mainstream model and convert it into a text string; A text defect extraction module, used to extract defect category information from the text string recognized by OCR through a matching algorithm and generate a work order defect label list; A pixel comparison module, used to compare the pixels of the images within the corresponding borders of the corrected electricity meter diagram and the standard diagram to judge defects and generate an electricity meter defect label list; A defect label consistency comparison module, used to compare whether the information is consistent by judging the elements through the intersection of the defect label lists of the work order and the electricity meter image, and review the safety inspection work order information.