Method and device for recognizing and sorting electric energy meter images

Abstract

The present invention discloses a method and device for recognizing and sorting electric energy meters. The method identifies the type and degree of defects of electric energy meters based on image recognition technology, reduces labor costs, and significantly improves efficiency. The method comprises the following steps: inputting the merged image into a trained YOLO model for detection, identifying the core area of ​​the sample image belonging to the electric energy meter, reading the coding and text parts in the merged image, and identifying the type of the current electric energy meter in combination with the size of the merged image; the sample image is resampled, normalized, and pre-processed, and segmented to remove the background, and then input into a trained first defect detection network to obtain the coordinates of the first defect detection area; the defect degree is determined based on whether the position coordinates of the first defect detection area coordinates intersect with the identified core area; and whether the position coordinates of the first defect detection area coordinates intersect with one of the four boundaries passing through the top view portion.

Description

Technical Field

[0001] The patent of this invention relates to the field of image recognition and sorting, and in particular to an image recognition technology that determines the meter type of different types of electricity meters, and sorts out single-phase meters and three-phase or terminal meters, and simultaneously performs preliminary sorting of electricity meter defects. Background Art

[0002] For meter-replacing production lines that require the operation of various types of electricity meters, determining the meter type is a very important step in producing electricity meters. Now more and more companies are using automated equipment to replace manual judgment, which can effectively improve recognition efficiency and reduce the probability of errors.

[0003] Currently, most companies rely on manual methods to determine meter types, which can easily lead to misjudgments and omissions. Therefore, it is crucial to design a reliable, convenient, and adaptive image recognition and sorting mechanism that can adapt to different types of meters and meet the needs of different manufacturers. This mechanism also requires a certain level of defect detection capability. Summary of the Invention

[0004] The present invention aims to overcome the above-mentioned deficiencies in the prior art and provides an electric energy meter image recognition and sorting method and device, which identifies the type and defect degree of the electric energy meter based on image recognition technology, reduces labor cost investment, and significantly improves efficiency.

[0005] To achieve the above object, the present invention adopts the following technical solutions:

[0006] The present invention provides an electric energy meter image recognition and sorting method, the method comprising the following steps:

[0007] Step 1: Use a camera to take high-definition photos of the electric energy meter from multiple perspectives, including at least a top view and four side views, and stitch these views into a combined image based on their spatial positions, and mark the four boundaries of the top view portion;

[0008] Step 2: Input the merged image into the trained YOLO model for detection, identify the core area of ​​the electric energy meter in the sample image, and mark the core area on the sample image.

[0009] Step 3: Read the code and text in the merged image, and identify the type of the current electric energy meter based on the size of the merged image. The electric energy meter types include single-phase electric energy meter, three-phase electric energy meter, and terminal electric energy meter.

[0010] Step 4: After the sample image is resampled, normalized, and segmented to remove the background, it is input into the trained first defect detection network to obtain the coordinates of the first defect detection area;

[0011] Determine the degree of the defect based on whether the position coordinates of the first defect detection area intersect with the core area identified in step 2; and whether the position coordinates of the first defect detection area intersect with one of the four boundaries passing through the top view portion in step 1;

[0012] If it is determined that the coordinates of the first defect detection area of ​​the currently identified electric energy meter intersect with the core area and pass through one of the four boundaries of the top view portion, the current electric energy meter is classified as a serious defect electric energy meter;

[0013] If it is determined that the coordinates of the first defect detection area of ​​the currently identified electric energy meter intersect with the core area but do not pass through one of the four boundaries of the top view portion, or if it is determined that the coordinates of the first defect detection area of ​​the currently identified electric energy meter do not intersect with the core area but pass through one of the four boundaries of the top view portion, the current electric energy meter is classified as a medium electric energy meter;

[0014] If it is determined that the coordinates of the first defect detection area of ​​the currently identified electric energy meter do not intersect with the core area and do not pass through one of the four boundaries of the top view portion, the current electric energy meter is classified as a pending defective electric energy meter;

[0015] Step 5: sort the electric energy meters according to their types and defect levels using a sorting device.

[0016] Preferably, the first defect detection network adopts a RetinaNet detection network.

[0017] Preferably, the training method of the first defect detection network comprises the following steps:

[0018] Step 201: Obtain at least one top view and four side views from a photo of a normal electric energy meter and a photo of a damaged electric energy meter, and combine the top view and the four side views into a single merged image based on their spatial positional relationships.

[0019] Step 202: Manually annotate the defective areas in the merged image of the damaged electric energy meter in detail to form bounding boxes with category labels; the defect categories include scratches, dents, cracks, and shell deformation, and construct a sample set, validation set, and test set for training the first defect detection network;

[0020] Step 203: input the pre-processed sample set into the first defect detection network, and train the first defect detection network using the manual annotations in step 202 as supervision information;

[0021] Step 204: Run the trained model on the test set to generate prediction results and evaluate the model performance using standard metrics.

[0022] Preferably, the training method of the YOLO model comprises the following steps:

[0023] Step 301, data acquisition: collecting a large number of high-resolution images including screens and other key information areas from normal and damaged electricity meters;

[0024] Step 302, image annotation: manually draw bounding boxes for each key area in each image and assign corresponding category labels;

[0025] Step 303: Using the image annotations as supervision information, train the YOLO model on an independent test set to generate prediction results and evaluate the model performance using standard metrics.

[0026] Step 304: Use the non-maximum suppression (NMS) algorithm to select the optimal bounding box and set a confidence threshold to filter low-quality predictions.

[0027] The present invention also provides an electric energy meter image recognition and sorting device, the device comprising:

[0028] An image synthesis module is used to take high-definition photos of the electric energy meter from multiple perspectives using a camera, including at least a top view and four side views, and stitch these views into a combined image based on their spatial positions, and mark the four boundaries of the top view portion;

[0029] a region detection module, configured to input the merged image into the trained first YOLO model for detection, identify the core area of ​​the electric energy meter in the sample image, the core area including the display screen, terminal blocks, programming cover, code, and text, and mark the core area on the sample image;

[0030] A meter type recognition module is used to read the code and text parts in the merged image and identify the type of the current electric energy meter based on the size of the merged image. The electric energy meter types include single-phase electric energy meter, three-phase electric energy meter and terminal electric energy meter;

[0031] The defect determination module inputs the sample image into the trained first defect detection network after resampling, normalization preprocessing, segmentation and background removal to obtain the coordinates of the first defect detection area;

[0032] Determine the degree of the defect based on whether the position coordinates of the first defect detection area intersect with the core area identified in step 2; and whether the position coordinates of the first defect detection area intersect with one of the four boundaries passing through the top view portion in step 1;

[0033] If it is determined that the coordinates of the first defect detection area of ​​the currently identified electric energy meter intersect with the core area and pass through one of the four boundaries of the top view portion, the current electric energy meter is classified as a serious defect electric energy meter;

[0034] If it is determined that the coordinates of the first defect detection area of ​​the currently identified electric energy meter intersect with the core area but do not pass through one of the four boundaries of the top view portion, or if it is determined that the coordinates of the first defect detection area of ​​the currently identified electric energy meter do not intersect with the core area but pass through one of the four boundaries of the top view portion, the current electric energy meter is classified as a medium electric energy meter;

[0035] If it is determined that the coordinates of the first defect detection area of ​​the currently identified electric energy meter do not intersect with the core area and do not pass through one of the four boundaries of the top view portion, the current electric energy meter is classified as a pending defective electric energy meter;

[0036] The sorting module is used to sort the electric energy meters according to the type and degree of defects of the electric energy meters through a sorting device.

[0037] An embodiment of the present application also provides an electronic device, including a memory and a processor; a computer program is stored on the memory; when the computer program is executed on the processor, the electric energy meter image recognition and sorting method is implemented.

[0038] And a computer-readable storage medium having computer instructions stored thereon; when the computer instructions are executed on the electronic device as claimed in claim 7, the electric energy meter image recognition and sorting method is implemented.

[0039] Therefore, the present invention has the following beneficial effects: (1) the type and defect degree of the electric energy meter are identified based on image recognition technology, which reduces the labor cost investment and significantly improves the efficiency; (2) in this scheme, the first defect detection network based on RetinaNet can accurately identify defects on small objects such as single-phase electric energy meters, and only needs to detect the regional coordinate position, without the need for overly high-precision determination of the defect type. The defect degree is mainly determined based on the position of the defect, which is more efficient and has lower computing power costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 The present invention is a flowchart of an electric energy meter image recognition and sorting method.

[0041] Figure 2 The figure is a principle block diagram of an electric energy meter image recognition and sorting device of the present invention. DETAILED DESCRIPTION

[0042] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0043] like Figure 1 As shown, the embodiment of the present application provides a method for identifying and sorting electric energy meters using images, the method comprising the following steps:

[0044] Step 1: Use a camera to take high-definition photos of the electric energy meter from multiple perspectives, including at least a top view and four side views, and stitch these views into a combined image based on their spatial positions, and mark the four boundaries of the top view portion;

[0045] Step 2: Input the merged image into the trained YOLO model for detection, identify the core area of ​​the electric energy meter in the sample image, and mark the core area on the sample image.

[0046] Step 3: Read the code and text in the merged image, and identify the type of the current electric energy meter based on the size of the merged image. The electric energy meter types include single-phase electric energy meter, three-phase electric energy meter, and terminal electric energy meter.

[0047] Step 4: After the sample image is resampled, normalized, and segmented to remove the background, it is input into the trained first defect detection network to obtain the coordinates of the first defect detection area;

[0048] Determine the degree of the defect based on whether the position coordinates of the first defect detection area intersect with the core area identified in step 2; and whether the position coordinates of the first defect detection area intersect with one of the four boundaries passing through the top view portion in step 1;

[0049] If it is determined that the coordinates of the first defect detection area of ​​the currently identified electric energy meter intersect with the core area and pass through one of the four boundaries of the top view portion, the current electric energy meter is classified as a serious defect electric energy meter;

[0050] If it is determined that the coordinates of the first defect detection area of ​​the currently identified electric energy meter intersect with the core area but do not pass through one of the four boundaries of the top view portion, or if it is determined that the coordinates of the first defect detection area of ​​the currently identified electric energy meter do not intersect with the core area but pass through one of the four boundaries of the top view portion, the current electric energy meter is classified as a medium electric energy meter;

[0051] If it is determined that the coordinates of the first defect detection area of ​​the currently identified electric energy meter do not intersect with the core area and do not pass through one of the four boundaries of the top view portion, the current electric energy meter is classified as a pending defective electric energy meter;

[0052] Step 5: sort the electric energy meters according to their types and defect levels using a sorting device.

[0053] Existing solutions for surface defect detection on disassembled surfaces primarily focus on overcoming the difficulty of identifying defect types. This approach is costly in terms of computing power and model accuracy, hindering widespread adoption. The core of this application focuses on sorting defects based on their location and impact, replacing manual initial screening and thereby improving the efficiency of the entire production line.

[0054] Defect determination takes two aspects into consideration: on the one hand, the impact of the defect location on the key parts of the electricity meter, such as the display screen, is taken into account. Generally, defects such as deformation and fragmentation will greatly affect the reuse of the electricity meter and are considered serious defects. On the other hand, it is also considered that the defects of the electricity meter damaged by external forces are mainly concentrated in the top view, and the depth of the damage will affect the four side views. Therefore, when the regional coordinates of the detected defect pass through the top view and the side view at the same time, it can be determined that the depth of the defect is large and can be reflected in the side view on the image. Therefore, the above two aspects are combined as the basis for defect determination. For example, if the regional coordinates of a crack are detected and intersect with the display screen and the right view at the same time, then this defect not only occurs on the display screen but also has a certain depth on the right view. This means that it is not a fine crack on the display screen, but a larger crack that extends to the right side of the electricity meter, which must be a more serious defect.

[0055] Specifically, the first defect detection network adopts the RetinaNet detection network. The specific training method includes the following steps:

[0056] Step 201: Obtain at least one top view and four side views from a photo of a normal electric energy meter and a photo of a damaged electric energy meter, and combine the top view and the four side views into a single merged image based on their spatial positional relationships.

[0057] Step 202: Manually annotate the defective areas in the merged image of the damaged electric energy meter in detail to form bounding boxes with category labels; the defect categories include scratches, dents, cracks, and shell deformation, and construct a sample set, validation set, and test set for training the first defect detection network;

[0058] Step 203: input the pre-processed sample set into the first defect detection network, and train the first defect detection network using the manual annotations in step 202 as supervision information;

[0059] Step 204: Run the trained model on the test set to generate prediction results and evaluate the model performance using standard metrics.

[0060] RetinaNet is a deep learning network for object detection proposed by Facebook AI Research (FAIR). It aims to address the difficulties faced by traditional one-stage detectors in dealing with class imbalance. It combines the Feature Pyramid Network (FPN) with the FocalLoss function to significantly improve small object detection and achieve a good balance between accuracy and speed.

[0061] In this solution, the first defect detection network based on RetinaNet can accurately identify defects on small objects such as single-phase electricity meters. It only needs to detect the regional coordinate position, without the need for overly high-precision determination of the defect type. The defect severity is mainly determined based on the defect location, which is more efficient and has lower computing power costs.

[0062] The training method of the YOLO model includes the following steps:

[0063] Step 301, data acquisition: collecting a large number of high-resolution images including screens and other key information areas from normal and damaged electricity meters;

[0064] Step 302, image annotation: manually draw bounding boxes for each key area in each image and assign corresponding category labels;

[0065] Step 303: Using the image annotations as supervision information, train the YOLO model on an independent test set to generate prediction results and evaluate the model performance using standard metrics.

[0066] Step 304: Use the non-maximum suppression (NMS) algorithm to select the optimal bounding box and set a confidence threshold to filter low-quality predictions.

[0067] YOLOYou Only Look Once is a pre-trained model for real-time object detection that offers an excellent balance between speed and accuracy, enabling real-time processing. The core idea of ​​YOLO is to treat the object detection problem as a single regression problem, predicting bounding boxes and category probabilities directly from image pixels.

[0068] like Figure 2 As shown, the present invention also provides an electric energy meter image recognition and sorting device, the device comprising:

[0069] Image synthesis module 1 is used to take high-definition photos of the electric energy meter from multiple perspectives using a camera, including at least a top view and four side views, and stitch these views into a combined image according to their spatial positions, and mark the four boundaries of the top view part;

[0070] Region detection module 2, configured to input the merged image into the trained first YOLO model for detection, identify the core area of ​​the electric energy meter in the sample image, the core area including the display screen, terminal blocks, programming cover, code, and text, and mark it on the sample image;

[0071] The meter type recognition module 3 is used to read the code and text parts in the merged image and identify the type of the current electric energy meter based on the size of the merged image. The electric energy meter types include single-phase electric energy meter, three-phase electric energy meter and terminal electric energy meter;

[0072] Defect determination module 4: after the sample image is resampled, normalized, and segmented to remove the background, it is input into the trained first defect detection network to obtain the coordinates of the first defect detection area;

[0073] Determine the degree of the defect based on whether the position coordinates of the first defect detection area intersect with the core area identified in step 2; and whether the position coordinates of the first defect detection area intersect with one of the four boundaries passing through the top view portion in step 1;

[0074] If it is determined that the coordinates of the first defect detection area of ​​the currently identified electric energy meter intersect with the core area and pass through one of the four boundaries of the top view portion, the current electric energy meter is classified as a serious defect electric energy meter;

[0075] If it is determined that the coordinates of the first defect detection area of ​​the currently identified electric energy meter intersect with the core area but do not pass through one of the four boundaries of the top view portion, or if it is determined that the coordinates of the first defect detection area of ​​the currently identified electric energy meter do not intersect with the core area but pass through one of the four boundaries of the top view portion, the current electric energy meter is classified as a medium electric energy meter;

[0076] If it is determined that the coordinates of the first defect detection area of ​​the currently identified electric energy meter do not intersect with the core area and do not pass through one of the four boundaries of the top view portion, the current electric energy meter is classified as a pending defective electric energy meter;

[0077] The sorting module 5 is used to sort the electric energy meters according to the type and degree of defects of the electric energy meters through a sorting device.

[0078] An embodiment of the present application also provides an electronic device, including a memory and a processor; a computer program is stored on the memory; when the computer program is executed on the processor, the above-mentioned method for recognizing and sorting electric energy meters is implemented.

[0079] In addition, a computer-readable storage medium is provided, on which computer instructions are stored; when the computer instructions are executed on the electronic device as described in claim 7, the above-mentioned method for recognizing and sorting electric energy meters is implemented.

Claims

1. A method for identifying and sorting electric energy meters, characterized in that: The method comprises the following steps: Step 1: Use a camera to take high-definition photos of the electric energy meter from multiple perspectives, including at least a top view and four side views, and stitch these views into a combined image based on their spatial positions, and mark the four boundaries of the top view portion; Step 2: Input the merged image into the trained YOLO model for detection, identify the core area of ​​the electric energy meter in the merged image, and mark the core area on the merged image. Step 3: Read the code and text in the merged image, and identify the type of the current electric energy meter based on the size of the merged image. The electric energy meter types include single-phase electric energy meter, three-phase electric energy meter, and terminal electric energy meter. Step 4: After the merged image is resampled, normalized, and segmented to remove the background, it is input into the trained first defect detection network to obtain the coordinates of the first defect detection area; Determine the degree of the defect based on whether the position coordinates of the first defect detection area intersect with the core area identified in step 2; and whether the position coordinates of the first defect detection area intersect with one of the four boundaries passing through the top view portion in step 1; If it is determined that the coordinates of the first defect detection area of ​​the currently identified electric energy meter intersect with the core area and pass through one of the four boundaries of the top view portion, the current electric energy meter is classified as a serious defect electric energy meter; If it is determined that the coordinates of the first defect detection area of ​​the currently identified electric energy meter intersect with the core area but do not pass through one of the four boundaries of the top view portion, or if it is determined that the coordinates of the first defect detection area of ​​the currently identified electric energy meter do not intersect with the core area but pass through one of the four boundaries of the top view portion, the current electric energy meter is classified as a medium electric energy meter; If it is determined that the coordinates of the first defect detection area of ​​the currently identified electric energy meter do not intersect with the core area and do not pass through one of the four boundaries of the top view portion, the current electric energy meter is classified as a pending defective electric energy meter; Step 5: sort the electric energy meters according to their types and defect levels using a sorting device.

2. The method for image recognition and sorting of electric energy meters according to claim 1, characterized in that: The first defect detection network adopts the RetinaNet detection network.

3. The method for image recognition and sorting of electric energy meters according to claim 2, wherein: The training method of the first defect detection network comprises the following steps: Step 201: Obtain at least one top view and four side views from a photo of a normal electric energy meter and a photo of a damaged electric energy meter, and stitch the top view and four side views into a combined image based on their spatial positional relationship. Step 202: Manually annotate the defective areas in the merged image of the damaged electric energy meter in detail to form bounding boxes with category labels; the defect categories include scratches, dents, cracks, and shell deformation, and construct a sample set, validation set, and test set for training the first defect detection network; Step 203: input the pre-processed sample set into the first defect detection network, and train the first defect detection network using the manual annotations in step 202 as supervision information; Step 204: Run the trained model on the test set to generate prediction results and evaluate the model performance using standard metrics.

4. The method for image recognition and sorting of electric energy meters according to claim 1, wherein: The training method of the YOLO model includes the following steps: Step 301, data acquisition: collecting a large number of high-resolution images including screens and other key information areas from normal and damaged electricity meters; Step 302, image annotation: manually draw bounding boxes for each key area in each image and assign corresponding category labels; Step 303: Using the image annotations as supervision information, train the YOLO model on an independent test set to generate prediction results and evaluate the model performance using standard metrics. In step 304 , a non-maximum suppression (NMS) algorithm is used to select the optimal bounding box, and a confidence threshold is set to filter out low-quality predictions.

5. An electric energy meter image recognition and sorting device, characterized in that: The device comprises: An image synthesis module (1) is used to take high-definition photos of the electric energy meter from multiple viewing angles using a camera, including at least a top view and side views in four directions, and to stitch these views together into a combined image according to spatial positions, and to mark the four boundaries of the top view portion; An area detection module (2) is used to input the merged image into the trained first YOLO model for detection, identify the core area of ​​the electric energy meter in the merged image, the core area including the display screen, the terminal block, the programming cover, the code and the text part, and mark it on the merged image; A meter type recognition module (3) is used to read the code and text parts in the merged image, and identify the type of the current electric energy meter in combination with the size of the merged image. The electric energy meter types include single-phase electric energy meters, three-phase electric energy meters and terminal electric energy meters; Defect determination module (4), the merged image is subjected to resampling, normalization preprocessing, segmentation and background removal, and then input into the trained first defect detection network to obtain the coordinates of the first defect detection area; Determine the degree of the defect based on whether the position coordinates of the first defect detection area intersect with the core area identified in step 2; and whether the position coordinates of the first defect detection area intersect with one of the four boundaries passing through the top view portion in step 1; If it is determined that the coordinates of the first defect detection area of ​​the currently identified electric energy meter intersect with the core area and pass through one of the four boundaries of the top view portion, the current electric energy meter is classified as a serious defect electric energy meter; If it is determined that the coordinates of the first defect detection area of ​​the currently identified electric energy meter intersect with the core area but do not pass through one of the four boundaries of the top view portion, or if it is determined that the coordinates of the first defect detection area of ​​the currently identified electric energy meter do not intersect with the core area but pass through one of the four boundaries of the top view portion, the current electric energy meter is classified as a medium electric energy meter; If it is determined that the coordinates of the first defect detection area of ​​the currently identified electric energy meter do not intersect with the core area and do not pass through one of the four boundaries of the top view portion, the current electric energy meter is classified as a pending defective electric energy meter; The sorting module (5) is used to sort the electric energy meters according to the type and degree of defects of the electric energy meters through a sorting device.

6. An electronic device, characterized in that: It comprises a memory and a processor; a computer program is stored in the memory; when the computer program is executed on the processor, the electric energy meter image recognition and sorting method according to any one of claims 1 to 4 is implemented.

7. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions; when the computer instructions are executed on the electronic device according to claim 6, the electric energy meter image recognition and sorting method according to any one of claims 1 to 4 is implemented.

Images