Electric energy meter image recognition and sorting method and device

Abstract

The invention discloses an electric energy meter image recognition and sorting method and device, and aims at recognizing the type and defect degree of an electric energy meter based on an image recognition technology, reducing the manpower cost investment and remarkably improving the efficiency. The method comprises the following steps: inputting a combined picture into a trained YOLO model for detection, identifying a core region belonging to the electric energy meter in a sample picture, reading a code and a character part in the combined picture, combining the size of the combined picture, and identifying the type of the current electric energy meter, carrying out resampling and normalization preprocessing on the sample picture, segmenting and removing a background, and inputting the sample picture into a trained first defect detection network to obtain a first defect detection area coordinate; judging whether the position coordinates of the first defect detection area coordinates are intersected with the identified core area or not; and judging whether the position coordinates of the first defect detection area coordinates intersect with one of the four boundaries passing through the top view part or not.

Description

Technical Field

[0001] The patent of this invention relates to the field of image recognition and sorting, and in particular to an image recognition technology that determines the meter type of different types of electric energy meters, and sorts out single-phase meters and three-phase or terminal meters, and simultaneously performs preliminary sorting of electric energy meter defects. Background Art

[0002] For dismantling meter production lines that require operation of various types of electricity meters, meter type judgment is a very important step in the production of electricity meters. Now more and more companies are using automated equipment to replace manual judgment, which can effectively improve recognition efficiency and reduce the probability of errors.

[0003] At present, most companies use manual methods to judge the type of electric energy meters, which is prone to misjudgment and missed judgment. Therefore, it is very necessary to design a new reliable, convenient and adaptive image recognition and sorting mechanism to adapt to different types of electric energy meters to meet the sorting needs of different types of meters from various manufacturers. At the same time, it is also necessary to have a certain defect judgment function. Summary of the invention

[0004] The present invention aims to overcome the above-mentioned deficiencies in the prior art and to provide an electric energy meter image recognition and sorting method and device, which can identify the type and defect degree of the electric energy meter based on image recognition technology, reduce labor cost investment and significantly improve efficiency.

[0005] To achieve the above object, the present invention adopts the following technical solutions:

[0006] The present invention provides an electric energy meter image recognition and sorting method, the method comprising the following steps:

[0007] Step 1: take high-definition photos of the electric energy meter from multiple viewing angles, including at least a top view and side views in four directions, and stitch these views into a combined picture according to spatial positions, and mark the four boundaries of the top view part;

[0008] Step 2: input the merged image into the trained YOLO model for detection, identify the core area of ​​the electric energy meter in the sample image, and mark the core area on the sample image.

[0009] Step 3, reading the code and text parts in the merged image, and identifying the type of the current electric energy meter in combination with the size of the merged image, wherein the electric energy meter types include single-phase electric energy meter, three-phase electric energy meter and terminal electric energy meter;

[0010] Step 4, after the sample image is resampled, normalized and preprocessed, and the background is removed by segmentation, the sample image is input into the trained first defect detection network to obtain the coordinates of the first defect detection area;

[0011] Determine the degree of defect according to whether the position coordinates of the first defect detection area coordinates intersect with the core area identified in step 2; and whether the position coordinates of the first defect detection area coordinates intersect with one of the four boundaries passing through the top view portion in step 1;

[0012] Determining that the coordinates of the first defect detection area of ​​the currently identified electric energy meter intersect with the core area and pass through one of the four boundaries of the top view portion at the same time, the current electric energy meter is classified as a serious defect electric energy meter;

[0013] It is determined that the coordinates of the first defect detection area of ​​the currently identified electric energy meter intersect with the core area at the same time, but do not pass through one of the four boundaries of the top view part, or it is determined that the coordinates of the first defect detection area of ​​the currently identified electric energy meter do not intersect with the core area at the same time, but pass through one of the four boundaries of the top view part, then the current electric energy meter is classified as a medium electric energy meter;

[0014] Determining that the coordinates of the first defect detection area of ​​the currently identified electric energy meter do not intersect with the core area and do not cross one of the four boundaries of the top view portion, the current electric energy meter is classified as a pending defective electric energy meter;

[0015] Step 5: sorting the electric energy meters according to the type and degree of defects through a sorting device.

[0016] Preferably, the first defect detection network adopts a RetinaNet detection network.

[0017] Preferably, the training method of the first defect detection network comprises the following steps:

[0018] Step 201, obtaining at least one top view and four side views from the photos of the appearance of a normal electric energy meter and the photos of the appearance of a damaged electric energy meter, and combining the top view and the four side views into a combined picture according to the spatial position relationship;

[0019] Step 202, manually annotating the defective areas in the merged image of the damaged electric energy meter in detail to form a bounding box with a category label; the defect categories include scratches, dents, cracks, and shell deformation, and constructing a sample set, a verification set, and a test set for training a first defect detection network;

[0020] Step 203, inputting the sample set into the first defect detection network after preprocessing, and training the first defect detection network using the manual annotation in step 202 as supervision information;

[0021] Step 204, run the trained model on the test set, generate prediction results, and evaluate the model performance using standard metrics.

[0022] Preferably, the training method of the YOLO model comprises the following steps:

[0023] Step 301, data acquisition: collecting a large number of high-resolution images including screens and other key information areas from normal and damaged electric energy meters;

[0024] Step 302, image annotation: manually draw bounding boxes for each key area in each image and assign corresponding category labels;

[0025] Step 303, using the image annotation as supervision information, training the YOLO model on an independent test set, generating prediction results, and evaluating the model performance using standard indicators;

[0026] Step 304: Use the non-maximum suppression (NMS) algorithm to select the optimal bounding box and set a confidence threshold to filter low-quality predictions.

[0027] The present invention also provides an electric energy meter image recognition and sorting device, the device comprising:

[0028] The picture synthesis module is used to take high-definition photos of the electric energy meter from multiple viewing angles through a camera, including at least a top view and side views in four directions, and stitch these views into a combined picture according to spatial positions, and mark the four boundaries of the top view part;

[0029] A region detection module is used to input the merged image into the trained first YOLO model for detection, identify the core area of ​​the electric energy meter in the sample image, the core area includes the display screen, the terminal block, the programming cover, the code and the text part, and mark it on the sample image;

[0030] A meter type recognition module, used to read the code and text parts in the merged image, and identify the type of the current electric energy meter in combination with the size of the merged image. The electric energy meter types include single-phase electric energy meter, three-phase electric energy meter and terminal electric energy meter;

[0031] Defect judgment module, the sample image is resampled, normalized and preprocessed, and segmented to remove the background, and then input into the trained first defect detection network to obtain the coordinates of the first defect detection area;

[0032] Determine the degree of defect according to whether the position coordinates of the first defect detection area coordinates intersect with the core area identified in step 2; and whether the position coordinates of the first defect detection area coordinates intersect with one of the four boundaries passing through the top view portion in step 1;

[0033] Determining that the coordinates of the first defect detection area of ​​the currently identified electric energy meter intersect with the core area and pass through one of the four boundaries of the top view portion at the same time, the current electric energy meter is classified as a serious defect electric energy meter;

[0034] It is determined that the coordinates of the first defect detection area of ​​the currently identified electric energy meter intersect with the core area at the same time, but do not pass through one of the four boundaries of the top view part, or it is determined that the coordinates of the first defect detection area of ​​the currently identified electric energy meter do not intersect with the core area at the same time, but pass through one of the four boundaries of the top view part, then the current electric energy meter is classified as a medium electric energy meter;

[0035] Determining that the coordinates of the first defect detection area of ​​the currently identified electric energy meter do not intersect with the core area and do not cross one of the four boundaries of the top view portion, the current electric energy meter is classified as a pending defective electric energy meter;

[0036] The sorting module is used to sort the electric energy meters according to the type and degree of defects of the electric energy meters through a sorting device.

[0037] An embodiment of the present application also provides an electronic device, including a memory and a processor; a computer program is stored in the memory; when the computer program is executed on the processor, the electric energy meter image recognition and sorting method is implemented.

[0038] And a computer-readable storage medium, on which computer instructions are stored; when the computer instructions are executed on the electronic device as claimed in claim 7, the electric energy meter image recognition and sorting method is implemented.

[0039] Therefore, the present invention has the following beneficial effects: (1) the type and defect degree of the electric energy meter are identified based on image recognition technology, which reduces the manpower cost investment and significantly improves the efficiency; (2) in this scheme, the first defect detection network based on RetinaNet can accurately identify defects on small objects such as single-phase electric energy meters, and only needs to detect the regional coordinate position, without the need for overly high-precision determination of the defect type. The defect degree is mainly determined based on the location of the defect, which is more efficient and has lower computing power costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 The present invention is a flow chart of an electric energy meter image recognition and sorting method.

[0041] Figure 2 The present invention is a block diagram of the principle of an electric energy meter image recognition and sorting device. DETAILED DESCRIPTION

[0042] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments.

[0043] like Figure 1 As shown, the embodiment of the present application provides a method for identifying and sorting electric energy meters, the method comprising the following steps:

[0044] Step 1: take high-definition photos of the electric energy meter from multiple viewing angles, including at least a top view and side views in four directions, and stitch these views into a combined picture according to spatial positions, and mark the four boundaries of the top view part;

[0045] Step 2: input the merged image into the trained YOLO model for detection, identify the core area of ​​the electric energy meter in the sample image, and mark the core area on the sample image.

[0046] Step 3, reading the code and text parts in the merged image, and identifying the type of the current electric energy meter in combination with the size of the merged image, wherein the electric energy meter types include single-phase electric energy meter, three-phase electric energy meter and terminal electric energy meter;

[0047] Step 4, after the sample image is resampled, normalized and preprocessed, and the background is removed by segmentation, the sample image is input into the trained first defect detection network to obtain the coordinates of the first defect detection area;

[0048] Determine the degree of defect according to whether the position coordinates of the first defect detection area coordinates intersect with the core area identified in step 2; and whether the position coordinates of the first defect detection area coordinates intersect with one of the four boundaries passing through the top view portion in step 1;

[0049] Determining that the coordinates of the first defect detection area of ​​the currently identified electric energy meter intersect with the core area and pass through one of the four boundaries of the top view portion at the same time, the current electric energy meter is classified as a serious defect electric energy meter;

[0050] It is determined that the coordinates of the first defect detection area of ​​the currently identified electric energy meter intersect with the core area at the same time, but do not pass through one of the four boundaries of the top view part, or it is determined that the coordinates of the first defect detection area of ​​the currently identified electric energy meter do not intersect with the core area at the same time, but pass through one of the four boundaries of the top view part, then the current electric energy meter is classified as a medium electric energy meter;

[0051] Determining that the coordinates of the first defect detection area of ​​the currently identified electric energy meter do not intersect with the core area and do not cross one of the four boundaries of the top view portion, the current electric energy meter is classified as a pending defective electric energy meter;

[0052] Step 5: sorting the electric energy meters according to the type and degree of defects through a sorting device.

[0053] The existing technical solutions for dismantling surface defect detection mainly focus on overcoming the identification of defect types. This solution is not easy to popularize due to the high cost of computing power and model accuracy. The core of this application focuses on sorting according to the location and impact of defects, replacing manual first preliminary selection to improve the efficiency of the entire production line.

[0054] Defect determination takes two aspects into consideration: on the one hand, the influence of the defect position on the key parts of the electric energy meter, such as the display screen, is taken into consideration. Generally, defects such as deformation and fragmentation will greatly affect the reuse of the electric energy meter and are considered serious defects. On the second aspect, it is also considered that the defects of the electric energy meter damaged by external forces are mainly concentrated in the top view, and the depth of the damage will affect the four side views. Therefore, when the regional coordinates of the defect are detected to pass through the top view and the side view at the same time, it can be determined that the depth of the defect is large and can be reflected on the side view on the image. Therefore, the above two aspects are combined as the basis for defect determination. For example, if the regional coordinates of a crack are detected to intersect with the display screen and the right view at the same time, then this defect not only occurs on the display screen but also can reflect a certain depth on the right view, indicating that it is not a fine crack on the display screen, but already has a larger crack and extends to the right side of the electric energy meter, which must be a more serious defect.

[0055] Specifically, the first defect detection network adopts the RetinaNet detection network. The specific training method includes the following steps:

[0056] Step 201, obtaining at least one top view and four side views from the photos of the appearance of a normal electric energy meter and the photos of the appearance of a damaged electric energy meter, and combining the top view and the four side views into a combined picture according to the spatial position relationship;

[0057] Step 202, manually annotating the defective areas in the merged image of the damaged electric energy meter in detail to form a bounding box with a category label; the defect categories include scratches, dents, cracks, and shell deformation, and constructing a sample set, a verification set, and a test set for training a first defect detection network;

[0058] Step 203, inputting the sample set into the first defect detection network after preprocessing, and training the first defect detection network using the manual annotation in step 202 as supervision information;

[0059] Step 204, run the trained model on the test set, generate prediction results, and evaluate the model performance using standard metrics.

[0060] RetinaNet is a deep learning network for object detection proposed by Facebook AI Research (FAIR) to solve the difficulties encountered by traditional one-stage detectors in dealing with class imbalance problems. It combines the Feature Pyramid Network (FPN) and the FocalLoss function to significantly improve the effect of small object detection and achieve a good balance between accuracy and speed.

[0061] In this solution, the first defect detection network based on RetinaNet can accurately identify defects on small objects such as single-phase electricity meters, and only needs to detect the regional coordinate position, without the need for overly high-precision determination of the defect type. The degree of the defect is mainly determined based on the location of the defect, which is more efficient and has lower computing power costs.

[0062] The training method of the YOLO model includes the following steps:

[0063] Step 301, data acquisition: collecting a large number of high-resolution images including screens and other key information areas from normal and damaged electric energy meters;

[0064] Step 302, image annotation: manually draw bounding boxes for each key area in each image and assign corresponding category labels;

[0065] Step 303, using the image annotation as supervision information, training the YOLO model on an independent test set, generating prediction results, and evaluating the model performance using standard indicators;

[0066] Step 304: Use the non-maximum suppression (NMS) algorithm to select the optimal bounding box and set a confidence threshold to filter low-quality predictions.

[0067] YOLOYou Only Look Once is a pre-trained model for real-time object detection that strikes an excellent balance between speed and accuracy and enables real-time processing. The core idea of ​​YOLO is to treat the object detection problem as a single regression problem, predicting bounding boxes and category probabilities directly from image pixels.

[0068] like Figure 2 As shown, the present invention also provides an electric energy meter image recognition and sorting device, the device comprising:

[0069] The picture synthesis module 1 is used to take high-definition photos of the electric energy meter from multiple viewing angles through a camera, including at least a top view and side views in four directions, and splice these views into a combined picture according to spatial positions, and mark the four boundaries of the top view part;

[0070] The region detection module 2 is used to input the merged image into the first trained YOLO model for detection, identify the core area of ​​the electric energy meter in the sample image, and mark the core area on the sample image.

[0071] The meter type recognition module 3 is used to read the code and text parts in the merged image, and identify the type of the current electric energy meter in combination with the size of the merged image. The electric energy meter types include single-phase electric energy meter, three-phase electric energy meter and terminal electric energy meter;

[0072] Defect judgment module 4, the sample image is resampled, normalized and preprocessed, and the background is removed by segmentation, and then input into the trained first defect detection network to obtain the coordinates of the first defect detection area;

[0073] Determine the degree of defect according to whether the position coordinates of the first defect detection area coordinates intersect with the core area identified in step 2; and whether the position coordinates of the first defect detection area coordinates intersect with one of the four boundaries passing through the top view portion in step 1;

[0074] Determining that the coordinates of the first defect detection area of ​​the currently identified electric energy meter intersect with the core area and pass through one of the four boundaries of the top view portion at the same time, the current electric energy meter is classified as a serious defect electric energy meter;

[0075] It is determined that the coordinates of the first defect detection area of ​​the currently identified electric energy meter intersect with the core area at the same time, but do not pass through one of the four boundaries of the top view part, or it is determined that the coordinates of the first defect detection area of ​​the currently identified electric energy meter do not intersect with the core area at the same time, but pass through one of the four boundaries of the top view part, then the current electric energy meter is classified as a medium electric energy meter;

[0076] Determining that the coordinates of the first defect detection area of ​​the currently identified electric energy meter do not intersect with the core area and do not cross one of the four boundaries of the top view portion, the current electric energy meter is classified as a pending defective electric energy meter;

[0077] The sorting module 5 is used to sort the electric energy meters according to the type and degree of defects of the electric energy meters through a sorting device.

[0078] An embodiment of the present application also provides an electronic device, including a memory and a processor; a computer program is stored on the memory; when the computer program is executed on the processor, the electric energy meter image recognition and sorting method described above is implemented.

[0079] In addition, a computer-readable storage medium is provided, on which computer instructions are stored; when the computer instructions are executed on the electronic device as described in claim 7, the above-mentioned method for recognizing and sorting electric energy meter images is implemented.

Claims

1. A method for identifying and sorting electric energy meters, characterized in that: The method comprises the following steps: Step 1: take high-definition photos of the electric energy meter from multiple viewing angles, including at least a top view and side views in four directions, and stitch these views into a combined picture according to spatial positions, and mark the four boundaries of the top view part; Step 2: input the merged image into the trained YOLO model for detection, identify the core area of ​​the electric energy meter in the sample image, and mark the core area on the sample image. Step 3, reading the code and text parts in the merged image, and identifying the type of the current electric energy meter in combination with the size of the merged image, wherein the electric energy meter types include single-phase electric energy meter, three-phase electric energy meter and terminal electric energy meter; Step 4, after the sample image is resampled, normalized and preprocessed, and the background is removed by segmentation, the sample image is input into the trained first defect detection network to obtain the coordinates of the first defect detection area; Determine the degree of defect according to whether the position coordinates of the first defect detection area coordinates intersect with the core area identified in step 2; and whether the position coordinates of the first defect detection area coordinates intersect with one of the four boundaries passing through the top view portion in step 1; Determining that the coordinates of the first defect detection area of ​​the currently identified electric energy meter intersect with the core area and pass through one of the four boundaries of the top view portion at the same time, the current electric energy meter is classified as a serious defect electric energy meter; It is determined that the coordinates of the first defect detection area of ​​the currently identified electric energy meter intersect with the core area at the same time, but do not pass through one of the four boundaries of the top view part, or it is determined that the coordinates of the first defect detection area of ​​the currently identified electric energy meter do not intersect with the core area at the same time, but pass through one of the four boundaries of the top view part, then the current electric energy meter is classified as a medium electric energy meter; Determining that the coordinates of the first defect detection area of ​​the currently identified electric energy meter do not intersect with the core area and do not cross one of the four boundaries of the top view portion, the current electric energy meter is classified as a pending defective electric energy meter; Step 5: sorting the electric energy meters according to the type and degree of defects through a sorting device.

2. The method for identifying and sorting electric energy meters according to claim 1, characterized in that: The first defect detection network adopts the RetinaNet detection network.

3. The method for identifying and sorting electric energy meters according to claim 2, characterized in that: The training method of the first defect detection network comprises the following steps: Step 201, obtaining at least one top view and four side views from the photos of the appearance of a normal electric energy meter and the photos of the appearance of a damaged electric energy meter, and combining the top view and the four side views into a combined picture according to the spatial position relationship; Step 202, manually annotating the defective areas in the merged image of the damaged electric energy meter in detail to form a bounding box with a category label; the defect categories include scratches, dents, cracks, and shell deformation, and constructing a sample set, a verification set, and a test set for training a first defect detection network; Step 203, inputting the sample set into the first defect detection network after preprocessing, and training the first defect detection network using the manual annotation in step 202 as supervision information; Step 204, run the trained model on the test set, generate prediction results, and evaluate the model performance using standard metrics.

4. The method for identifying and sorting electric energy meters according to claim 1, characterized in that: The training method of the YOLO model includes the following steps: Step 301, data acquisition: collecting a large number of high-resolution images including screens and other key information areas from normal and damaged electric energy meters; Step 302, image annotation: manually draw bounding boxes for each key area in each image and assign corresponding category labels; Step 303, using the image annotation as supervision information, training the YOLO model on an independent test set, generating prediction results, and evaluating the model performance using standard indicators; Step 304 , using a non-maximum suppression (NMS) algorithm to select the optimal bounding box, and setting a confidence threshold to filter out low-quality predictions.

5. An electric energy meter image recognition and sorting device, characterized in that: The device comprises: The image synthesis module (1) is used to take high-definition photos of the electric energy meter from multiple viewing angles through a camera, including at least a top view and side views in four directions, and stitch these views into a combined image according to spatial positions, and mark the four boundaries of the top view portion; A region detection module (2) is used to input the merged image into the trained first YOLO model for detection, identify the core area of ​​the electric energy meter in the sample image, the core area includes the display screen, the wiring terminal, the programming cover, the code and the text part, and mark it on the sample image; A meter type identification module (3), used to read the code and text parts in the merged image, and identify the type of the current electric energy meter in combination with the size of the merged image, wherein the electric energy meter types include single-phase electric energy meters, three-phase electric energy meters and terminal electric energy meters; Defect determination module (4), the sample image is subjected to resampling, normalization preprocessing, segmentation and background removal, and then input into a trained first defect detection network to obtain the coordinates of a first defect detection area; Determine the degree of defect according to whether the position coordinates of the first defect detection area coordinates intersect with the core area identified in step 2; and whether the position coordinates of the first defect detection area coordinates intersect with one of the four boundaries passing through the top view portion in step 1; Determining that the coordinates of the first defect detection area of ​​the currently identified electric energy meter intersect with the core area and pass through one of the four boundaries of the top view portion at the same time, the current electric energy meter is classified as a serious defect electric energy meter; It is determined that the coordinates of the first defect detection area of ​​the currently identified electric energy meter intersect with the core area at the same time, but do not pass through one of the four boundaries of the top view part, or it is determined that the coordinates of the first defect detection area of ​​the currently identified electric energy meter do not intersect with the core area at the same time, but pass through one of the four boundaries of the top view part, then the current electric energy meter is classified as a medium electric energy meter; Determining that the coordinates of the first defect detection area of ​​the currently identified electric energy meter do not intersect with the core area and do not cross one of the four boundaries of the top view portion, the current electric energy meter is classified as a pending defective electric energy meter; The sorting module (5) is used to sort the electric energy meters according to the type and degree of defects of the electric energy meters through a sorting device.

6. An electronic device, characterized in that: It comprises a memory and a processor; the memory stores a computer program; when the computer program is executed on the processor, the electric energy meter image recognition and sorting method as described in any one of claims 1 to 4 is implemented.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions; when the computer instructions are executed on the electronic device as described in claim 7, the electric energy meter image recognition and sorting method as described in any one of claims 1 to 4 is implemented.

Images