Automatic sorting method for returned electric energy meters based on assembly line

Abstract

The invention discloses an automatic sorting method for returned electric energy meters based on an assembly line, and belongs to the technical field of automatic sorting of the returned electric energy meters. The method comprises the steps that a basic feature library is established, and the basic feature library comprises product images, attribute features and damage types of the returned electric energy meters; determining representative features in the attribute features, determining representative features corresponding to each damage type and a feature value range, collecting a first image of the returned electric energy meter, and preprocessing the first image to obtain a second image; extracting actual appearance features of the returned electric energy meter based on the identification model, and extracting actual representative features from the actual appearance features and actual electrical features obtained through detection; and the classification model divides the returned electric energy meters into corresponding damage types based on the actual representative features, and sorts the returned electric energy meters to corresponding recovery lines based on the damage types. According to the invention, the automatic sorting efficiency and effect of the returned electric energy meters can be greatly improved.

Description

Technical Field

[0001] The invention belongs to the technical field of automatic sorting of returned electric energy meters, and in particular relates to an assembly line-based automatic sorting method for returned electric energy meters. Background Art

[0002] Returned energy meters refer to energy meters that are recovered from users due to failure, replacement, or expiration of their use cycle. With the widespread use of energy meters, the number of returned energy meters has increased year by year, and their status usually includes various types, such as repairable, scrapped, and upgrade requirements. These energy meters need to be sorted before they can enter the repair, reuse, or scrapping process. However, the traditional manual sorting method is inefficient and prone to classification errors due to human factors, which is not conducive to large-scale processing needs.

[0003] In recent years, the application of image recognition technology in the appearance recognition of returned electricity meters has gradually emerged. Image recognition algorithms based on deep learning can quickly identify the appearance features of electricity meters, such as shell integrity, display panel status, model identification, etc. For example, a Chinese patent document with publication number CN104700425A discloses an outdoor electricity meter damage degree detection platform. This method uses a high-definition camera to shoot the electricity meter to obtain an electricity meter image, and performs image processing on the electricity meter image to extract the meter frame image in the electricity meter image. The reference electricity meter frame pattern is pre-stored in the memory. Based on the matching result between the reference electricity meter frame pattern and the meter frame image, the degree of damage of the electricity meter is determined; for example A Chinese patent document with publication number CN114202523A discloses a device, method, equipment and medium for detecting the appearance quality of a dismantled electric energy meter. The method uses a camera to collect an image of the dismantled electric energy meter and pre-processes the image of the electric energy meter. Then, the pre-processed image of the electric energy meter is expanded to obtain a denoised electric energy meter image. The denoised electric energy meter image is compared with a standard electric energy meter image to obtain the number of pixels in the denoised electric energy meter image that are inconsistent with the standard electric energy meter image. Whether the appearance of the dismantled electric energy meter is qualified is determined based on the number of pixels, and the appearance quality detection result of the dismantled electric energy meter is output.

[0004] Pixel color judgment technology usually relies on manually set thresholds or simple rule algorithms, and lacks intelligent learning and optimization capabilities. When the appearance of the electric energy meter changes or a new type of electric energy meter appears, it is necessary to manually adjust the parameters, resulting in poor system adaptability and high maintenance costs. Therefore, this method gradually reveals its limitations under the current requirements of efficient and automated sorting, and needs to be improved and upgraded through more advanced image recognition technology. Summary of the invention

[0005] In order to solve the above problems, the present invention provides an automated sorting method for returned electric energy meters based on an assembly line to solve the problems in the prior art.

[0006] In order to achieve the above-mentioned purpose of the invention, the present invention proposes an automated sorting method for returned electric energy meters based on an assembly line, comprising:

[0007] Establishing a basic feature library, the basic feature library includes product images of returned electric energy meters and multiple categories of attribute features, the attribute features include appearance features and electrical features, and each of the product images is marked with a damage type of the returned electric energy meter;

[0008] Establishing a recognition model based on the basic feature library, determining representative features in the attribute features, determining representative features and feature value ranges corresponding to each damage type in combination with the basic feature library and the representative features, and establishing a classification model based on the feature value ranges;

[0009] Capturing a first image of a newly-entered or returned electric energy meter at a plurality of preset angles, and preprocessing the first image to obtain a second image;

[0010] Segmenting the second image into a plurality of recognition areas based on the recognition model, extracting actual appearance features of the returned electric energy meter based on the recognition areas, and extracting actual representative features from the actual appearance features and the actual electrical features obtained by detection;

[0011] The classification model classifies the returned electric energy meters into the corresponding damage types based on the actual representative features, and sorts the returned electric energy meters to the corresponding recycling lines based on the damage types.

[0012] Furthermore, determining the representative feature in the attribute features comprises the following steps:

[0013] A dimensionality reduction matrix is ​​constructed based on the PCA dimensionality reduction algorithm and the attribute features, and the attribute features are converted into representative features based on the dimensionality reduction matrix.

[0014] Further, determining the characteristic value range representing the characteristic comprises the following steps:

[0015] Dividing the product images in the basic feature library into a training group and a verification group, locating the maximum and minimum values ​​of the feature values ​​corresponding to the representative features of the same category in the training group, and setting the numerical range between the maximum and minimum values ​​as the candidate range of the representative features;

[0016] Discretize the candidate range into a predetermined number of range intervals, count the occurrence frequencies of the product images of different damage types in the training group within each range interval, and calculate the first association probability of each range interval corresponding to each damage type based on the occurrence frequencies, wherein the first association probability with the largest value is defined as the second association probability;

[0017] Based on the second association probability, the damage type of the product images in the verification group is classified, and the classification accuracy is calculated. If the classification accuracy is less than the second threshold, the predetermined number of intervals for dividing the range is adjusted and the second association probability is recalculated. This step is repeated until the classification accuracy is greater than or equal to the second threshold. In each representative feature, the range interval where the maximum second association probability appears is determined to be the feature value range corresponding to the damage type.

[0018] Further, classifying the damage type of the product image based on the second association probability comprises the following steps:

[0019] Obtain feature values ​​of the representative features of the product image in the verification group, determine the candidate range in which the feature values ​​are located, determine the damage type corresponding to each representative feature according to the second association probability of the candidate range, and use the damage type with the largest number of occurrences as the damage type of the product image based on a voting method.

[0020] Furthermore, establishing the classification model based on the feature value range includes the following steps:

[0021] The classification model is a random forest model. The feature value range representing the feature is used as the split feature of the decision tree of the classification model. The model parameters in the classification model are set. The basic feature library is divided into multiple subsets through the self-help method. A decision tree is constructed based on the subsets. During the construction process, information gain, information gain ratio and Gini coefficient are used as constraints, and the classification model is evaluated through a confusion matrix.

[0022] Furthermore, the recognition model extracts the actual appearance features of the returned electric energy meter, including the following steps:

[0023] The recognition model includes a first model and a plurality of second models, the first model divides the second image into a plurality of recognition areas, each of the second models has a convolutional layer and a fully connected layer, and the fully connected layer of each of the second models has a first output neuron and a plurality of second output neurons, wherein the output result of the first output neuron represents that the returned electric energy meter does not have an appearance feature representing damage, and the output result of the second output neuron represents an appearance feature related to damage of the returned electric energy meter;

[0024] Each of the identification areas is identified using the corresponding second model, and identification results generated by each of the second models are obtained, and the identification results are fused to generate actual appearance features for the identification area.

[0025] Furthermore, fusing the recognition results comprises the following steps:

[0026] If the recognition results of the multiple second models are not mutually exclusive, all the recognition results are output as actual appearance features. If the recognition results are mutually exclusive, the smaller number of mutually exclusive recognition results are deleted according to the number of each recognition result, and the remaining recognition results are output as actual appearance features.

[0027] Furthermore, dividing the second image into a plurality of the identification areas comprises the following steps:

[0028] Extract pixel values ​​of all pixels in the second image, construct a color histogram based on the pixel values, determine the number of clusters based on the color histogram, cluster the pixel values ​​based on the number of clusters and the clustering algorithm and obtain clustering results, assign a color cluster label to each pixel based on the clustering results, group pixels with the same color cluster label into one identification area, smooth the boundaries of the identification area based on a morphological algorithm, and apply a region growing algorithm to connect the identification areas of the same type.

[0029] Further, the recovery line includes mild, moderate and severe.

[0030] Furthermore, the preprocessing includes denoising and contrast enhancement.

[0031] Compared with the prior art, the beneficial effects of the present invention are at least as follows:

[0032] The present invention constructs a comprehensive basic feature library through the images of electric energy meters collected from multiple angles. Then, the appearance features and electrical features of the electric energy meter are efficiently analyzed to achieve multimodal feature fusion; before building the classification model, the basic feature library is optimized by extracting representative features and determining the relationship between representative features and feature values ​​and various damage types, so that the training speed is improved and memory consumption is reduced when the classification model is subsequently built. In the detection link of new electric energy meters entering the warehouse, the recognition area is accurately divided through image segmentation and other technologies, and then each recognition area is individually recognized, which further improves the accuracy of image recognition.

[0033] Compared with the traditional classification method that relies only on pixels or single features, the present invention not only covers various types of damage to the electric energy meter, such as shell damage, terminal damage, current transformer damage, display screen damage, etc., but also distinguishes the degree of damage, greatly improving the granularity of classification. At the same time, by introducing data-driven statistical analysis and machine learning models, the present invention is much better than manual detection or traditional algorithm classification methods in classification efficiency, accuracy and automation. By efficiently allocating returned electric energy meters to slightly, moderately or severely damaged lines, it provides a clear process allocation for subsequent maintenance, disassembly and resource recovery, which helps to achieve efficient utilization of electric energy meter resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 The present invention is a flowchart of the steps of the automated sorting method for returned electric energy meters based on an assembly line. DETAILED DESCRIPTION

[0035] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0036] It is understood that the terms "first", "second", etc. used in this application may be used herein to describe various elements, but unless otherwise specified, these elements are not limited by these terms. These terms are only used to distinguish a first element from another element. For example, without departing from the scope of this application, a first xx script may be referred to as a second xx script, and similarly, a second xx script may be referred to as a first xx script.

[0037] like Figure 1 As shown, a method for automatically sorting returned electric energy meters based on an assembly line comprises:

[0038] S1: Establish a basic feature library, which includes product images of returned electricity meters and multiple categories of attribute features. The attribute features include appearance features and electrical features. Each product image is marked with the damage type of the returned electricity meter.

[0039] Specifically, the basic feature library includes product images of various angles of returned electric energy meters collected in advance and the corresponding damage types. The product images include images from the front, back, side and top perspectives. The damage types include shell damage, terminal damage, current transformer damage, and display screen damage. The shell damage includes mild damage, moderate damage, and severe damage, and the outline of the damaged area is also marked in the product image. In particular, the present invention also adds images of undamaged de-energized electric energy meters to the basic database; different damage types correspond to different performance characteristics, namely attribute characteristics, such as attribute characteristics such as the length, width, crack length, etc. of the shell. When the length and width are small, or cracks appear, it can be judged that the shell is damaged. Electrical characteristics include high or low measurement errors, and the displayed power or power consumption does not match the actual power consumption. Finally, the product image and its corresponding appearance features, electrical features, and damage type labels are stored in the database to form a basic feature library.

[0040] S2: Establish a recognition model based on the basic feature library, determine the representative features in the attribute features, combine the basic feature library and the representative features to determine the representative features and feature value range corresponding to each damage type, and establish a classification model based on the feature value range.

[0041] First, the images in the basic feature library are uniformly resized and normalized to ensure the consistency of model training. Then, data enhancement techniques such as flipping, scaling and rotating are used to expand the data set to improve the generalization ability of the subsequent recognition model. The recognition model of this embodiment includes an improved CNN convolutional neural network model, that is, a convolutional neural model is trained based on product images in the basic feature library. For representative features, statistical analysis can be used to obtain representative features with actual physical significance, or machine learning methods can be used to obtain abstract features representing overall feature information. After that, comprehensive statistical analysis is performed to determine which representative features are associated with the damage type and the corresponding eigenvalue range, such as severe damage to the shell corresponding to crack length, eigenvalue range of 2-10mm, burn area, eigenvalue range of 5-15 cm². Finally, a classification model based on the random forest algorithm is established based on the representative features and eigenvalue range. In other embodiments, a classification model can also be established based on the SVM algorithm.

[0042] S3: Capture a first image of a newly-entered or returned electric energy meter at a plurality of preset angles, and pre-process the first image to obtain a second image.

[0043] Among them, preprocessing includes denoising and contrast enhancement.

[0044] S4: Segment the second image into a plurality of recognition areas based on the recognition model, extract actual appearance features of the returned electric energy meter based on the recognition areas, and extract actual representative features from the actual appearance features and the actual electrical features obtained by detection.

[0045] The preset angle is consistent with the acquisition direction of the basic feature library image to ensure the consistency and accuracy of the classification model. The first image is processed using Gaussian filtering and histogram equalization to remove noise interference and increase contrast. In addition, Canny edge detection can be introduced to extract key features of the appearance, and segmentation is performed based on the extracted features to obtain multiple identification areas, such as terminal areas, display areas, and shell areas. This embodiment uses a histogram to segment image areas, which will be described in detail later. Each identification area is then identified separately to obtain actual appearance features, such as cracks, burns and stains, including cracks. For actual electrical characteristics, standard test equipment is used to detect electrical parameters such as voltage, current, and power factor of the electric energy meter.

[0046] Finally, according to the representative features determined previously, the actual representative features are screened out from the actual appearance features and electrical features, or the actual appearance features and electrical features are compressed and converted to obtain the actual representative features.

[0047] S5: The classification model classifies the returned electric energy meters into corresponding damage types based on the actual representative features, and sorts the returned electric energy meters to the corresponding recycling lines based on the damage types.

[0048] In this embodiment, the recovery lines include mild, moderate and severe.

[0049] Use the trained classification model to classify the actual representative features and determine the damage type of the returned electricity meter. According to the classification results, the returned electricity meters are sorted to the corresponding recycling lines. Each recycling line corresponds to one or more types of damage. For example, the slightly damaged line: the electricity meter with small cracks and slight stains, the moderately damaged line: the electricity meter with a large burn area but repairable, and the severely damaged line: the electricity meter that cannot be repaired, enters the disassembly or recycling stage. Through the above-mentioned implementation mode, the present invention can efficiently and accurately complete the classification and sorting of returned electricity meters. For example, when an image of the front of the electricity meter is input, the length of the extracted crack is 8mm, which is located at the edge of the display screen. The classification model outputs the damage type as "shell crack", and the sorting is terminated at the "slightly damaged line".

[0050] The present invention constructs a comprehensive basic feature library through the images of electric energy meters collected from multiple angles, and performs normalization and data enhancement processing on them, which significantly improves the generalization ability of the model. Then, based on the improved convolutional neural network (CNN) and random forest classification model, the appearance features and electrical features of the electric energy meter are efficiently analyzed to achieve multimodal feature fusion; before building the classification model, the basic feature library is optimized by extracting representative features and determining the relationship between representative features and feature values ​​and each damage type, so that when the classification model is subsequently built, the training speed is improved and the memory consumption is reduced. In the detection link of new electric energy meters entering the warehouse, the recognition area is accurately divided through Canny edge detection and histogram segmentation, and then each recognition area is individually recognized, which further improves the accuracy of image recognition.

[0051] Compared with the traditional classification method that only relies on pixels or single features, the present invention not only covers various types of damage to the electric energy meter, such as shell damage, terminal damage, current transformer damage, display screen damage, etc., but also can distinguish the degree of damage (such as moderate and severe), greatly improving the granularity of classification. At the same time, by introducing data-driven statistical analysis and machine learning models. It is significantly superior to manual detection or traditional algorithm classification methods in terms of classification efficiency, accuracy and automation. By efficiently allocating returned electric energy meters to slightly, moderately or severely damaged lines, it provides a clear process allocation for subsequent maintenance, disassembly and resource recovery, which helps to achieve efficient use of electric energy meter resources.

[0052] It is particularly noteworthy that the present invention can greatly improve the efficiency and effect of automatic sorting of returned electric energy meters.

[0053] In this embodiment, determining the representative feature from the attribute features includes the following steps:

[0054] A dimensionality reduction matrix is ​​constructed based on the PCA dimensionality reduction algorithm and attribute features, and the attribute features are converted into representative features based on the dimensionality reduction matrix.

[0055] After obtaining the attribute features, the attribute features of each product image and the corresponding damage type are converted into an attribute matrix, and the attribute matrix is ​​standardized so that the mean of each feature is 0 and the standard deviation is 1. Then, the covariance matrix of the standardized feature data is calculated, and the covariance matrix is ​​subjected to eigendecomposition to obtain the eigenvalues ​​and the corresponding eigenvectors. The size of the eigenvalue reflects the contribution rate of each principal component to the overall variance of the original data. The eigenvalues ​​are arranged in descending order, and the cumulative contribution rate of each principal component is calculated. The present invention sets the first threshold to 95%, that is, the eigenvectors corresponding to the principal components with a cumulative contribution rate greater than 95% are selected as the column vectors of the dimensionality reduction matrix. For example, the first 5 principal components are selected, which represents that the original data is reduced to 5 dimensions. Then, the attribute matrix is ​​multiplied by the dimensionality reduction matrix to obtain a representative feature matrix. Each eigenvalue in the representative feature matrix represents the comprehensive value of the original attribute features and has no specific physical meaning. After receiving the new returned electric energy meter data, the returned electric energy meter is multiplied by the dimensionality reduction matrix previously calculated based on the basic feature library to obtain the actual representative feature. In particular, the inverse transformation method can also be used to restore the principal components, so as to determine which attribute features have a greater impact on the judgment result of the damage type, so as to screen out representative features. The specific details are prior art and will not be described in detail here.

[0056] In this embodiment, determining the feature value range of the representative feature includes the following steps:

[0057] The product images in the basic feature library are divided into a training group and a verification group, the maximum and minimum feature values ​​corresponding to the representative features of the same category in the training group are located, and the numerical range between the maximum and minimum values ​​is set as the candidate range of the representative features.

[0058] The candidate range is discretized into a predetermined number of range intervals, and the occurrence frequency of product images with different damage types in the training group within each range interval is counted. The first association probability corresponding to each damage type in each range interval is calculated based on the occurrence frequency, where the first association probability with the largest value is defined as the second association probability.

[0059] Specifically, 70% of the product images are divided into the training group and 30% of the images are divided into the validation group. For each representative feature of each product image in the training group, the representative feature can be an actual feature, such as scratch length, or an abstract feature obtained after dimensionality reduction; if there are 100 product images, the maximum value of the representative feature A is 10 and the minimum value is 1, then the value range of 1-10 is set as the candidate range of the representative feature A.

[0060] Then, the candidate range is evenly divided into multiple range intervals according to a predetermined number (such as 10 intervals). For example, if the candidate range [1, 10] is divided into 10 intervals, the width of each interval is 0.9, and the division results are [1, 1.9], [1.9, 2.8], …, [9.1, 10]. After the segmentation is completed, each range interval is assigned a unique identification number for subsequent statistics and analysis, such as 1, 2, and 3. For each range interval, the number of images of different damage types in the interval is counted and recorded as the frequency of occurrence. For example, in a certain interval [1, 1.9], there are 100 images of crack type, 50 images of scratch type, and 30 images of deformation type, then their frequencies of occurrence are 100, 50, and 30 respectively.

[0061] Then, the first association probability between the kth representative feature and the nth damage type in each range interval is calculated by the first formula, and the first formula is specifically: , where P is the first association probability, M is the total number of product images in the training set, is the frequency of occurrence of the nth damage type in k ranges. For example, there are 100 product images in the training group. For the first damage type, there are 30 corresponding product images, and the value of their representative feature A is in the range [1,1.9], so its first association probability is 30 / 100=0.3. For each range of the representative feature A, the first association probability with the largest value is determined as the second association probability. For example, for the representative feature A, in the range [1.9, 2.8], the first association probability of damage type 1 is the largest, which is 80%. That is to say, when the value of the representative feature A of the product image is in [1.9, 2.8], there is an 80% probability of damage type A.

[0062] Based on the second association probability classification, the damage type of the product images in the verification group is verified, and the classification accuracy is calculated. If the classification accuracy is less than the second threshold, the predetermined number of divided range intervals is adjusted and the second association probability is recalculated. This step is repeated until the classification accuracy is greater than or equal to the second threshold. In each representative feature, the range interval where the maximum second association probability appears is determined to be the feature value range corresponding to the damage type.

[0063] Specifically, classifying the damage type of a product image based on association probability includes the following steps:

[0064] The feature values ​​of the representative features of the product images in the verification group are obtained, and the candidate range in which the feature values ​​are located is determined. The damage type corresponding to each representative feature is determined according to the second association probability of the candidate range. The damage type with the most occurrences is used as the damage type of the product image based on the voting method.

[0065] After obtaining the second association probability for each representative feature, the product images in the validation set are classified. Specifically, the classification is performed by the following method: assuming that for product image 1, there are representative features A, B, and C, where representative feature A is in candidate range 1, and the corresponding second association probability is 30%, corresponding to damage type 1; representative feature B is in candidate range 3, and the corresponding second association probability is 50%, corresponding to damage type 2; representative feature C is in candidate range 10, and the corresponding second association probability is 60%, corresponding to damage type 2. Since damage type 2 appears twice, it is set as the damage type of the product image.

[0066] Finally, the classification accuracy of the product images of the verification group is calculated. If the accuracy is less than the second threshold, such as 50%, the predetermined number is adjusted to re-discretize the range interval. The predetermined number can be adjusted from 10 to 15 to increase the refinement of the interval, and then the above method is repeated to recalculate the second association probability until the classification accuracy of the verification group reaches or exceeds the second threshold. Finally, the feature value range of each representative feature can be obtained. For example, if the representative feature A has the highest second association probability in range interval 1, which corresponds to damage type 1, then range interval 1 of the representative feature is set as the feature value range of damage type 1.

[0067] In this embodiment, establishing a classification model based on the feature value range includes the following steps:

[0068] The classification model is a random forest model. The range of eigenvalues ​​representing the features is used as the splitting feature of the decision tree of the classification model. The model parameters in the classification model are set. The basic feature library is divided into multiple subsets through the self-service method. The decision tree is constructed based on the subsets. During the construction process, information gain, information gain ratio and Gini coefficient are used as constraints, and the classification model is evaluated through the confusion matrix.

[0069] After obtaining the representative features and feature value ranges, they are annotated in the product image as training features to replace the original attribute features. For example, product image 1 shows that the shell is slightly damaged, and the attribute feature is a crack length of 1 cm. After replacement, the crack length becomes 0.8cm-1.1cm, or 0.02-0.05 (abstract features after PCA dimensionality reduction). The replacement can improve the training speed and accuracy of the random forest model. When establishing a random forest model, the model parameters of the random forest model are first specified. The model parameters include the number of decision trees, the maximum depth of the decision tree, the minimum number of sample splits, and the minimum number of leaf node samples. When constructing each decision tree, multiple subsets are generated through sampling with replacement, that is, the bootstrap method, and each subset contains part of the data in the basic database.

[0070] When constructing a decision tree, we choose to split the features with the largest reduction in information gain entropy, the largest information gain ratio, and the smallest Gini coefficient to ensure the highest purity of nodes in each layer of the decision tree. Finally, the classification effect of the classification model is evaluated by the precision rate, recall rate, and other evaluation indicators. The evaluation indicators are obtained through the confusion matrix. The specific method is the existing technology and will not be introduced here.

[0071] In this embodiment, the recognition model extracts the actual appearance features of the returned electric energy meter, including the following steps:

[0072] The recognition model includes a first model and multiple second models, the first model divides the second image into multiple recognition areas, each second model has a convolutional layer and a fully connected layer, and the fully connected layer of each second model has a first output neuron and multiple second output neurons, wherein the output result of the first output neuron represents that the returned electric energy meter does not have the appearance features representing damage, and the output result of the second output neuron represents the appearance features related to the damage of the returned electric energy meter.

[0073] Each identification area is identified using the corresponding second model, and the identification result generated by each second model is obtained, and the identification results are fused to generate actual appearance features for the identification area.

[0074] The fusion of recognition results includes the following steps:

[0075] If the recognition results of multiple second models are not mutually exclusive, all recognition results are output as actual appearance features. If the recognition results are mutually exclusive, the smaller number of mutually exclusive recognition results are deleted according to the size of each recognition result, and the remaining recognition results are output as actual appearance features.

[0076] The recognition model in this embodiment is a cascade model composed of multiple different categories of models. After the newly stored returned electric energy meter arrives at the production line, images of it at different angles are taken. The first model uses technologies such as Canny edge detection and histogram segmentation to divide the recognition area, such as the recognition area such as the display screen area, the mounting hole area, the sealing cover area, etc., and then uses the second model to identify these areas separately. The first model can be trained using all product images in the basic feature library. The second model is introduced below. Each second model has different recognition functions, such as the second model 1 is used to identify cracks and oil stains, and the second model 2 is used to identify wear and oil stains. The fully connected layer of the second model 1 has multiple output neurons. For example, when the first neuron is activated to output the result, it represents that there is no damage feature in the image, and when the second neuron is activated to output the result, it represents that there is a crack in the image. When the first neuron of the second model 2 is activated to output the result, it represents that there is no damage feature in the image, and when the second neuron is activated to output the result, it represents that there is oil stain in the image. Recognition is performed by different second models, which improves the robustness of the recognition result.

[0077] In particular, if the second model 1 outputs that the product image is not damaged, and the second model 2 outputs that the product image has oil stains, it indicates that the two are mutually exclusive. There are currently 4 second models, and the other two second models both output recognition results that the product image is not damaged, indicating that the recognition results representing oil stains are less than the recognition results representing no damage. They are deleted, and finally the recognition result that the returned electricity meter is not damaged is output.

[0078] In this embodiment, dividing the second image into a plurality of identification areas comprises the following steps:

[0079] The pixel values ​​of all pixels in the second image are extracted, a color histogram is constructed based on the pixel values, the number of clusters is determined based on the color histogram, the pixel values ​​are clustered based on the number of clusters and the clustering algorithm and the clustering results are obtained, a color cluster label is assigned to each pixel based on the clustering results, pixels with the same color cluster label are grouped into an identification area, the boundaries of the identification areas are smoothed based on the morphological algorithm, and the identification areas of the same type are connected using a region growing algorithm.

[0080] Extract the values ​​of each pixel in the second image in three channels (R, G, B), count the frequency of occurrence of color combinations in each channel, and draw a histogram based on the frequency of occurrence to determine the main range of color occurrence, thereby determining the number of clusters. The number of clusters can be further determined by determining the number of peaks based on the histogram. Then use the K-means clustering algorithm to cluster the pixels based on the pixel value as the input feature, and note the color cluster for each pixel based on the clustering result, for example, the main color clusters are white, gray, and black; in the second image, divide the pixels with the same color cluster into the same recognition area, so as to achieve the preliminary segmentation of the recognition area, such as dividing the image into the display area and the black plastic shell area. Then, in order to optimize the boundary of the recognition area, apply morphological operations (such as expansion and corrosion) to smooth the clustering results, and finally connect the recognition areas of the same type through the regional growing algorithm to ensure the integrity of the segmented area. Specifically, the Flood Fill algorithm can be used, and the growth conditions can be set based on the differences in the color histogram.

[0081] It should be understood that the various technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the various technical features in the above-mentioned embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0082] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. An automated sorting method for returned electric energy meters based on an assembly line, characterized in that: include: Establishing a basic feature library, the basic feature library includes product images of returned electric energy meters and multiple categories of attribute features, the attribute features include appearance features and electrical features, and each of the product images is marked with a damage type of the returned electric energy meter; Establishing a recognition model based on the basic feature library, determining representative features in the attribute features, determining representative features and feature value ranges corresponding to each damage type in combination with the basic feature library and the representative features, and establishing a classification model based on the feature value ranges; Capturing a first image of a newly-entered or returned electric energy meter at a plurality of preset angles, and preprocessing the first image to obtain a second image; Segmenting the second image into a plurality of recognition areas based on the recognition model, extracting actual appearance features of the returned electric energy meter based on the recognition areas, and extracting actual representative features from the actual appearance features and the actual electrical features obtained by detection; The classification model classifies the returned electric energy meters into the corresponding damage types based on the actual representative features, and sorts the returned electric energy meters to the corresponding recycling lines based on the damage types.

2. The method according to claim 1, characterized in that: Determining a representative feature from the attribute features comprises the following steps: A dimensionality reduction matrix is ​​constructed based on the PCA dimensionality reduction algorithm and the attribute features, and the attribute features are converted into representative features based on the dimensionality reduction matrix.

3. The method according to claim 1, characterized in that Determining the range of eigenvalues ​​that represent a feature includes the following steps: Dividing the product images in the basic feature library into a training group and a verification group, locating the maximum and minimum values ​​of the feature values ​​corresponding to the representative features of the same category in the training group, and setting the numerical range between the maximum and minimum values ​​as the candidate range of the representative features; Discretize the candidate range into a predetermined number of range intervals, count the occurrence frequencies of the product images of different damage types in the training group within each range interval, and calculate the first association probability of each range interval corresponding to each damage type based on the occurrence frequencies, wherein the first association probability with the largest value is defined as the second association probability; Based on the second association probability, the damage type of the product images in the verification group is classified, and the classification accuracy is calculated. If the classification accuracy is less than the second threshold, the predetermined number of intervals for dividing the range is adjusted and the second association probability is recalculated. This step is repeated until the classification accuracy is greater than or equal to the second threshold. In each representative feature, the range interval where the maximum second association probability appears is determined to be the feature value range corresponding to the damage type.

4. The method according to claim 3, characterized in that Classifying the damage type of the product image based on the second association probability comprises the following steps: Obtain feature values ​​of the representative features of the product image in the verification group, determine the candidate range in which the feature values ​​are located, determine the damage type corresponding to each representative feature according to the second association probability of the candidate range, and use the damage type with the largest number of occurrences as the damage type of the product image based on a voting method.

5. The method according to claim 1 or 3, characterized in that: Establishing the classification model based on the feature value range includes the following steps: The classification model is a random forest model. The feature value range representing the feature is used as the split feature of the decision tree of the classification model. The model parameters in the classification model are set. The basic feature library is divided into multiple subsets through the self-help method. A decision tree is constructed based on the subsets. During the construction process, information gain, information gain ratio and Gini coefficient are used as constraints, and the classification model is evaluated through a confusion matrix.

6. The method according to claim 1, characterized in that The recognition model extracts the actual appearance features of the returned electric energy meter and comprises the following steps: The recognition model includes a first model and a plurality of second models, the first model divides the second image into a plurality of recognition areas, each of the second models has a convolutional layer and a fully connected layer, and the fully connected layer of each of the second models has a first output neuron and a plurality of second output neurons, wherein the output result of the first output neuron represents that the returned electric energy meter does not have an appearance feature representing damage, and the output result of the second output neuron represents an appearance feature related to damage of the returned electric energy meter; Each of the identification areas is identified using the corresponding second model, and identification results generated by each of the second models are obtained, and the identification results are fused to generate actual appearance features for the identification area.

7. The method according to claim 6, characterized in that The fusion of the recognition results comprises the following steps: If the recognition results of the multiple second models are not mutually exclusive, all the recognition results are output as actual appearance features. If the recognition results are mutually exclusive, the smaller number of mutually exclusive recognition results are deleted according to the number of each recognition result, and the remaining recognition results are output as actual appearance features.

8. The method according to claim 6, characterized in that Segmenting the second image into a plurality of the identification areas comprises the following steps: Extract pixel values ​​of all pixels in the second image, construct a color histogram based on the pixel values, determine the number of clusters based on the color histogram, cluster the pixel values ​​based on the number of clusters and the clustering algorithm and obtain clustering results, assign a color cluster label to each pixel based on the clustering results, group pixels with the same color cluster label into one identification area, smooth the boundaries of the identification area based on a morphological algorithm, and apply a region growing algorithm to connect the identification areas of the same type.

9. The method according to claim 3, characterized in that: The recovery lines include mild, moderate and severe.

10. The method according to claim 1, characterized in that The preprocessing includes denoising and contrast enhancement.

Images