A method and system for detecting appearance defects of electric energy meters based on neural networks

Abstract

The present invention discloses a method and system for detecting appearance defects of electric energy meters based on a neural network. The method for detecting appearance defects of electric energy meters according to the present invention includes: acquiring original image data of the appearance of an electric energy meter, and performing preprocessing and enhancement processing on the original image data to obtain enhanced image data; constructing a multi-layer convolutional neural network as a feature extractor to extract visual features at different levels in the enhanced image data, and then performing feature fusion processing to obtain fused feature data, and analyzing the fused feature data to realize the classification and positioning of appearance defects of the electric energy meter, so as to achieve the purpose of detecting appearance defects of the electric energy meter. The present invention can effectively extract tiny and complex appearance defects, enabling subsequent detection to more accurately identify subtle defects; the present invention fuses visual features at different levels, improving the detection accuracy and generalization ability for complex appearance defects.

Description

Technical Field

[0001] The present invention relates to the technical field of appearance detection, and in particular to a method and system for detecting appearance defects of electric energy meters based on a neural network. Background Art

[0002] In the industrial production process, the appearance quality of electric energy meters has an important impact on their market competitiveness and user satisfaction. Therefore, the detection of appearance defects of electric energy meters has become a key link in quality control. Traditional detection of appearance defects of electric energy meters usually relies on manual inspection. However, there are many deficiencies in manual detection: First, the efficiency of manual detection is low and it is easily affected by the experience and state of the inspectors. It is not only difficult to meet the needs of large-scale production, but also the stability and consistency of the detection results are poor. In addition, it is often difficult for manual detection to identify minute or complex defects (such as fine scratches, unevenness, surface stains, etc.), and missed detections and misdetections often occur. Especially during long-term work, the fatigue effect of manual detection further exacerbates the risk of misjudgment.

[0003] In summary, there are at least the following technical problems: The processing of appearance data during the detection of the appearance of electric energy meters is not accurate enough and the detection accuracy is relatively low. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to overcome the defects existing in the above-mentioned prior art, and provide a method and system for detecting appearance defects of electric energy meters based on a neural network, so as to effectively extract minute and complex appearance defects, and enable subsequent detections to more accurately identify minute defects.

[0005] To this end, the present invention adopts the following technical solutions.

[0006] In the first aspect, the present invention provides a method for detecting appearance defects of electric energy meters based on a neural network, which includes the steps of:

[0007] S1. Obtain the original image data of the appearance of the electric energy meter, and perform preprocessing and enhancement processing on the original image data to obtain the enhanced image data;

[0008] S2. Construct a multi-layer convolutional neural network as a feature extractor, extract visual features at different levels from the enhanced image data, then perform feature fusion processing to obtain the fused feature data, and analyze the fused feature data to realize the classification and positioning of the appearance defects of the electric energy meter, so as to achieve the purpose of detecting the appearance defects of the electric energy meter.

[0009] Further, the S1 specifically includes: comprehensively capturing the original image data of the appearance of the electricity meter using an industrial camera for detecting defect details, performing preprocessing on the original image data including denoising, background elimination, and normalization, and enhancing the preprocessed image data using a multi-scale recursive perturbation and reverse correction image enhancement algorithm.

[0010] Furthermore, in the S1, the multi-scale recursive perturbation and reverse correction image enhancement algorithm incorporates multi-scale non-linear transformation, recursive enhancement, multi-dimensional random perturbation fusion, and reverse reconstruction.

[0011] Furthermore, in the implementation process of the multi-scale recursive perturbation and reverse correction image enhancement algorithm, to separate different frequency components in the appearance image so that high-frequency details and low-frequency structures can be processed independently, multi-scale decomposition is performed on the preprocessed image data; to amplify potential detail defects and suppress background noise, a dual non-linear transformation is introduced based on the multi-scale decomposition, and non-linear transformation is performed on the image at each scale; recursive enhancement processing is performed on the multi-scale images after non-linear transformation, and by repeatedly extracting and enhancing the effective features in the image, noise interference is eliminated.

[0012] Furthermore, in the implementation process of the multi-scale recursive perturbation and reverse correction image enhancement algorithm, the recursively enhanced image may still contain background noise, and texture-structure decomposition is introduced to decompose the recursively enhanced image into a texture part and a structure part.

[0013] Furthermore, in the implementation process of the multi-scale recursive perturbation and reverse correction image enhancement algorithm, after obtaining the recursively enhanced and decomposed image, multi-dimensional random perturbation processing is introduced to simulate image changes under different noise environments and improve robustness; to fuse the perturbation information at different scales, the perturbed images at all scales are weighted and fused to obtain the final fused image.

[0014] Furthermore, in the implementation process of the multi-scale recursive perturbation and reverse correction image enhancement algorithm, to avoid introducing pseudo-defects and noise while the fused image contains rich detail information, a reverse perturbation correction mechanism is introduced to perform a reverse operation on the fused image and the perturbation matrix to eliminate the pseudo-defects caused by excessive perturbation; after obtaining the image through reverse perturbation correction, to restore the global structure and detail information of the image, multi-scale reconstruction is performed on the image after reverse perturbation correction.

[0015] Further, in the S2, the visual features at different levels are subjected to feature fusion processing using an adaptive interactive feature fusion algorithm. The adaptive interactive feature fusion algorithm performs interactive mapping, adaptive weighting, and feature fusion processing on the visual features at different levels, and finally generates an efficient feature representation to improve the accuracy and robustness of the appearance defect detection of the electricity meter.

[0016] Furthermore, in the adaptive interactive feature fusion algorithm, to achieve hierarchical feature transformation, visual features at different levels are transformed into feature tensors that can express richer semantic information, and linear transformation and non-linear activation operations are performed on the visual features at each level; after the transformed feature tensors at each level are formed, multiple interactive mapping processing is performed on the transformed feature tensors. The purpose of multiple interactive mapping is to perform two-way interaction on feature tensors at different levels and different resolutions, so that each feature tensor can fuse the semantic information of features at other levels; after obtaining the interactive mapping matrix, according to the information in the interactive mapping matrix, different weights are assigned to the feature tensors at each level, thereby adaptively fusing multi-level features.

[0017] In a second aspect, the present invention provides an electric energy meter appearance defect detection system based on a neural network for implementing the above-mentioned electric energy meter appearance defect detection method, which includes:

[0018] An image data enhancement processing unit: acquires the original image data of the electric energy meter appearance, and performs preprocessing and enhancement processing on the original image data to obtain the enhanced image data;

[0019] A defect recognition unit: constructs a multi-layer convolutional neural network as a feature extractor, extracts visual features at different levels from the enhanced image data, then performs feature fusion processing to obtain the fused feature data, and analyzes the fused feature data to realize the classification and localization of the electric energy meter appearance defects, achieving the purpose of electric energy meter appearance defect detection.

[0020] The beneficial effects of the present invention are:

[0021] 1. The present invention can effectively extract tiny and complex appearance defects, enabling subsequent detection to more accurately identify subtle defects;

[0022] 2. The present invention fuses visual features at different levels to generate an efficient feature representation, enabling each feature to contain richer context information, and improving the detection accuracy and generalization ability for complex appearance defects. Description of the Drawings

[0023] Figure 1 is a flowchart of an electric energy meter appearance defect detection method based on a neural network according to the present invention;

[0024] Figure 2 is an architecture diagram of an electric energy meter appearance defect detection system based on a neural network according to the present invention. Detailed Embodiments

[0025] To further illustrate the technical means and effects adopted by the present invention to achieve the intended invention purpose, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative work shall fall within the protection scope of the present invention.

[0026] Unless otherwise defined, all technical and scientific terms used in the present invention have the same meaning as commonly understood by those skilled in the technical field to which the present invention belongs.

[0027] The following specifically describes the specific technical solution of an electric energy meter appearance defect detection based on a neural network provided by the present invention in conjunction with the accompanying drawings.

[0028] Refer to the attached Figure 1 , which shows a flowchart of a method for detecting electric energy meter appearance defects based on a neural network according to the present invention. The method includes the following steps:

[0029] S1. Obtain the original image data of the electric energy meter appearance, and perform preprocessing and enhancement processing on the original image data to obtain the enhanced image data.

[0030] Professional technicians select an industrial camera suitable for detecting defect details to comprehensively capture the original image data of the electric energy meter appearance, and perform preprocessing such as denoising, background elimination, and normalization on the original image data. The technical means adopted in the preprocessing is a well-known technology in the art and will not be elaborated here.

[0031] The preprocessed image data is enhanced by using a multi-scale recursive perturbation and reverse correction image enhancement algorithm. The multi-scale recursive perturbation and reverse correction image enhancement algorithm combines multi-scale non-linear transformation, recursive enhancement, multi-dimensional random perturbation fusion, and reverse reconstruction, and is gradually realized in combination with mathematical formulas. The specific implementation process is as follows:

[0032] First, in order to separate different frequency components in the appearance image so that high-frequency details and low-frequency structures can be processed independently, the preprocessed image data is subjected to multi-scale decomposition for processing detail information at different scales. Specifically, is decomposed into sub-images of scales (where represents the th scale), which is achieved through the convolution operation of the Gaussian kernel function and the original image . The formula is:

[0033]

[0034] Among them, represents a Gaussian kernel function with scale ; represents a convolution operation.

[0035] To amplify potential detail defects and suppress background noise, based on multi-scale decomposition, a non-linear transformation is performed on the image of each scale. A double non-linear transformation is introduced, combining the hyperbolic tangent function and the composite sine function:

[0036]

[0037] Among them, is the image of the scale after non-linear transformation, containing amplified detail information; is the amplitude coefficient of the non-linear transformation, controlling the intensity of overall detail amplification; is the high-frequency detail enhancement coefficient, controlling the sensitivity of the hyperbolic tangent transformation; is the low-frequency perturbation amplification coefficient, controlling the amplitude of the sine function; is the frequency parameter of the sine function, controlling the frequency of periodic transformation in the image.

[0038] In particular, to avoid the problem of over-enhancement caused by improper parameter selection, the range of parameter values is restricted: .

[0039] Furthermore, the multi-scale image after non-linear transformation is subjected to recursive enhancement processing. By repeatedly extracting and enhancing the effective features in the image, noise interference is eliminated. The recursive enhancement is achieved through the following recursive formula:

[0040]

[0041] Among them, represents the image of the scale after the th iteration; is the image of the scale after the th iteration; is the enhancement coefficient, controlling the amplitude of each recursive enhancement; represents the gradient operation of logarithmic amplification, used to smooth the noise interference in the recursive enhancement process; is a small constant to avoid numerical overflow. The above formula gradually amplifies the effective information in the image through recursive operations, while smoothing the influence of noise. The recursive enhancement processing continues until the iteration number reaches the set threshold Stop iteration.

[0042] The recursively enhanced image may still contain background noise. Introduce texture-structure decomposition to decompose the image into a texture part and a structure part , and only enhance the texture part. The specific decomposition process is as follows:

[0043]

[0044]

[0045]

[0046] where is the structure part image at the scale, which is the result after median filtering and represents the global structure information of the image; is the texture part image at the scale, which is the residue of the recursively enhanced image minus the structure part and represents the local detailed texture information; represents the median filtering operation to extract the smooth structure information of the image.

[0047] Furthermore, after obtaining the recursively enhanced and decomposed images and , introduce multi-dimensional random perturbation processing to simulate the image changes under different noise environments and improve the robustness. Specifically, add the corresponding random perturbation matrices and to the texture part image and of each scale respectively, and obtain the perturbed texture image and the structure image :

[0048]

[0049]

[0050]

[0051]

[0052] where is the texture image at the scale after adding random perturbation; is the structure image at the scale after adding random perturbation; , , , is the perturbation intensity coefficient, which respectively controls the perturbation amplitudes of the texture and structure parts in the horizontal and vertical directions; is the th horizontal perturbation coefficient, satisfying a uniform distribution; is the th vertical perturbation coefficient, satisfying a uniform distribution; , are respectively the period parameters of the horizontal and vertical perturbations of the texture part, controlling the frequency of the sine wave; , are respectively the horizontal and vertical perturbation phase offsets of the texture part, simulating texture perturbations with different phases; , are respectively the period parameters of the horizontal and vertical perturbations of the structure part, controlling the frequency of the sine wave; is the number of perturbation terms; , respectively represent the numbers of the th perturbation in the x - direction and y - direction.

[0053] Furthermore, in order to fuse the perturbation information at different scales, the perturbed images and at all scales are weighted and fused to obtain the final fused image :

[0054]

[0055] where, is the image after multi - scale fusion, containing perturbation information at different scales; is the fusion weight of the th scale, controlling the contribution ratio of different - scale information in the final image; The choice of the fusion weight is determined according to the importance of different - scale information. Usually, the weight of low - frequency structure information is larger, while the weight of high - frequency texture information is smaller; is the total number of scale levels.

[0056] In order to avoid introducing pseudo - defects and noise while the fused image contains rich detail information, a reverse - perturbation correction mechanism is introduced. The fused image is operated in reverse with the perturbation matrix to eliminate the pseudo - defects caused by excessive perturbation. Specifically, the perturbed image is corrected in reverse:

[0057]

[0058] where, is the image after reverse correction, removing the high - frequency noise caused by random perturbation; is the correction coefficient, which is used to adjust the contribution of each perturbation term to the correction process and satisfies the normal distribution; is the two-dimensional perturbation term, which is used to simulate and the combined effect of the tangent perturbations in the and directions, where and are the period parameters of the tangent perturbation and the reverse perturbation, respectively controlling the period parameters in the and directions; 、 are the two-dimensional spatial coordinates of the image, that is, the pixel coordinates, representing the position of each pixel in the image.

[0059] After the reverse correction to obtain the image in order to restore the global structure and detail information of the image, multi-scale reconstruction is performed on the corrected image. During the reconstruction process, the images at each scale are recombined to obtain the final enhanced image :

[0060]

[0061] where, is the finally reconstructed image, that is, the image data after enhancement processing, which contains the effective information at all scales and eliminates the perturbation noise; is the reconstruction weight at the scale, which controls the contribution ratio of the information at each scale during the reconstruction process; represents the inverse Gaussian kernel function at the scale, which is used to restore the original resolution and hierarchical information of the image.

[0062] S2. Construct a multi-layer convolutional neural network as a feature extractor, extract the visual features at different levels in the enhanced image data, then perform feature fusion processing to obtain the fused feature data, and analyze the fused feature data to achieve the classification and localization of the appearance defects of the electric energy meter, so as to achieve the purpose of detecting the appearance defects of the electric energy meter.

[0063] First, construct a multi-layer convolutional neural network. The construction structure is as follows: input layer, take the enhanced image data as the input of the input layer, perform weighted processing and then use it as the output of the input layer; then, through the processing of the convolutional layer (stacking the initial convolutional layer and residual blocks), pooling layer, global average pooling layer, and fully connected layer, visual features at different levels are obtained. This part uses existing technologies and will not be elaborated here.

[0064] Furthermore, the visual features at different levels are processed by an adaptive interactive feature fusion algorithm for feature fusion. The adaptive interactive feature fusion algorithm performs complex interactive mapping, adaptive weighting, and feature fusion on the visual features at different levels, and finally generates an efficient feature representation to improve the accuracy and robustness of appearance defect detection. The specific implementation process is as follows:

[0065] To achieve hierarchical feature transformation, the visual features at different levels are transformed into feature tensors that can express richer semantic information, and linear transformation and non-linear activation operations are performed on the visual features at each level The specific mathematical formula is as follows:

[0066]

[0067] Among them, is the new feature tensor after the transformation of the convolutional features of the th layer; represents a non-linear activation function (such as ReLU), which is used to introduce non-linear features and improve the feature expression ability; is the convolutional kernel parameter matrix of the th layer feature transformation; is the bias term of the th layer feature transformation, which is used for translation correction of the convolutional operation; is the convolutional operation symbol; is the number of channels of the th layer feature tensor, representing the size of the feature dimension; is the convolutional kernel parameter matrix of the th layer and the th channel, which is the weight parameter for separately transforming each channel; is the feature value of the th layer and the th channel; is the bias term of the th layer and the th channel.

[0068] After the transformed feature tensor is formed at each level, multiple interactive mapping processing is performed on the transformed feature tensor. The purpose of multiple interactive mapping is to perform two-way interaction on the feature tensors at different levels and different resolutions, so that each feature tensor can fuse the semantic information of the features at other levels. The interactive mapping operation is specifically defined as:

[0069]

[0070] Among them, is the feature tensor of the th layer and the th layer and any element of the interaction mapping matrix. This matrix contains the correlation measure between two layers of feature tensors; denotes element-wise multiplication (Hadamard Product), representing the element-level product between two feature tensors; is a very small constant used to avoid the problem of zero in the numerator and denominator; is the th layer and the

[0071] After obtaining the interaction mapping matrix composed of , different weights are assigned to each layer of feature tensors according to the information in the interaction mapping matrix, so as to adaptively fuse multi-level features. The specific calculation formula of the weighting coefficient is as follows:

[0072]

[0073] where is the th layer and the is the total number of feature layers, representing the number of all extracted feature layers;

[0074] The feature fusion after dynamic weighting is expressed as:

[0075]

[0076] where is the fused feature tensor, that is, the fused feature data, which contains the comprehensive information of all levels and resolutions and is used as the final fused feature representation; denotes the element-wise addition operation.

[0077] Furthermore, the fused feature data is used to classify and locate the appearance defects of the electric energy meter by using the existing region-based convolutional neural network model. The region-based convolutional neural network model includes the following main steps and technical implementation processes:

[0078] First, the fused feature obtained after fusion is input into the region proposal network to generate candidate regions, and the candidate region R is defined. The region proposal network scans through a sliding window mechanism and matches it with a set of preset sizes called K anchor boxes, and each anchor box generates a candidate region. The output is a set of candidate regions R, where each candidate region contains position coordinates and a target score.

[0079] Next, the elements in each candidate region R are regressed and corrected using a regression formula (given according to the specific scenario) to obtain a more accurate bounding box B.

[0080] In the case of the more accurate bounding box B, for each candidate region, the corresponding features are cropped from and size normalization is performed, which is called the ROI (Region of Interest) pooling operation. The purpose of ROI pooling is to map the features of all candidate regions of different sizes to a feature map of the same size. The feature map after ROI pooling is denoted as .

[0081] Each feature map is input into a fully connected layer for classification to obtain the probability that each candidate region belongs to each category , as well as the probability of the background class . At the same time, the bounding box correction parameters are calculated through the regression branch. The final output of each candidate region is the category probability and the corrected bounding box, and the calculation of the category probability uses the Softmax function.

[0082] Furthermore, after obtaining the final classification and corrected bounding box of each candidate region, the non-maximum suppression (NMS) algorithm is used to remove duplicate detection results. For each category, the bounding boxes are sorted according to the category confidence (i.e., the category probability) , and the intersection over union (IoU) between the candidate bounding box and the bounding box with the highest confidence is calculated in turn. If the IoU is greater than a set threshold (such as 0.5), the candidate box is considered a redundant box and is removed. Finally, the set of optimal bounding boxes for each category is output.

[0083] At this time, the output result of each category includes the defect category class, the confidence and the bounding box. To further improve the accuracy of classification and localization, the average position offset of all detection results of the category is used to further correct each detection result to obtain the final detection result.

[0084] Furthermore, an adaptive threshold determination is performed on the final detection results. First, a statistical analysis is performed on the confidence of all detection results to calculate its mean and standard deviation . According to the confidence distribution characteristics, a dynamic threshold setting strategy is used for determination. When If it is greater than, then the detection result is determined to be a valid defect. If it is less than or equal to, the detection result is considered noise or false detection and is ignored. Among them, is an adjustment factor, and its value generally ranges from 1 to 2.

[0085] The final output result includes all the detection results that pass the threshold determination. Map these detection results back to the original image, label each defect with a bounding box of a different color, and display the corresponding defect category and confidence level on each bounding box. Save the labeled image as the detection result image, and record each detection result in the database to generate a detection report.

[0086] During the process of generating the detection report, count all the detected defect types and their quantities, calculate the confidence distribution of each defect, and draw a distribution histogram. At the same time, record the image coordinates, confidence level, and category information of each detection result to generate a detailed detection record document. The report content includes: the original image, the detection result image, the defect list, the confidence distribution diagram, and the threshold setting strategy description.

[0087] The beneficial effects of the above technical solutions are as follows:

[0088] 1. The multi-scale recursive perturbation and reverse correction image enhancement algorithm can separate different frequency components in the image, and gradually amplify the detailed information in the image through non-linear transformation and recursive enhancement, while suppressing background noise, and can effectively extract small and complex appearance defects, enabling subsequent detection to more accurately identify subtle defects; among them, texture-structure decomposition and multi-dimensional random perturbation can simulate image changes in various noise environments and improve robustness. By introducing a perturbation term, the image changes under different illumination, noise, and interference conditions are simulated, so that in the face of different actual production environments, the efficient defect detection ability can still be maintained. The reverse perturbation correction mechanism effectively eliminates the pseudo-defects and noise interference caused by excessive perturbation, and while maintaining the enhanced details, avoids false detection and false reporting phenomena.

[0089] 2. Through the adaptive interactive feature fusion algorithm, visual features at different levels are dynamically weighted and fused to generate an efficient feature representation. Multiple interactive mapping processing can effectively fuse features at different levels and different resolutions, so that each feature contains richer context information, improving the detection accuracy and generalization ability of the model for complex appearance defects.

[0090] Refer to the appendix Figure 2 , which shows the architecture diagram of an electric energy meter appearance defect detection system based on a neural network according to the present invention, used to implement the above electric energy meter appearance defect detection method, and it includes:

[0091] Image data enhancement processing unit: Obtain the original image data of the appearance of the electricity meter, and perform preprocessing and enhancement processing on the original image data to obtain the enhanced image data;

[0092] Defect recognition unit: Construct a multi-layer convolutional neural network as a feature extractor to extract visual features at different levels from the enhanced image data, and then perform feature fusion processing to obtain the fused feature data. Analyze the fused feature data to achieve the classification and positioning of the appearance defects of the electricity meter, and achieve the purpose of detecting the appearance defects of the electricity meter.

[0093] It should be noted that each unit in the above-mentioned electricity meter appearance defect detection system based on neural network can be implemented in whole or in part by software, hardware and their combination. The above-mentioned units can be embedded in the processor of the computer device in hardware form or independent of it, or stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to the above-mentioned modules. For the specific limitations of an electricity meter appearance defect detection system based on neural network, refer to the limitations of an electricity meter appearance defect detection method based on neural network in the above text. The two have the same functions and effects, and will not be elaborated here.

[0094] Next, a detailed description will be made for each key part in the electricity meter appearance defect detection method.

[0095] Embodiment 1: Image processing of the appearance of the electricity meter based on the multi-scale recursive perturbation and reverse correction image enhancement algorithm

[0096] Step 1: Acquisition and preprocessing of the original image data

[0097] In the industrial production environment, professional technicians select high-resolution industrial cameras suitable for detecting defect details to comprehensively capture the original image data of the appearance of the electricity meter. These image data may include various types of surface defects of the electricity meter, such as scratches, dents, stains, etc. In order to improve the detection accuracy and reliability, it is necessary to preprocess the collected original image data. The preprocessing includes the following steps:

[0098] First, perform denoising processing on the original image data `I`, and use median filtering or bilateral filtering algorithms to eliminate high-frequency noise in the image while maintaining edge detail information. Then, perform background elimination processing on the image, and use edge detection algorithms or watershed algorithms to separate the target electricity meter from the background to ensure that only the surface features of the electricity meter are concerned in the subsequent processing. Finally, perform normalization processing on the image data, map the pixel values to the range of [0,1] to reduce the influence of illumination changes on the detection. The above preprocessing methods have been widely used in this field and will not be elaborated here.

[0099] Step 2: Multi-scale Recursive Perturbation and Reverse Correction Image Enhancement

[0100] Apply the multi-scale recursive perturbation and reverse correction image enhancement algorithm to the preprocessed image data `I` to improve the visibility of tiny defects in the image while suppressing background noise. The specific implementation steps are as follows:

[0101] First, perform multi-scale decomposition on the preprocessed image, decomposing it into sub-images of 4 scales. The multi-scale decomposition is achieved by convolving the preprocessed image with Gaussian kernels of different standard deviations. The scale parameters of the Gaussian function are respectively and . The sizes of the Gaussian kernels are 3x3, 5x5, 7x7, and 9x9 respectively.

[0102] Next, perform a double non-linear transformation on each scale of the image to amplify potential detailed defects and suppress background noise. Introduce the hyperbolic tangent function and the composite sine function to perform non-linear transformation on the image; the parameters are set as: .

[0103] Then, perform recursive enhancement processing on the multi-scale images after non-linear transformation, gradually extract and amplify the effective features in the image, and eliminate noise interference. Set the recursive enhancement parameter as: .

[0104] Furthermore, in order to remove the background noise introduced during the recursive enhancement process, decompose the recursively enhanced image into a texture part and a structure part.

[0105] To further improve the robustness of image enhancement, introduce multi-dimensional random perturbation processing, and add random perturbation matrices to the texture part and the structure part of each scale respectively.

[0106] Finally, perform weighted fusion on the perturbed images of all scales to obtain the final enhanced image.

[0107] After obtaining the fused image, perform reverse perturbation correction to eliminate the pseudo-defects and noise caused by excessive perturbation.

[0108] Finally, perform multi-scale reconstruction on the image after reverse correction to restore the global structure and detailed information of the image, and obtain the final enhanced image.

[0109] Example 2: Multi-level Feature Extraction and Fusion Based on Convolutional Neural Network

[0110] Step 1: Feature Extraction of Multi-layer Convolutional Neural Network

[0111] Use the enhanced processed image data as the input of the convolutional neural network. The network structure includes an input layer, an initial convolutional layer, a stack of multiple residual blocks, a global average pooling layer, and a fully connected layer. The input layer weights the enhanced processed image data and uses it as the input of the initial convolutional layer. The initial convolutional layer extracts basic low-level features such as edges and corners. The stack of residual blocks extracts high-level semantic features through multiple layers of convolution and residual connections and alleviates the vanishing gradient problem in deep networks. Finally, through the processing of the global average pooling layer and the fully connected layer, visual features at different levels are obtained.

[0112] Step 2: Adaptive interactive feature fusion

[0113] For the extracted visual features at different levels, use the adaptive interactive feature fusion algorithm for feature fusion. Each level of visual features generates a new feature tensor through a transformation formula. Then, perform two-way interactive mapping on the transformed feature tensors of all levels, calculate the interactive mapping matrix, calculate the dynamic weighting coefficients based on the information in the interactive mapping matrix, and obtain the final fused feature representation based on the dynamic weighting coefficients.

[0114] Step 3: Defect classification and localization based on region convolutional neural network

[0115] Input the fused features into the region proposal network to generate candidate regions, and obtain more accurate bounding boxes through regression correction. For each candidate region, crop the corresponding features from the fused features and perform ROI pooling to obtain a feature map of a fixed size.

[0116] Perform a fully connected layer process on the feature map of the fixed size to obtain the class probability and background probability of each candidate region. At the same time, calculate the bounding box correction parameters through the regression branch. The final output of each candidate region is the class probability and the corrected bounding box.

[0117] After the final classification and bounding box determination of all candidate regions, use non-maximum suppression (NMS) to remove redundant detection results. For each category, sort the bounding boxes according to the confidence level, and calculate the IoU between the candidate bounding box and the bounding box with the highest confidence level. If the IoU is greater than the set threshold, then the candidate box is considered a redundant box and is removed.

[0118] Example 3: Adaptive threshold determination and detection result output

[0119] Step 1: Adaptive threshold determination

[0120] Perform statistical analysis on the confidence levels of all detection results, and calculate the mean and standard deviation. Adopt a dynamic threshold setting strategy. When the confidence level is greater than or equal to the dynamic threshold, then determine that the detection result is a valid defect; if it is less than or equal to, then ignore the result.

[0121] Step 2: Detection Result Output and Visualization

[0122] Map all the detection results determined by the threshold back to the original image, label each type of defect with bounding boxes of different colors, and display the corresponding defect category and confidence level on each bounding box. Save the labeled image as the detection result image and generate a detection report.

[0123] Step 3: Generate a Detection Report

[0124] Record all the detected defect types and their quantities, calculate the confidence level distribution of each type of defect, and draw a distribution histogram. Generate a detailed detection record document, including: the original image, the detection result image, the defect list, the confidence level distribution graph, and the description of the threshold setting strategy.

[0125] The foregoing embodiments have described the present invention in detail. Those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A method for detecting appearance defects of electric energy meters based on neural networks, characterized in that: Includes steps: S1. Obtaining the original image data of the appearance of the electric energy meter, and preprocessing and enhancing the original image data to obtain enhanced image data; The preprocessed image data is enhanced using a multi-scale recursive perturbation and inverse correction image enhancement algorithm; In the process of implementing the multi-scale recursive perturbation and inverse correction image enhancement algorithm, in order to separate the different frequency components in the appearance image, the pre-processed image data is decomposed at multiple scales; in order to amplify the potential detail defects and suppress the background noise, a double nonlinear transformation is introduced on the basis of multi-scale decomposition, and a nonlinear transformation is performed on the image of each scale; Recursive enhancement is performed on the multi-scale image after nonlinear transformation to eliminate noise interference by repeatedly extracting and enhancing the effective features in the image. The image after recursive enhancement may still contain background noise. Texture-structure decomposition is introduced to decompose the image after recursive enhancement into texture part and structure part. After obtaining the recursively enhanced and decomposed images, multi-dimensional random perturbation processing is introduced to simulate image changes in different noise environments; in order to fuse the perturbation information at different scales, the perturbed images of all scales are weighted fused to obtain the final fused image; an inverse perturbation correction mechanism is introduced to perform inverse operations on the fused image and the perturbation matrix to eliminate pseudo defects caused by excessive perturbations; after the image is obtained by inverse perturbation correction, the image after inverse perturbation correction is reconstructed at multiple scales to restore the global structure and detail information of the image; S2. Construct a multi-layer convolutional neural network as a feature extractor to extract visual features at different levels from the enhanced image data, then perform feature fusion processing to obtain fused feature data, analyze the fused feature data, and realize the classification and positioning of appearance defects of the electric energy meter.

2. The method for detecting appearance defects of electric energy meters based on a neural network according to claim 1, characterized in that: The S1 specifically includes: An industrial camera for detecting defect details is used to comprehensively capture the original image data of the appearance of the electric energy meter, and the original image data is preprocessed by denoising, background removal and normalization.

3. The method for detecting appearance defects of electric energy meters based on neural networks according to claim 1, characterized in that: In S2, the visual features at different levels are subjected to feature fusion processing using an adaptive interactive feature fusion algorithm. The adaptive interactive feature fusion algorithm processes the visual features at different levels through interactive mapping, adaptive weighting and feature fusion, and finally generates an efficient feature representation.

4. The method for detecting appearance defects of electric energy meters based on a neural network according to claim 3 is characterized in that: In the adaptive interactive feature fusion algorithm, in order to realize hierarchical feature transformation, visual features at different levels are converted into feature tensors that can express richer semantic information, and linear transformation and nonlinear activation operations are performed on the visual features at each level; after the transformation feature tensor at each level is formed, the transformation feature tensor is subjected to multiple interactive mapping processing. The purpose of multiple interactive mapping is to bidirectionally interact feature tensors of different levels and resolutions so that each feature tensor can fuse the semantic information of features at other levels; after obtaining the interactive mapping matrix, different weights are assigned to the feature tensors at each level according to the information in the interactive mapping matrix, so as to adaptively fuse multi-level features.

5. A system for detecting appearance defects of electric energy meters based on a neural network, used to implement the method for detecting appearance defects of electric energy meters according to any one of claims 1 to 4, characterized in that: include: Image data enhancement processing unit: obtains original image data of the appearance of the electric energy meter, and performs preprocessing and enhancement processing on the original image data to obtain enhanced image data; Defect recognition unit: Construct a multi-layer convolutional neural network as a feature extractor to extract visual features at different levels from the enhanced image data, then perform feature fusion processing to obtain fused feature data, analyze the fused feature data, and realize the classification and positioning of appearance defects of the electric energy meter.

Images