Electric energy meter appearance defect detection method and system based on neural network

Abstract

The invention discloses an electric energy meter appearance defect detection method and system based on a neural network. The method for detecting the appearance defects of the electric energy meter comprises the following steps: acquiring original image data of the appearance of the electric energy meter, and preprocessing and enhancing the original image data to obtain enhanced image data; and constructing a multi-layer convolutional neural network as a feature extractor, extracting visual features of different levels in the image data after enhancement processing, then performing feature fusion processing to obtain fused feature data, and analyzing the fused feature data to realize classification and positioning of the appearance defects of the electric energy meter. The objective of electric energy meter appearance defect detection is achieved. According to the method, tiny and complex appearance defects can be effectively extracted, so that the subtle defects can be more accurately identified in subsequent detection; according to the method, visual features of different levels are fused, and the detection precision and generalization ability of complex appearance defects are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of appearance detection, and in particular to a method and system for detecting appearance defects of an electric energy meter based on a neural network. Background Art

[0002] In the industrial production process, the appearance quality of the electric energy meter has an important impact on its market competitiveness and user satisfaction. Therefore, the appearance defect detection of the electric energy meter has become a key link in quality control. Traditional electric energy meter appearance defect detection usually relies on manual inspection, but manual inspection has many shortcomings: First, manual inspection is inefficient and easily affected by the experience and status of the inspectors. It is not only difficult to meet the needs of large-scale production, but also the stability and consistency of the inspection results are poor. In addition, manual inspection is often difficult to identify small or complex defects (such as fine scratches, unevenness, surface stains, etc.), and missed detection and false detection often occur. Especially in long-term work, the fatigue effect of manual inspection further exacerbates the risk of misjudgment.

[0003] In summary, there are at least the following technical problems: the appearance data processing during the appearance inspection of the electric energy meter is not accurate enough and the inspection accuracy is low. Summary of the invention

[0004] The technical problem to be solved by the present invention is to overcome the defects of the above-mentioned prior art and provide a method and system for detecting appearance defects of an electric energy meter based on a neural network, so as to effectively extract tiny and complex appearance defects, so that subsequent detection can more accurately identify subtle defects.

[0005] To this end, the present invention adopts the following technical solution.

[0006] In a first aspect, the present invention provides a method for detecting appearance defects of an electric energy meter based on a neural network, which comprises the steps of: S1. Obtaining the original image data of the appearance of the electric energy meter, and preprocessing and enhancing the original image data to obtain enhanced image data; S2. Construct a multi-layer convolutional neural network as a feature extractor to extract visual features at different levels from the enhanced image data, then perform feature fusion processing to obtain fused feature data, analyze the fused feature data, and realize the classification and positioning of the appearance defects of the electric energy meter, so as to achieve the purpose of appearance defect detection of the electric energy meter.

[0007] Furthermore, the S1 specifically includes: using an industrial camera for detecting defect details to comprehensively capture the original image data of the appearance of the electric energy meter, pre-processing the original image data by denoising, background elimination and normalization, and enhancing the pre-processed image data by using a multi-scale recursive perturbation and inverse correction image enhancement algorithm.

[0008] Furthermore, in S1, the multi-scale recursive perturbation and inverse correction image enhancement algorithm integrates multi-scale nonlinear transformation, recursive enhancement, multi-dimensional random perturbation fusion and inverse reconstruction.

[0009] Furthermore, in the process of implementing the multi-scale recursive perturbation and inverse correction image enhancement algorithm, in order to separate the different frequency components in the appearance image so that the high-frequency details and low-frequency structures can be processed independently, the pre-processed image data is decomposed at a multi-scale; in order to amplify potential detail defects and suppress background noise, a double nonlinear transformation is introduced on the basis of multi-scale decomposition, and a nonlinear transformation is performed on the image of each scale; the multi-scale image after the nonlinear transformation is recursively enhanced to eliminate noise interference by repeatedly extracting and enhancing the effective features in the image.

[0010] Furthermore, in the process of implementing the multi-scale recursive perturbation and inverse correction image enhancement algorithm, the recursively enhanced image may still contain background noise. Texture-structure decomposition is introduced to decompose the recursively enhanced image into a texture part and a structure part.

[0011] Furthermore, in the implementation process of the multi-scale recursive perturbation and inverse correction image enhancement algorithm, after obtaining the recursively enhanced and decomposed image, multi-dimensional random perturbation processing is introduced to simulate the image changes in different noise environments and improve the robustness; in order to fuse the perturbation information at different scales, the perturbed images of all scales are weightedly fused to obtain the final fused image.

[0012] Furthermore, in the process of implementing the multi-scale recursive perturbation and inverse correction image enhancement algorithm, in order to avoid the introduction of pseudo defects and noise in the fused image while containing rich detail information, an inverse perturbation correction mechanism is introduced, and the fused image and the perturbation matrix are reversely operated to eliminate the pseudo defects caused by excessive perturbation; after the image is obtained by inverse perturbation correction, in order to restore the global structure and detail information of the image, the image after inverse perturbation correction is reconstructed at multiple scales.

[0013] Furthermore, in S2, the visual features at different levels are subjected to feature fusion processing using an adaptive interactive feature fusion algorithm. The adaptive interactive feature fusion algorithm processes the visual features at different levels through interactive mapping, adaptive weighting and feature fusion, and finally generates an efficient feature representation to improve the accuracy and robustness of the detection of appearance defects of electric energy meters.

[0014] Furthermore, in the adaptive interactive feature fusion algorithm, in order to realize hierarchical feature transformation, visual features at different levels are converted into feature tensors that can express richer semantic information, and linear transformation and nonlinear activation operations are performed on the visual features at each level; after the transformation feature tensor at each level is formed, the transformation feature tensor is subjected to multiple interactive mapping processing. The purpose of multiple interactive mapping is to bidirectionally interact feature tensors of different levels and resolutions, so that each feature tensor can fuse the semantic information of features at other levels; after obtaining the interactive mapping matrix, different weights are assigned to the feature tensors at each level according to the information in the interactive mapping matrix, so as to adaptively fuse multi-level features.

[0015] In a second aspect, the present invention provides an electric energy meter appearance defect detection system based on a neural network, which is used to implement the above-mentioned electric energy meter appearance defect detection method, and comprises: Image data enhancement processing unit: obtains original image data of the appearance of the electric energy meter, and performs preprocessing and enhancement processing on the original image data to obtain enhanced image data; Defect recognition unit: Construct a multi-layer convolutional neural network as a feature extractor to extract visual features at different levels from the enhanced image data, then perform feature fusion processing to obtain fused feature data, analyze the fused feature data, and classify and locate the appearance defects of the electric energy meter, so as to achieve the purpose of appearance defect detection of the electric energy meter.

[0016] The present invention has the following beneficial effects: 1. The present invention can effectively extract small and complex appearance defects, so that subsequent inspections can more accurately identify subtle defects; 2. The present invention fuses visual features at different levels to generate efficient feature representation, so that each feature contains richer contextual information, thereby improving the detection accuracy and generalization ability of complex appearance defects. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a flow chart of a method for detecting appearance defects of electric energy meters based on a neural network according to the present invention; Figure 2 This is an architecture diagram of a neural network-based electric energy meter appearance defect detection system of the present invention. DETAILED DESCRIPTION

[0018] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the technical scheme in the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiment of the present invention. Obviously, the described embodiment is only a part of the embodiment of the present invention, not all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0019] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0020] The specific technical solution for detecting appearance defects of electric energy meters based on neural networks provided by the present invention is described in detail below in conjunction with the accompanying drawings.

[0021] See attached Figure 1 , which shows a flow chart of a method for detecting appearance defects of an electric energy meter based on a neural network according to the present invention, the method comprising the following steps: S1. Obtaining original image data of the appearance of the electric energy meter, and performing preprocessing and enhancement processing on the original image data to obtain enhanced image data.

[0022] Professional technicians select industrial cameras suitable for detecting defect details to comprehensively capture the original image data of the appearance of the electric energy meter, and perform denoising, background removal, and normalization preprocessing on the original image data. The technical means used in the preprocessing are well known in the art and will not be described in detail here.

[0023] The preprocessed image data is enhanced using a multi-scale recursive perturbation and inverse correction image enhancement algorithm, which integrates multi-scale nonlinear transformation, recursive enhancement, multi-dimensional random perturbation fusion and inverse reconstruction, and is gradually implemented in combination with mathematical formulas. The specific implementation process is as follows: First, in order to separate the different frequency components in the appearance image so that the high-frequency details and low-frequency structures can be processed independently, the preprocessed image data Multi-scale decomposition is performed to process detail information at different scales. Specifically, is decomposed into Sub-images of different scales (in Indicates scales), through the Gaussian kernel function With the original image The convolution operation is implemented. The formula is:

[0024] in, The scale is Gaussian kernel function; Represents a convolution operation.

[0025] In order to amplify potential detail defects and suppress background noise, nonlinear transformation is performed on each scale image based on multi-scale decomposition. Double nonlinear transformation is introduced to combine the hyperbolic tangent function with the composite sine function:

[0026] in, is the first Scale image, containing magnified detail information; It is the amplitude coefficient of the nonlinear transformation, which controls the intensity of overall detail amplification; is the high-frequency detail enhancement coefficient, which controls the sensitivity of the hyperbolic tangent transform; is the low-frequency disturbance amplification factor, which controls the amplitude of the sine function; is the sine function frequency parameter, which controls the frequency of periodic transformation in the image.

[0027] In particular, in order to avoid the problem of over-enhancement due to improper parameter selection, the parameter value range is constrained: .

[0028] Furthermore, for the multi-scale image after nonlinear transformation Perform recursive enhancement processing. By repeatedly extracting and enhancing the effective features in the image, noise interference is eliminated. Recursive enhancement is achieved through the following recursive formula:

[0029] in, Indicates After the iteration Scale image; It is After the iteration Scale image; is the enhancement coefficient, which controls the magnitude of each recursive enhancement; represents the gradient operation of logarithmic amplification, which is used to smooth the noise interference in the recursive enhancement process; A small constant to avoid numerical overflow. The above formula gradually amplifies the effective information in the image through recursive operations, while smoothing the noise effect. Recursive enhancement processing until the number of iterations Reaching the set threshold Stop iteration.

[0030] Image after recursive enhancement It may still contain background noise, so texture-structure decomposition is introduced to decompose the image into texture parts. and structural parts , and only the texture part is enhanced. The specific decomposition process is as follows:

[0031]

[0032]

[0033] in, It is The result of median filtering of the structural part of the image at a certain scale represents the global structural information of the image; It is The texture part image of the scale is the residue after the recursive enhancement image minus the structure part, which represents the local detail texture information; Represents the median filter operation, which extracts the smooth structure information of the image.

[0034] Further, after obtaining the recursively enhanced and decomposed image and Afterwards, multi-dimensional random perturbation processing is introduced to simulate image changes in different noise environments and improve robustness. Specifically, for each scale of the texture part image And the structure part image Add the corresponding random perturbation matrices respectively and , get the perturbed texture image and structural images :

[0035]

[0036]

[0037]

[0038] in, is the first Scale texture images; is the first Scale structure images; , , , is the perturbation intensity coefficient, which controls the perturbation amplitude of the texture and structure parts in the horizontal and vertical directions respectively; It is horizontal disturbance coefficients, satisfying uniform distribution; It is vertical disturbance coefficients, satisfying uniform distribution; , They are the periodic parameters of the horizontal and vertical disturbances of the texture part, respectively, controlling the frequency of the sine wave; , They are the horizontal and vertical perturbation phase offsets of the texture part, simulating texture perturbations of different phases; , They are the periodic parameters of the horizontal and vertical disturbances of the structural part, respectively, controlling the frequency of the sine wave; is the number of disturbance terms; , Respectively represent The number of the perturbations in the x and y directions.

[0039] Furthermore, in order to fuse the perturbation information at different scales, the perturbation images of all scales are and Perform weighted fusion to obtain the final fused image :

[0040] in, It is a multi-scale fused image, containing disturbance information of different scales; For the The scale fusion weight controls the contribution ratio of information of different scales in the final image. The selection of fusion weight is determined according to the importance of information of different scales. Usually, the weight of low-frequency structural information is larger, while the weight of high-frequency texture information is smaller. is the total number of scale layers.

[0041] In order to avoid the fused image While containing rich detail information, it introduces pseudo defects and noise, and introduces a reverse disturbance correction mechanism to reversely operate the fused image and the disturbance matrix to eliminate pseudo defects caused by excessive disturbance. Specifically, reverse correction is performed on the disturbed image:

[0042] in, It is the image after reverse correction, removing the high-frequency noise caused by random disturbance; is the correction coefficient, which is used to adjust the contribution of each disturbance term to the correction process to satisfy the normal distribution; is a two-dimensional disturbance term used to simulate and The combined effect of the tangential disturbance in the direction, where and are the period parameters of the tangent disturbance and the reverse disturbance, respectively controlling and Period parameter of direction; It is a two-dimensional reverse disturbance term, which is used to balance the high-frequency noise in the tangent disturbance and reduce the error in the correction process; is the number of inverse disturbance correction terms; , It is the two-dimensional spatial coordinate of the image, that is, the pixel coordinate, which indicates the position of each pixel in the image.

[0043] The image is obtained by reverse correction Finally, in order to restore the global structure and detail information of the image, the corrected image is reconstructed at multiple scales. During the reconstruction process, the images at each scale are recombined to obtain the final enhanced image. :

[0044] in, It is the final reconstructed image, that is, the image data after enhancement processing, which contains effective information of all scales and eliminates disturbance noise; For the The reconstruction weight of the scale controls the contribution ratio of each scale information in the reconstruction process; Indicates The inverse Gaussian kernel function of the scale is used to restore the original resolution and layer information of the image.

[0045] S2. Construct a multi-layer convolutional neural network as a feature extractor to extract visual features at different levels from the enhanced image data, then perform feature fusion processing to obtain fused feature data, analyze the fused feature data, and realize the classification and positioning of the appearance defects of the electric energy meter, so as to achieve the purpose of appearance defect detection of the electric energy meter.

[0046] First, a multi-layer convolutional neural network is constructed with the following structure: the input layer uses the enhanced image data as the input of the input layer, and the weighted processing is used as the output of the input layer; then the convolution layer (initial convolution layer and residual block stacking), pooling layer, global average pooling layer, and fully connected layer are processed to obtain visual features at different levels. This part adopts the existing technology and will not be described in detail here.

[0047] The adaptive interactive feature fusion algorithm is further used to perform feature fusion processing on visual features at different levels. The adaptive interactive feature fusion algorithm processes visual features at different levels through complex interactive mapping, adaptive weighting, and feature fusion, and finally generates an efficient feature representation to improve the accuracy and robustness of appearance defect detection. The specific implementation process is as follows: In order to achieve hierarchical feature transformation, the visual features at different levels are transformed into feature tensors that can express richer semantic information. Perform linear transformation and nonlinear activation operations. The specific mathematical formula is as follows:

[0048] in, It is The new feature tensor after the layer convolution features are transformed; Represents a nonlinear activation function (such as ReLU), which is used to introduce nonlinear features and improve feature expression capabilities; It is Convolution kernel parameter matrix of layer feature transformation; It is The bias term of the layer feature transformation is used for translation correction of the convolution operation; is the convolution operation symbol; It is The number of channels of the layer feature tensor, indicating the size of the feature dimension; It is Layer The convolution kernel parameter matrix of the channels is the weight parameter for transforming each channel separately; It is Layer The characteristic value of each channel; It is Layer The offset term for each channel.

[0049] Transformed feature tensor at each level After formation, the transformed feature tensor is subjected to multiple interactive mapping. The purpose of multiple interactive mapping is to bidirectionally interact feature tensors of different levels and resolutions so that each feature tensor can integrate the semantic information of features at other levels. The interactive mapping operation is specifically defined as:

[0050] in, It is Layer and Layer feature tensor and Any element of the interaction mapping matrix of . This matrix contains the correlation measure between the two layers of feature tensors; Represents element-wise multiplication (Hadamard Product), which represents the element-level product between two feature tensors; is a very small constant used to avoid the problem of the numerator and denominator being zero; It is Layer and The number of channels of the layer feature tensor represents the cosine function of the feature dimension.

[0051] In getting by The interaction mapping matrix Afterwards, different weights are assigned to each level feature tensor according to the information in the interaction mapping matrix, so as to adaptively fuse multi-level features. The specific weight coefficient calculation formula is as follows:

[0052] in, It is Layer and The dynamic weighting coefficient between the layer feature tensors indicates the contribution of the two feature tensors to the final fusion feature; is the total number of feature layers, indicating the number of all feature layers extracted; The dynamic weighted feature fusion is expressed as:

[0053] in, It is the fused feature tensor, i.e. the fused feature data, which contains the comprehensive information of all levels and resolutions as the final fused feature representation; Represents an element-by-element addition operation.

[0054] Furthermore, the fused feature data is used to classify and locate the appearance defects of the electric energy meter using an existing region-based convolutional neural network model. The region-based convolutional neural network model includes the following main steps and technical implementation process: First, the fusion feature is obtained after fusion Input into the region proposal network to generate candidate regions and define the candidate region R. The region proposal network scans through a sliding window mechanism , and use a set of preset sizes called K anchor boxes for matching, each of which generates a candidate region. The output of is a set of candidate regions R, each of which contains location coordinates and target scores.

[0055] Next, the elements in each candidate region R are regressed and corrected using a regression formula (given according to the specific scenario) to obtain a more accurate bounding box B.

[0056] In the case of a more accurate bounding box B, for each candidate region, The corresponding features are cut out from the image and normalized to the same size, which is called ROI (Region of Interest) pooling operation. The purpose of ROI pooling is to map all candidate region features of different sizes to feature maps of the same size. The feature map after ROI pooling is recorded as .

[0057] Each The feature map is input to the fully connected layer for classification to obtain the probability that each candidate region belongs to each category. , and the probability of the background class At the same time, the bounding box correction parameters are calculated through the regression branch. Finally, the output of each candidate region is the category probability And the corrected bounding box, the category probability is calculated using the Softmax function.

[0058] Furthermore, after obtaining the final classification and corrected bounding box of each candidate region, the Non-Maximum Suppression (NMS) algorithm is used to remove duplicate detection results. For each category, the category confidence (i.e., category probability) is used. Sort the bounding boxes and calculate the intersection over union (IoU) between the candidate bounding box and the highest confidence bounding box. If the IoU is greater than the set threshold (such as 0.5), the candidate box is considered a redundant box and is removed. Finally, the optimal bounding box set for each category is output.

[0059] At this time, the output results of each category include defect category class, confidence In order to further improve the accuracy of classification and positioning, the average position offset of all detection results of the category is used to further calibrate each detection result to obtain the final detection result.

[0060] Furthermore, the final detection results are subjected to adaptive threshold determination. First, the confidence of all detection results is Perform statistical analysis and calculate the mean and standard deviation According to the confidence distribution characteristics, a dynamic threshold setting strategy is used for judgment. When , the test result is determined to be a valid defect. If it is less than or equal to , the test result is considered to be noise or false detection and ignored. It is an adjustment factor, and its value is generally between 1 and 2.

[0061] The final output contains all the test results that pass the threshold judgment. These test results are mapped back to the original image, and each defect is annotated with a bounding box of different colors, and the corresponding defect category and confidence level are displayed on each bounding box. The annotated image is saved as a test result map, and each test result is recorded in the database to generate a test report.

[0062] In the process of generating the inspection report, all the detected defect types and their quantities are counted, the confidence distribution of each defect is calculated, and the distribution histogram is drawn. At the same time, the image coordinates, confidence and category information of each inspection result are recorded to generate a detailed inspection record document. The report content includes: original image, inspection result map, defect list, confidence distribution map and threshold setting strategy description.

[0063] The above technical solution has the following beneficial effects: 1. The multi-scale recursive perturbation and reverse correction image enhancement algorithm can separate the different frequency components in the image, and gradually amplify the detailed information in the image through nonlinear transformation and recursive enhancement, while suppressing the background noise, which can effectively extract small and complex appearance defects, so that subsequent detection can more accurately identify subtle defects; among them, texture-structure decomposition and multi-dimensional random perturbation can simulate image changes in various noise environments and improve robustness. By introducing perturbation terms, the image changes under different lighting, noise and interference conditions are simulated, so that when facing different actual production environments, efficient defect detection capabilities can still be maintained. The reverse perturbation correction mechanism effectively eliminates pseudo-defects and noise interference caused by excessive perturbations, while maintaining enhanced details, avoiding false detection and false reporting.

[0064] 2. Through the adaptive interactive feature fusion algorithm, visual features at different levels are dynamically weighted and fused to generate efficient feature representation. Multiple interactive mapping processing can effectively fuse features at different levels and resolutions, so that each feature contains richer contextual information, improving the detection accuracy and generalization ability of the model for complex appearance defects.

[0065] See attached Figure 2 , which shows an architecture diagram of an electric energy meter appearance defect detection system based on a neural network of the present invention, which is used to implement the above-mentioned electric energy meter appearance defect detection method, and includes: Image data enhancement processing unit: obtains original image data of the appearance of the electric energy meter, and performs preprocessing and enhancement processing on the original image data to obtain enhanced image data; Defect recognition unit: Construct a multi-layer convolutional neural network as a feature extractor to extract visual features at different levels from the enhanced image data, then perform feature fusion processing to obtain fused feature data, analyze the fused feature data, and classify and locate the appearance defects of the electric energy meter, so as to achieve the purpose of appearance defect detection of the electric energy meter.

[0066] It should be noted that each unit in the above-mentioned electric energy meter appearance defect detection system based on a neural network can be fully or partially implemented by software, hardware and a combination thereof. The above-mentioned units can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above-mentioned modules. For the specific definition of an electric energy meter appearance defect detection system based on a neural network, please refer to the definition of an electric energy meter appearance defect detection method based on a neural network above. The two have the same functions and effects, which will not be repeated here.

[0067] The following is a detailed description of the key parts of the electric energy meter appearance defect detection method.

[0068] Example 1: Electricity meter appearance image processing based on multi-scale recursive perturbation and inverse correction image enhancement algorithm Step 1: Collection and preprocessing of raw image data In an industrial production environment, professional technicians select high-resolution industrial cameras suitable for detecting defect details to fully capture the original image data of the appearance of the electric energy meter. These image data may include various types of surface defects of the electric energy meter, such as scratches, dents, stains, etc. In order to improve the accuracy and reliability of detection, the collected original image data needs to be preprocessed. Preprocessing includes the following steps: First, the original image data `I` is denoised, and a median filter or bilateral filter algorithm is used to remove high-frequency noise in the image while maintaining edge detail information. Next, the image is subjected to background removal, and an edge detection algorithm or a watershed algorithm is used to separate the target electric energy meter from the background, ensuring that only the surface features of the electric energy meter are focused on during subsequent processing. Finally, the image data is normalized, and the pixel values ​​are mapped to the range of [0,1] to reduce the impact of illumination changes on detection. The above preprocessing methods have been widely used in this field and will not be described in detail here.

[0069] Step 2: Multi-scale recursive perturbation and inverse correction image enhancement For the preprocessed image data `I`, a multi-scale recursive perturbation and inverse correction image enhancement algorithm is applied to improve the visibility of tiny defects in the image while suppressing background noise. The specific implementation steps are as follows: First, the preprocessed image is decomposed into four sub-images at four scales. The multi-scale decomposition is achieved by convolving the preprocessed image with Gaussian kernels of different standard deviations. The scale parameter of the Gaussian function is They are and The sizes of the Gaussian kernels are 3x3, 5x5, 7x7, and 9x9.

[0070] Next, a double nonlinear transformation is performed on the image at each scale to amplify potential detail defects and suppress background noise. The hyperbolic tangent function and the composite sine function are introduced to perform a nonlinear transformation on the image; the parameters are set as follows: .

[0071] Then, the multi-scale image after nonlinear transformation is recursively enhanced to gradually extract and amplify the effective features in the image and eliminate noise interference. The recursive enhancement parameters are set as: .

[0072] Furthermore, in order to remove the background noise introduced in the recursive enhancement process, the recursively enhanced image is decomposed into a texture part and a structure part.

[0073] In order to further improve the robustness of image enhancement, multi-dimensional random perturbation processing is introduced, and random perturbation matrices are added to the texture part and structure part of each scale respectively.

[0074] Finally, the perturbed images of all scales are weighted fused to obtain the final enhanced image.

[0075] After obtaining the fused image, inverse perturbation correction is performed to eliminate pseudo defects and noise caused by excessive perturbation.

[0076] Finally, the inversely corrected image is reconstructed at multiple scales to restore the global structure and detail information of the image and obtain the final enhanced image.

[0077] Example 2: Multi-level feature extraction and fusion based on convolutional neural network Step 1: Feature extraction with multi-layer convolutional neural network The enhanced image data is used as the input of the convolutional neural network. The network structure includes an input layer, an initial convolution layer, multiple residual block stacks, a global average pooling layer, and a fully connected layer. The input layer uses the enhanced image data as the input of the initial convolution layer after weighted processing. The initial convolution layer extracts basic low-level features such as edges and corners. The residual block stack extracts high-level semantic features through multiple layers of convolution and residual connections, and alleviates the gradient vanishing problem in the deep network. Finally, after processing by the global average pooling layer and the fully connected layer, visual features at different levels are obtained.

[0078] Step 2: Adaptive interactive feature fusion The adaptive interactive feature fusion algorithm is used to fuse the extracted visual features at different levels. The visual features at each level generate new feature tensors through the transformation formula. Then, the transformed feature tensors at all levels are bidirectionally interactively mapped, and the interactive mapping matrix is ​​calculated. According to the information in the interactive mapping matrix, the dynamic weighting coefficient is calculated, and the final fused feature representation is obtained based on the dynamic weighting coefficient.

[0079] Step 3: Defect classification and localization based on regional convolutional neural network The fused features are input into the region proposal network to generate candidate regions, and a more accurate bounding box is obtained through regression correction. For each candidate region, the corresponding features are cropped from the fused features and ROI pooling is performed to obtain a fixed-size feature map.

[0080] The fixed-size feature map is processed by the fully connected layer to obtain the category probability and background probability of each candidate region. At the same time, the bounding box correction parameters are calculated through the regression branch. The final output of each candidate region is the category probability and the corrected bounding box.

[0081] After the final classification and bounding box of all candidate regions are determined, non-maximum suppression (NMS) is used to remove redundant detection results. For each category, the bounding boxes are sorted according to the confidence level, and the IoU between the candidate bounding box and the highest confidence bounding box is calculated. If the IoU is greater than the set threshold, the candidate box is considered to be a redundant box and is removed.

[0082] Embodiment 3: Adaptive threshold determination and detection result output Step 1: Adaptive threshold determination Perform statistical analysis on the confidence of all test results, and calculate the mean and standard deviation. Use a dynamic threshold setting strategy. When the confidence is greater than or equal to the dynamic threshold, the test result is determined to be a valid defect. If it is less than or equal to the dynamic threshold, the result is ignored.

[0083] Step 2: Output and visualization of detection results Map all the test results that pass the threshold judgment back to the original image, use different colored bounding boxes to mark each defect, and display the corresponding defect category and confidence level on each bounding box. Save the annotated image as a test result map and generate a test report.

[0084] Step 3: Generate a test report Record all detected defect types and their quantities, calculate the confidence distribution of each defect, and draw a distribution histogram. Generate detailed inspection record documents, including: original image, inspection result map, defect list, confidence distribution map and threshold setting strategy description.

[0085] The foregoing embodiments have described the present invention in detail. Those skilled in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features therein; and these modifications or substitutions do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A method for detecting appearance defects of electric energy meters based on neural networks, characterized in that: Includes steps: S1. Obtaining the original image data of the appearance of the electric energy meter, and preprocessing and enhancing the original image data to obtain enhanced image data; S2. Construct a multi-layer convolutional neural network as a feature extractor to extract visual features at different levels from the enhanced image data, then perform feature fusion processing to obtain fused feature data, analyze the fused feature data, and realize the classification and positioning of appearance defects of the electric energy meter.

2. The method for detecting appearance defects of electric energy meters based on a neural network according to claim 1, characterized in that: The S1 specifically includes: An industrial camera for detecting defect details is used to comprehensively capture the original image data of the appearance of the electric energy meter. The original image data is preprocessed by denoising, background elimination and normalization. The preprocessed image data is enhanced using a multi-scale recursive perturbation and inverse correction image enhancement algorithm.

3. The method for detecting appearance defects of electric energy meters based on neural networks according to claim 2 is characterized in that: In S1, the multi-scale recursive perturbation and inverse correction image enhancement algorithm integrates multi-scale nonlinear transformation, recursive enhancement, multi-dimensional random perturbation fusion and inverse reconstruction.

4. The method for detecting appearance defects of electric energy meters based on a neural network according to claim 3 is characterized in that: In the process of implementing the multi-scale recursive perturbation and inverse correction image enhancement algorithm, in order to separate the different frequency components in the appearance image so that the high-frequency details and low-frequency structures can be processed independently, the pre-processed image data is decomposed at multiple scales; in order to amplify potential detail defects and suppress background noise, double nonlinear transformation is introduced on the basis of multi-scale decomposition, and nonlinear transformation is performed on the image of each scale; the multi-scale image after nonlinear transformation is recursively enhanced to eliminate noise interference by repeatedly extracting and enhancing the effective features in the image.

5. The method for detecting appearance defects of electric energy meters based on neural networks according to claim 4 is characterized in that: In the process of implementing the multi-scale recursive perturbation and inverse correction image enhancement algorithm, the recursively enhanced image may still contain background noise. Texture-structure decomposition is introduced to decompose the recursively enhanced image into a texture part and a structure part.

6. The method for detecting appearance defects of electric energy meters based on neural networks according to claim 5 is characterized in that: In the implementation process of the multi-scale recursive perturbation and inverse correction image enhancement algorithm, after obtaining the recursively enhanced and decomposed image, multi-dimensional random perturbation processing is introduced to simulate the image changes in different noise environments; in order to fuse the perturbation information at different scales, the perturbed images of all scales are weightedly fused to obtain the final fused image.

7. The method for detecting appearance defects of electric energy meters based on neural networks according to claim 6, characterized in that: In the process of implementing the multi-scale recursive perturbation and inverse correction image enhancement algorithm, in order to avoid the introduction of pseudo defects and noise in the fused image while containing rich detail information, an inverse perturbation correction mechanism is introduced to perform inverse operations on the fused image and the perturbation matrix to eliminate pseudo defects caused by excessive perturbation; after the image is obtained by inverse perturbation correction, in order to restore the global structure and detail information of the image, the image after inverse perturbation correction is reconstructed at multiple scales.

8. The method for detecting appearance defects of electric energy meters based on neural networks according to claim 1, characterized in that: In S2, the visual features at different levels are subjected to feature fusion processing using an adaptive interactive feature fusion algorithm. The adaptive interactive feature fusion algorithm processes the visual features at different levels through interactive mapping, adaptive weighting and feature fusion, and finally generates an efficient feature representation.

9. The method for detecting appearance defects of electric energy meters based on neural networks according to claim 8, characterized in that: In the adaptive interactive feature fusion algorithm, in order to realize hierarchical feature transformation, visual features at different levels are converted into feature tensors that can express richer semantic information, and linear transformation and nonlinear activation operations are performed on the visual features at each level; after the transformation feature tensor at each level is formed, the transformation feature tensor is subjected to multiple interactive mapping processing. The purpose of multiple interactive mapping is to bidirectionally interact feature tensors of different levels and resolutions so that each feature tensor can fuse the semantic information of features at other levels; after obtaining the interactive mapping matrix, different weights are assigned to the feature tensors at each level according to the information in the interactive mapping matrix, so as to adaptively fuse multi-level features.

10. A system for detecting appearance defects of electric energy meters based on a neural network, used to implement the method for detecting appearance defects of electric energy meters according to any one of claims 1 to 9, characterized in that: include: Image data enhancement processing unit: obtains original image data of the appearance of the electric energy meter, and performs preprocessing and enhancement processing on the original image data to obtain enhanced image data; Defect recognition unit: Construct a multi-layer convolutional neural network as a feature extractor to extract visual features at different levels from the enhanced image data, then perform feature fusion processing to obtain fused feature data, analyze the fused feature data, and realize the classification and positioning of appearance defects of the electric energy meter.

Images