Visual fault detection methods, systems, electronic devices, and storage media for smart meters

Abstract

This invention proposes a method, system, electronic device, and storage medium for smart meter fault detection, comprising: preprocessing smart meter image data to obtain an attention map corresponding to the smart meter image data; calculating the equivalent granularity based on the feature contour size of the attention map, and obtaining a discriminative granularity through cluster mining after traversing the smart meter image data, guiding each meter image in the smart meter image data to adaptively divide into multi-granularity mosaics; calculating the feature position distribution based on the binary image obtained from the attention map conversion, adaptively occluding the multi-granularity mosaics according to the feature position distribution rules, and randomly shuffling them to obtain multi-granularity masked obfuscated mosaics; using the multi-granularity masked obfuscated mosaics and the original image as input to the detection model to progressively train the smart meter visual fault detection model; and inputting the meter image data to be tested into the trained smart meter visual fault detection model to complete the fault category detection.

Description

[Technical Field]

[0001] This application belongs to the field of electricity metering technology, and more specifically, relates to a method, system, electronic device and storage medium for visual fault detection of smart meters. [Background Technology]

[0002] With the vigorous development of my country's economy and technology and the continuous increase in social demand, higher requirements have been placed on the safety and reliability of the power industry. Smart grids, characterized by informatization, automation, intelligence, and interactivity, are leading a new direction for the development of the power industry. The country is vigorously building a new power system, and the interconnection of things in the power system is taking shape. A new generation of smart meters is gradually emerging. Smart meters are one of the basic data acquisition devices for smart grids (especially smart distribution networks), undertaking the tasks of collecting, measuring, and transmitting raw power data. They are the foundation for information integration, analysis, optimization, and information display. The widespread adoption of smart meters is also driving the development of integrated businesses such as smart energy use and intelligent monitoring in the new energy vehicle, microgrid, and distributed power generation industries.

[0003] In new business scenarios and environments, higher demands are placed on the security and reliability of smart IoT meters. During meter operation, visible faults such as display screen malfunctions, damaged dials, burnt meters, and damaged seals are inevitable. These faults are typically only discovered and reported through regular inspections by maintenance personnel. However, this traditional method, which relies heavily on visual inspection and manual entry of visual fault information, is labor-intensive, inefficient, and prone to delays and inconsistencies due to the skill level and motivation of maintenance personnel. The explosive growth in the number of meters has significantly increased the difficulty of maintenance work, making it increasingly difficult for maintenance personnel to handle such high-intensity tasks. There is an urgent need to improve the intelligence and automation of maintenance models to ensure the stable operation of the power system and the smooth operation of electricity services.

[0004] Accurately identifying meter fault types using computer vision technology can effectively reduce the burden of existing operation and maintenance models and help improve the overall operating efficiency of the power system. Currently, most research uses template matching or manual feature extraction to detect visible meter faults in standard scenarios. However, the effectiveness of these methods relies on extensive parameter adjustments by experts in different standardized scenarios. In complex operation and maintenance scenarios, changes in the shooting environment and shooting angle significantly impact the recognition performance of traditional computer vision algorithms, resulting in poor robustness and insufficient intelligence. Deep learning models, on the other hand, can automatically mine the inherent patterns and deep representations of data from large amounts of images, solving the challenges of pattern recognition in complex image scenarios. When meters experience visible faults, the differences between different faults are small, exhibiting characteristics of small inter-class differences. For example, the difference between a meter with a blank screen and a normal meter is often only a local difference on the screen. While conventional image classification in deep learning tends to overlook detailed local features when learning global structures to mine high-level semantic features, fine-grained image recognition methods can focus on the local and detailed features of the target, making them more suitable for detecting visible meter faults.

[0005] Fine-grained image recognition methods are broadly categorized into two types: region localization and feature encoding. Region localization methods typically locate local regions of the target, extracting local features for processing and classification. These methods are often more suitable for structured objects, and predicting local feature bounding boxes under image-level supervision is computationally complex and lacks reliable metrics, resulting in poor applicability. Feature encoding methods, on the other hand, encode global features deeply, mining richer and more effective feature information for processing and classification, making them more direct and efficient. Therefore, how to deeply mine discriminative local features from structurally similar global features is a significant challenge in the field of fine-grained image recognition. Using multi-granularity mosaicking and masking can effectively mine multi-granularity features and reduce redundant information interference; however, unreasonable granularity settings can easily lead to incomplete or redundant features within mosaic pieces, and some occluded mosaic pieces may contain information that aids classification. [Summary of the Invention]

[0006] In view of this, the present invention provides a method, system, electronic device and storage medium for visual fault detection of smart meters, so as to improve the accuracy of visual fault detection of smart meters.

[0007] According to the first aspect of this application, a visual fault detection method for smart meters is provided, comprising the following steps:

[0008] Preprocess the smart meter image data to obtain the attention map corresponding to the smart meter image data;

[0009] The equivalent granularity is calculated based on the feature contour size of the attention map. After traversing the smart meter image data, the discriminative granularity is obtained through cluster mining, which guides each meter image in the smart meter image data to be adaptively divided into multi-granularity mosaics.

[0010] The feature position distribution is calculated based on the binary image obtained from the attention map transformation. The multi-granularity mosaic is adaptively occluded according to the feature position distribution pattern, and randomly shuffled to obtain a multi-granularity masked jigsaw puzzle.

[0011] The smart meter visual fault detection model is progressively trained by using multi-granularity mask obfuscation mosaic and the original meter image as input to the detection model, resulting in a trained smart meter visual fault detection model.

[0012] The image data of the meter under test is input into the trained smart meter visual fault detection model to complete the fault category detection.

[0013] The step of preprocessing the smart meter image data to obtain the attention map corresponding to the smart meter image data includes:

[0014] Each smart meter image in the image data is labeled with a fault category to form a training set;

[0015] The size of all samples in the training set is adjusted to R×R, where R is the pixel size of the image;

[0016] Gradient-weighted class activation mapping is calculated for each sample in the smart meter image data to obtain the attention map corresponding to the smart meter image data.

[0017] The step of calculating the gradient-weighted class activation mapping relationship for each sample in the smart meter image data to obtain the attention map corresponding to the smart meter image data includes:

[0018] All samples are input into a deep residual network (ResNet) to calculate the predicted class y of the current input sample. c , where c is the number of a certain category;

[0019] Extract the feature map output from the last feature extraction layer of the sample during the forward propagation of the deep residual network ResNet. k is the feature map The number of channels, i, j are the feature maps. The element indices in the row and column, i,j∈[0,z], where z is the feature map. Width and height dimensions;

[0020] During the backpropagation process of the deep residual network ResNet, the feature map of the current sample is calculated. Its predicted category y c gradient

[0021] gradient Perform global average pooling to obtain the importance value of the feature map of the sample in the k-th channel. The specific formula is as follows:

[0022]

[0023] By importance value By linearly combining the values ​​in different channels of the feature map, a preliminary representation of the attention map (AM) is obtained. pre The specific formula is as follows:

[0024]

[0025] The final representation AM of the attention map is obtained by linearly combining the nonlinear activation weights of the ReLU function.

[0026] The steps of calculating the equivalent granularity based on the feature contour size of the attention map, obtaining discriminative granularity through cluster mining after traversing the smart meter image data, and guiding each smart meter image in the smart meter image data to be adaptively divided into multi-granularity mosaics include:

[0027] The matrix size of the attention map for each sample in the image data of the smart meter is adjusted to be the same as the matrix size of the sample.

[0028] The attention map is normalized and converted into a grayscale matrix (Gray).

[0029] Binarize the grayscale matrix Gray to obtain the binary matrix Binary;

[0030] Calculate the minimum circumcircle cir of the feature contour reflected by the binary matrix Binary. l The diameter d of the circle is obtained. l Through the formula R / d l The equivalent granularity g of the feature is obtained. l , l represents the index of each smallest circumcircle;

[0031] Iterate through all samples to obtain the feature equivalent granularity group G. equivalent = [g1, ..., g m ], where m is the total number of feature contours calculated after traversing all samples;

[0032] In equivalent granularity group G equivalent Filtering out granularity g outside the boundary outlier The effective particle size group G is obtained.effect ;

[0033] For effective particle size group G effect Clustering is performed, and the discriminative granularity group G is obtained based on the cluster centers. discriminative = [g1, ..., g n ], where n represents the number of particle size categories included in the discriminative particle size group;

[0034] Each meter image in the smart meter image data is divided into g. o ×g o If there are multiple image blocks, o∈[1,n], then each meter image is divided into multiple granularity mosaics.

[0035] The equivalent granularity group G equivalent Filtering out granularity g outside the boundary outlier The steps include:

[0036] Calculate the equivalent granularity group G equivalent The upper quartile index q3 and the lower quartile index q1 are used to obtain the upper quartile Q3 and lower quartile Q1, and the interquartile range IQR is calculated. The equivalent granularity group G is determined based on Q3, Q1, and IQR. equivalent Upper bound B of outliers upper and the lower bound B lower The equivalent granularity group G equivalent Sort the granularity from smallest to largest, and group the equivalent granularity groups G. equivalent A certain particle size g l Its affiliation is determined by comparing it with the upper and lower bounds; the specific formula is as follows:

[0037]

[0038]

[0039]

[0040] IQR = Q3 - Q1

[0041]

[0042]

[0043]

[0044] round() is a rounding function, where m is the total number of feature contours calculated after traversing all samples.

[0045] The steps of calculating the feature position distribution based on the binary image obtained from the attention map transformation, adaptively occluding the auxiliary feature puzzle pieces in the multi-granularity puzzle according to the feature position distribution rules, and randomly shuffling them to obtain a multi-granularity masked obfuscated puzzle include:

[0046] The binary matrix Binary is obtained based on the attention map transformation.

[0047] Based on the binary matrix Binary, calculate the recession probability matrix M. fading ;

[0048] According to the recession probability matrix M fading The probability values ​​at different positions in the puzzle are used to occlude the multi-granularity puzzle pieces at the corresponding positions;

[0049] As the training cycle increases, the probability of occlusion is continuously reduced, and the positions of the multi-granularity puzzle pieces are randomly shuffled to obtain the multi-granularity mask obfuscation puzzle piece.

[0050] The decay probability matrix M is calculated based on the binary matrix Binary. fading The steps include:

[0051] The minEnclosingCircle algorithm in OpenCV is used to calculate the minimum circumcircle (cir) of the feature contour for each sample's binary matrix in the smart meter image data. l The center coordinates (x) l ,y l );

[0052] After traversing all samples of the smart meter image data, a series of feature contour center positions P = [(x1,y1),(x2,y2)...(x...] are obtained. m ,y m )], m is the total number of feature contours calculated after traversing all samples in the training set, which is consistent with the number of equivalent granularities;

[0053] Using the x and y coordinates as two dimensions, the hist2d library function in matplotlib.pyplot is used to perform two-dimensional histogram statistics on P, resulting in a two-dimensional histogram matrix M. hist Two-dimensional histogram matrix M hist The number of rows and columns are both set to the discriminative granularity group G. discriminative The granularity values ​​in the matrix are used to obtain multiple two-dimensional histogram matrices based on different granularity values;

[0054] For each two-dimensional histogram matrix M hist To perform max-min normalization, we obtain the normalized matrix M. normal ;

[0055] Take the normalized matrix M normal The mean of Ts is used as the probability threshold. probability The normalized matrix M normal The probability threshold Ts is greater than or equal to the probability threshold. probability Setting the elements to 1 yields the discriminative feature probability matrix M. discriminative ;

[0056] For the discriminative feature probability matrix M discriminative By inverting the probabilities, we obtain the auxiliary feature probability matrix M. complementary ;

[0057] Using the decay function F fading (x) Processing the auxiliary feature probability matrix M complementary The probability of auxiliary feature puzzle pieces being occluded is gradually reduced as the number of training rounds increases. The specific formula is as follows:

[0058]

[0059] Ts probability =mean(M normal )

[0060]

[0061] M complementary =1-M discriminative

[0062]

[0063] M fading =F fading (M complementary )

[0064] Where δ is the probability offset coefficient, β fading is the initial decay coefficient, and epoch is the current training epoch of the model.

[0065] The steps of progressively training the smart meter visual fault detection model by using multi-granularity masking obfuscation mosaic and the original meter image as input to the detection model to obtain the trained smart meter visual fault detection model include:

[0066] A smart meter visual fault detection model is constructed using the finest-grained puzzle input ResNet from the multi-granularity mask obfuscation puzzle.

[0067] The smart meter visual fault detection model is driven to prioritize fine-grained feature information in the mosaic, freeze the weight parameters of the last two feature extraction layers of the five feature extraction layers in ResNet, and let the first three feature extraction layers complete the feature information learning of the current granularity mosaic. After completion, the parameters of the three feature extraction layers are passed to the next stage through weight sharing, and the weight parameters of the next feature extraction layer are unfrozen and updated.

[0068] Input a coarser-grained mosaic from the multi-granularity mask obfuscation mosaic into the smart meter visual fault detection model, guide the smart meter visual fault detection model to gradually learn the coarse-grained feature information in the mosaic, unfreeze the weight parameter update of the last feature extraction layer, and pass the parameters of all previous feature extraction layers to the last stage.

[0069] In the final training phase, the original meter image is input into the smart meter visual fault detection model to guide the model to learn the global structure of the original meter image. The features output from the last three feature extraction layers of the original meter image in the deep residual network are fused into global features for classification to obtain the final predicted category.

[0070] The loss function of the smart meter visual fault detection model at different training stages adopts cross-entropy loss L. CEs :

[0071]

[0072] Where t is the true label distribution, y s Predict probability distributions for different feature learning stages, where s is the current feature learning stage, C is the total number of categories, and c is the category ID. Let y be the probability distribution s The element in the distribution t represents the predicted probability that the classifier will predict the input image as class c in this stage. When the true class of the image is c, the c-th element t in the true label distribution t is... c It is 1 if it is true, otherwise it is 0.

[0073] The step of inputting the image data of the meter under test into the trained smart meter visual fault detection model to complete the fault category detection includes:

[0074] The image data of the meter under test is input into the trained smart meter visual fault detection model. After feature extraction and classification at different granularity feature learning stages, the predicted probability distribution is obtained respectively.

[0075] In the final training phase, the features output from the last three feature extraction layers in the deep residual network are fused into global features for classification to obtain the predicted probability distribution. This yields the fault detection result for the meter image data under test. The predicted probability distribution is then represented by y. con The final prediction results are as follows:

[0076]

[0077] The argmax() function retrieves the index of the maximum element in a vector, which is the index of the predicted probability distribution y. s It contains the predicted probability for each category, and the index of the highest predicted probability is encoded as the final predicted category.

[0078] According to a second aspect of this application, a smart meter visual fault detection system is provided, including a data processing module, a model training module, and a recognition module;

[0079] The data processing module is used to preprocess smart meter image data to obtain an attention map corresponding to the smart meter image data.

[0080] The data processing module is also used to calculate the equivalent granularity based on the feature contour size of the attention map, and obtain the discriminative granularity by cluster mining after traversing the smart meter image data, so as to guide each meter image in the smart meter image data to be adaptively divided into multi-granularity mosaics.

[0081] The data processing module is further configured to calculate the feature position distribution based on the binary image obtained by the attention map conversion, and adaptively occlude the multi-granularity mosaic according to the feature position distribution pattern to obtain a multi-granularity masking and obfuscating mosaic.

[0082] The model training module is used to progressively train the smart meter visual fault detection model by using multi-granularity mask obfuscation mosaic and the original meter image as input to the detection model, so as to obtain a trained smart meter visual fault detection model.

[0083] The recognition module is used to input the image data of the meter under test into the trained smart meter visual fault detection model to complete the detection of fault categories.

[0084] According to a third aspect of this application, an electronic device is provided, including a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, characterized in that the processor executes the computer program to implement the steps of any one of the methods described in the first aspect.

[0085] According to a fourth aspect of this application, a storage medium is provided that stores a computer program thereon, characterized in that the computer program, when executed by a processor, implements the steps of the method described in any one of the first aspects above.

[0086] The present invention has the following beneficial effects:

[0087] In the technical solution of this invention, the granularity of the jigsaw puzzle is adaptively divided based on the contour size characteristics of the visible fault features of the electricity meter. This effectively avoids the problem of feature redundancy and incompleteness within the puzzle pieces due to improper granularity settings when dividing the puzzle into multiple granularities. Based on the decay probability function, the occlusion degree of auxiliary features is gradually reduced as the training rounds increase, effectively avoiding interference from redundant information in the early stages of training. Furthermore, auxiliary features are gradually added as classification criteria in the later stages of training. Using adaptively set multi-granularity puzzles and the original image as input data, combined with a progressive multi-granularity feature-guided learning training strategy, the robustness of the smart meter visible fault detection model is improved, thereby increasing the precision and recall of the smart meter visible fault detection results. [Attached Image Description]

[0088] Figure 1 This is a schematic diagram of the framework of the smart meter visual fault detection method provided by the present invention;

[0089] Figure 2 This is a schematic diagram illustrating the implementation process and subsequent processing of the adaptive multi-granularity jigsaw puzzle.

[0090] Figure 3 This refers to the calculation process of the attention map;

[0091] Figure 4 This is the training process of progressive multi-granularity feature-guided learning;

[0092] Figure 5 This is a schematic diagram of the smart meter visual fault detection system provided by the present invention;

[0093] Figure 6 This is a schematic diagram of the device structure of the electronic device provided by the present invention.

Detailed Implementation Methods

[0094] To better understand the technical solution of the present invention, the present invention will be described in detail below with reference to the accompanying drawings.

[0095] It should be understood that the described embodiments are merely some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0096] This application provides a method for visual fault detection of smart meters. To meet the requirements of visual fault detection in smart meters, this invention calculates the gradient-weighted class activation mapping of the meter image to obtain an attention map, which serves as the basis for calculating discriminative granularity and feature location distribution. Based on the feature contour size of the attention map, the equivalent granularity is calculated, and after traversing the dataset, clustering mining is performed to obtain the discriminative granularity, guiding the input image to adaptively divide into multi-granularity mosaics. The feature location distribution is calculated based on the binary image obtained from the attention map transformation, and the mosaic pieces are adaptively occluded according to the feature location distribution rules. As the number of training rounds increases, the occlusion degree of the mosaic pieces is gradually reduced. The multi-granularity mosaics and the original image are used as inputs to progressively train the smart meter visual fault detection model.

[0097] Figure 1 This is a schematic diagram of the framework of the smart meter visual fault detection method proposed in this invention. The method includes the following steps:

[0098] Step 101: Preprocess the smart meter image data to obtain the attention map corresponding to the smart meter image data, specifically as follows:

[0099] The smart meter image dataset is preprocessed, and each image is labeled with a fault category to form a training set; the size of all input samples in the training set is adjusted to R×R, where R is the pixel size of the image;

[0100] Then, all samples are input into a deep residual network ResNet; for each sample in the training set, ResNet is used to calculate the predicted class y of the current input sample. c , where c is the number of a certain category;

[0101] Extract the feature map of the last feature extraction layer output during the forward propagation of the ResNet deep residual network. k is the feature map The number of channels, i, j are the feature maps. The element indices in the row and column, i,j∈[0,z], where z is the feature map. Width and height dimensions;

[0102] During the backpropagation process of the deep residual network ResNet, the feature map of the current input sample is calculated. Its predicted category y c gradient

[0103] gradient Perform global pooling to obtain the importance value of the feature map of the sample in the k-th channel, and then use weights. express:

[0104]

[0105] Through this weight By linearly combining the values ​​in different channels of the feature map, a preliminary representation of the attention map (AM) is obtained. pre :

[0106]

[0107] The final attention map, AM, is obtained by linearly combining the nonlinear activation weights of the ReLU function.

[0108] Step 102: Calculate the equivalent granularity based on the feature contour size of the attention map. After traversing the smart meter image data, obtain the discriminative granularity through cluster mining. This guides each meter image in the smart meter image data to be adaptively divided into multi-granularity mosaics. Specifically:

[0109] Based on the attention map AM obtained in step 101, the discriminative granularity is calculated to guide the adaptive segmentation of the input image into multi-granularity mosaics:

[0110] The attention map AM for each sample can be viewed as a matrix of size z×z, and its size is adjusted to R×R to be the same as that of the input sample;

[0111] The attention map AM is normalized using the minimum-maximum normalization method and then converted into a gray matrix Gray.

[0112] Binarize the grayscale matrix Gray to obtain the binary matrix Binary;

[0113] The minEnclosingCircle algorithm in OpenCV is used to calculate the minimum circumcircle cir of the feature contour reflected by the binary matrix Binary. l The diameter d of the circle is obtained. l Through the formula R / d l The equivalent granularity g of the feature is obtained. l , l represents the index of each smallest circumcircle;

[0114] By traversing all samples in the training set, we obtain the feature equivalent granularity group G. equivalent = [g1, ..., g m ], where m is the total number of feature contours calculated after traversing all samples in the training set;

[0115] Calculate the equivalent granularity group G equivalent The upper quartile index q3 and the lower quartile index q1 are determined. `round()` is the rounding function. Based on the index values, the upper quartile Q3 and the lower quartile Q1 are obtained, and the interquartile range IQR is calculated. Based on these values, the equivalent granularity group G is determined. equivalentUpper bound B of outliers upper and the lower bound B lower The equivalent granularity group G equivalent Sort the granularity from smallest to largest, and group the equivalent granularity groups G. equivalent A certain particle size g l Its affiliation is determined by comparing it with the upper and lower bounds, and the calculation formula is as follows:

[0116]

[0117]

[0118]

[0119] IQR = Q3 - Q1

[0120]

[0121]

[0122]

[0123] In equivalent granularity group G equivalent Filtering out granularity g outside the boundary outlier The effective particle size group G is obtained. effect ;

[0124] For effective particle size group G effect Clustering is performed, and the discriminative granularity group G is obtained based on the cluster centers. discriminative = [g1, ..., g n ], where n represents the number of particle size categories included in the discriminative particle size group;

[0125] Each meter image in the training set is divided into g. o ×g o If there are multiple image blocks, o∈[1,n], then each image is divided into a jigsaw puzzle of multiple granularities.

[0126] Step 103: Calculate the feature position distribution based on the binary image obtained from the attention map transformation; adaptively occlude the multi-granularity mosaic according to the feature position distribution pattern; and randomly shuffle it to obtain a multi-granularity masked obfuscated mosaic. Specifically:

[0127] Based on the binary matrix Binary obtained in step 102, the decay probability matrix is ​​calculated to guide the adaptive decay occlusion of the jigsaw puzzle pieces:

[0128] The minEnclosingCircle algorithm in OpenCV is used to calculate the minimum circumcircle (cir) of the feature contour for each sample in the training set using the binary matrix (Binary). l The center coordinates (x)l ,y l );

[0129] After traversing all samples in the training set, a series of feature contour center positions are obtained: P = [(x1, y1), (x2, y2)...(x...). m ,y m )], m is the total number of feature contours calculated after traversing all samples in the training set, which is consistent with the number of equivalent granularities;

[0130] Using the x and y coordinates as two dimensions, the hist2d library function in matplotlib.pyplot is used to perform two-dimensional histogram statistics on P, resulting in a two-dimensional histogram matrix M. hist Two-dimensional histogram matrix M hist The number of rows and columns are both set to the discriminative granularity group G. discriminative The granularity value in the matrix allows for the generation of multiple two-dimensional histogram matrices based on different granularity values.

[0131] For each two-dimensional histogram matrix M hist To perform max-min normalization, we obtain the normalized matrix M. normal ;

[0132] Take the normalized matrix M normal The mean of Ts is used as the probability threshold. probability The normalized matrix M normal The probability threshold Ts is greater than or equal to the middle probability Setting the elements to 1 yields the discriminative feature probability matrix M. discriminative ;

[0133] For the discriminative feature probability matrix M discriminative By inverting the probabilities, we obtain the auxiliary feature probability matrix M. complementary ;

[0134] Using the decay function F fading (x) is used to process the auxiliary feature probability matrix M. complementary The probability of auxiliary feature puzzle pieces being occluded is gradually reduced as the number of training rounds increases, and the decay probability matrix M... fading The calculation formula is as follows:

[0135]

[0136] Ts probability =mean(M normal )

[0137]

[0138] M complementary =1-M discriminative

[0139]

[0140] M fading =F fading (M complementary )

[0141] Where δ is the probability offset coefficient, β fading The initial decay coefficient is denoted as epoch, which is the current training epoch of the model.

[0142] Each sample in the training set will be divided into multi-granularity mosaics, which will be adaptively masked by a decay mask. For a given sample, a mosaic at a specific granularity will be masked according to the decay probability matrix M. fading The probability value at different positions in the puzzle is used to occlude the corresponding puzzle pieces;

[0143] As the training cycle increases, the probability of occlusion is continuously reduced, and the positions of the multi-granularity puzzle pieces are randomly shuffled to obtain the multi-granularity mask obfuscation puzzle piece.

[0144] Step 104: Using multi-granularity masking to obfuscate the mosaic and the original meter image as input to the detection model, progressively train the smart meter visual fault detection model to obtain the trained smart meter visual fault detection model, specifically as follows:

[0145] In step 103, a multi-granularity mask obfuscation mosaic is obtained, which, along with the original image, serves as input data for the detection model. The training process is as follows:

[0146] A smart meter visual fault detection model is constructed using the finest-grained puzzle input ResNet from the multi-granularity mask obfuscation puzzle.

[0147] The smart meter visual fault detection model is driven to prioritize fine-grained feature information in the mosaic, freeze the weight parameters of the last two feature extraction layers of the five feature extraction layers in ResNet, and let the first three feature extraction layers complete the feature information learning of the current granularity mosaic. After completion, the parameters of the three feature extraction layers are passed to the next stage through weight sharing, and the weight parameters of the next feature extraction layer are unfrozen and updated.

[0148] Input a coarser-grained mosaic from the multi-granularity mask obfuscation mosaic into the smart meter visual fault detection model, guide the smart meter visual fault detection model to gradually learn the coarse-grained feature information in the mosaic, unfreeze the weight parameter update of the last feature extraction layer, and pass the parameters of all previous feature extraction layers to the last stage.

[0149] In the final training phase, the original meter image is input into the smart meter visual fault detection model to guide the model to learn the global structure of the original meter image. The features output from the last three feature extraction layers of the original meter image in the deep residual network are fused into global features for classification to obtain the final predicted category.

[0150] The loss function of the smart meter visual fault detection model at different training stages adopts cross-entropy loss L. CEs :

[0151]

[0152] Where t is the true label distribution, y s Predict probability distributions for different feature learning stages, where s is the current feature learning stage, C is the total number of categories, and c is the category ID. Let y be the probability distribution s The element in the distribution t represents the predicted probability that the classifier will predict the input image as class c in this stage. When the true class of the image is c, the c-th element t in the true label distribution t is... c It is 1 if it is true, otherwise it is 0.

[0153] Step 105: After completing steps 101 to 104, the training of the smart meter visual fault detection model is finished. The image data of the meter under test is then input into the trained smart meter visual fault detection model to complete the fault category detection. Specifically:

[0154] The image data of the meter under test is input into the trained smart meter visual fault detection model. After feature extraction and classification at different granularity feature learning stages, the predicted probability distribution is obtained respectively.

[0155] In the final training phase, the features output from the last three feature extraction layers in the deep residual network are fused into global features for classification to obtain the predicted probability distribution. This yields the fault detection result for the meter image data under test. The predicted probability distribution is then represented by y. con The final prediction results are as follows:

[0156]

[0157] The argmax() function retrieves the index of the maximum element in a vector, which is the index of the predicted probability distribution y. s It contains the predicted probability for each category, and the index of the highest predicted probability is encoded as the final predicted category. Figure 2This diagram illustrates the implementation process and subsequent processing of adaptive multi-granularity jigsaw puzzles. In the first stage, the gradient-weighted class activation mapping of the training set images is calculated to obtain the attention map of smart meter fault features. Based on the feature contours reflected in the attention map, an equivalent granularity transformation is performed. After traversing all equivalent granularities in the training set, clustering is performed to obtain a set of values ​​that can reflect the feature size of the dataset, called the discriminative granularity. Simultaneously, using the attention map, the center coordinates of the effective features are calculated. The distribution pattern of the effective features in the training set is obtained, and the feature position is determined and a mask is generated based on the feature distribution pattern. In the second stage, the image is divided into jigsaw puzzles of different granularities according to the discriminative granularity. Based on the feature distribution pattern, the jigsaw puzzle pieces are roughly divided into discriminative feature blocks and auxiliary feature blocks. In the first stage, auxiliary feature blocks are occluded according to their decay probability. In the early stage of training, redundant information is filtered out to highlight the target subject. In the later stage of training, the decay probability gradually decreases, and the probability of occlusion of auxiliary feature blocks decreases. The model incorporates the learning of auxiliary features while fully learning the discriminative features. The puzzle pieces are then obfuscated to complete the final transformation of the input image, thereby destroying the spatial semantic information of the boundaries and forcing the model to learn the corresponding granular feature information within the puzzle pieces. In the third stage, the original image and the transformed multi-granularity masked puzzle are fed into a progressive multi-granularity feature-guided learning network. The fine-grained texture features of the image are used to guide the network to learn the high-level semantic information of coarse-grained features. Finally, the multi-granularity feature information is combined to complete the classification.

[0158] Figure 3 This is a schematic diagram of the computation process for the attention map of an electricity meter image. The input image size is adjusted to R×R and input into a deep residual network (ResNet). The feature map output by the last residual module S5 during the forward propagation of the ResNet is extracted. During the backward propagation of the ResNet, the gradient of the predicted class for each pixel on the feature map is calculated. Global pooling is performed on the gradients to obtain the importance value of the feature map of class c on the k-th channel, using weight α. k c The initial representation of the attention map is obtained by linearly combining the values ​​on different channels of the feature map using the weights; the final representation of the attention map is obtained by linearly combining the nonlinear activation weights to eliminate the influence of negative values ​​generated by other classes on the attention map.

[0159] Figure 4This is a progressive multi-granularity feature-guided learning training process. The training of the fault detection model is divided into multiple stages. First, the finest-grained mosaic is input into the ResNet-constructed smart meter visual fault detection model, driving the model to prioritize feature information in the fine-grained mosaic. Parameters are passed to the next stage of model training through weight sharing, and the parameter updates of the next feature extraction layer are unfrozen. Then, a coarser-grained mosaic is input into the detection model, guiding it to gradually learn the coarser-grained feature information in that mosaic. This process continues until the final stage, where the original image is input, guiding the model to learn the global structure of the original image. Different classifiers are used for training at different granularity feature learning stages, independently optimizing the model to have the corresponding granularity of feature learning capability. Finally, the features from the last three stages are fused into global features for classification.

[0160] In a specific embodiment, visual fault datasets under different categories of smart meters were used for testing. The datasets collected a total of 5 types of smart meter image data. Due to the influence of human and environmental factors during the collection process, there were cases of missing features and incorrect labels in the collected images. If used directly, the model would have difficulty learning the correct features. After data cleaning, a total of 720 fault sample data were obtained. The dataset was divided into training and test sets in a 6:4 ratio.

[0161] Table 1 shows the datasets used in the specific embodiments.

[0162]

[0163] To verify the effectiveness of the proposed algorithm, four mainstream fine-grained image classification methods were compared in this embodiment of the invention; as shown in Table 2, the embodiments of the invention are represented in the table as AMGJ-FGIC (Adaptive multi-granularityjigsaw for fine-grained image classification).

[0164] Table 2 shows the algorithms compared in specific embodiments.

[0165]

[0166] In this embodiment of the invention, Recall and Precision metrics are used to evaluate the classification performance of the algorithm. Their values ​​range from 0 to 1, and the larger the value, the better the classification performance of the model.

[0167] Table 3 shows a comparison of the experimental results of the present invention's embodiments and mainstream fine-grained recognition methods in terms of recall for various fault categories in smart meters, while Table 4 shows a comparison of the experimental results in terms of precision for each category. It can be seen that the smart meter visual fault detection method based on adaptive multi-granularity mosaicking of the present invention outperforms other methods in both recall and precision across all categories, achieving the best classification performance. Combining the results in Tables 3 and 4, it is evident that the smart meter visual fault detection method of the present invention performs better than mainstream fine-grained image classification methods, achieving higher classification precision and recall.

[0168] Table 3. Experimental results of AMGJ-FGIC and mainstream methods on the Recall index under various fault categories in smart meters.

[0169]

[0170] Table 4. Experimental results of AMGJ-FGIC and mainstream methods on the Precision index under various fault categories in smart meters.

[0171]

[0172] Experimental comparisons with mainstream fine-grained image classification methods show that the proposed smart meter visual fault detection method, based on adaptive multi-granularity mosaicking, adaptively sets the granularity of mosaic partitioning, which to some extent avoids the problems of incomplete features and feature redundancy within mosaic pieces. Furthermore, by utilizing fading masks to reduce interference from redundant information in the early stages of training and gradually using auxiliary features in conjunction with discriminative features for decision-making in the later stages, the method can effectively improve the accuracy and recall of smart meter visual fault detection.

[0173] This application provides a visual fault detection system for smart meters, such as... Figure 5 As shown, it includes a data processing module 501, a model training module 502, and a recognition module 503;

[0174] The data processing module 501 is used to: preprocess smart meter image data, perform gradient weighted class activation mapping calculation on each sample in the training set to obtain its attention map; calculate the equivalent granularity based on the feature contour size of the attention map of each sample, and obtain the discriminative granularity by clustering mining after traversing the dataset.

[0175] The model training module 502 is used to: calculate the feature position distribution based on the binary image obtained by converting the attention map of the training set samples; adaptively occlude the puzzle pieces according to the feature position distribution rules; gradually reduce the occlusion degree of the puzzle pieces as the number of training rounds increases; and progressively train the smart meter image classification model by using multi-granularity masking to confuse the puzzle pieces and the original image as input to the classification model.

[0176] The recognition module 503 is used to: input the image to be tested into the trained meter visual fault model to complete the fault category detection.

[0177] This application provides an electronic device, such as... Figure 6 As shown, the electronic device includes a bus, a processor, a memory, and a communication interface. It may also include input / output interfaces and a display device. The various functional units can communicate with each other via the bus. The memory stores computer programs, and the processor executes the programs stored in the memory, performing the meter fault classification method described in the above embodiment.

[0178] This application provides a storage medium storing a computer program, which, when executed by a processor, implements the steps of the method for classifying electricity meter faults. Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented in hardware or with the aid of software and necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, or external hard drive), including several instructions to cause a computer device (such as a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0179] In summary, the embodiments of the present invention have the following beneficial effects:

[0180] In the technical solution of this invention, based on the construction method of adaptive multi-granularity jigsaw puzzle, the discriminative granularity of the jigsaw puzzle is adaptively divided according to the contour size characteristics of the visible fault features of the electricity meter. This effectively avoids the problem of feature redundancy and incomplete features within the jigsaw puzzle pieces due to improper granularity setting when dividing the multi-granularity jigsaw puzzle. According to the decay probability function, the degree of occlusion of auxiliary features is gradually reduced as the training rounds increase, effectively avoiding interference from redundant information in the early stage of training. Furthermore, auxiliary features are gradually added as classification criteria in the later stage of training. Using the adaptively set multi-granularity jigsaw puzzle and the original image as input data, combined with the progressive multi-granularity feature-guided learning training strategy, it is beneficial to improve the robustness of the smart meter visible fault detection model, thereby improving the precision and recall of the smart meter visible fault detection results.

[0181] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A visual fault detection method for smart meters, characterized in that, Includes the following steps: Preprocess the smart meter image data to obtain the attention map corresponding to the smart meter image data; The equivalent granularity is calculated based on the feature contour size of the attention map. After traversing the smart meter image data, a discriminative granularity is obtained through cluster mining. This guides each meter image in the smart meter image data to be adaptively divided into multi-granularity mosaics. Specifically: The matrix size of the attention map for each sample in the image data of the smart meter is adjusted to be the same as the matrix size of the sample. The attention map is normalized and converted into a grayscale matrix (Gray). Binarize the grayscale matrix Gray to obtain the binary matrix Binary; Calculate the minimum circumcircle cir of the feature contour reflected by the binary matrix Binary. l The diameter d of the circle is obtained. l Through the formula R / d l The equivalent granularity g of the feature is obtained. l , l represents the index of each smallest circumcircle; Iterate through all samples to obtain the feature equivalent granularity group G. equivalent = [g1, ..., g m ], where m is the total number of feature contours calculated after traversing all samples; In equivalent granularity group G equivalent Filtering out granularity g outside the boundary outlier The effective particle size group G is obtained. effect ; For effective particle size group G effect Clustering is performed, and the discriminative granularity group G is obtained based on the cluster centers. discriminative = [g1, ..., g n ], where n represents the number of particle size categories included in the discriminative particle size group; Each meter image in the smart meter image data is divided into g. o ×g o If there are multiple image blocks, o∈[1,n], then each meter image is divided into multiple granularity mosaics; The feature position distribution is calculated based on the binary image obtained from the attention map transformation. The multi-granularity mosaic is adaptively occluded according to the feature position distribution pattern, and randomly shuffled to obtain a multi-granularity masked jigsaw puzzle. The smart meter visual fault detection model is progressively trained by using multi-granularity mask obfuscation mosaic and the original meter image as input to the detection model, resulting in a trained smart meter visual fault detection model. The image data of the meter under test is input into the trained smart meter visual fault detection model to complete the fault category detection.

2. The smart meter visual fault detection method according to claim 1, characterized in that, The step of preprocessing the smart meter image data to obtain the attention map corresponding to the smart meter image data includes: Each smart meter image in the image data is labeled with a fault category to form a training set; The size of all samples in the training set is adjusted to R×R, where R is the pixel size of the image; Gradient-weighted class activation mapping is calculated for each sample in the smart meter image data to obtain the attention map corresponding to the smart meter image data.

3. The visual fault detection method for smart meters according to claim 2, characterized in that, The step of calculating the gradient-weighted class activation mapping relationship for each sample in the smart meter image data to obtain the attention map corresponding to the smart meter image data includes: All samples are input into a deep residual network (ResNet) to calculate the predicted class y of the current input sample. c , where c is the number of a certain category; Extract the feature map output from the last feature extraction layer of the sample during the forward propagation of the deep residual network ResNet. k is the feature map The number of channels, i, j are the feature maps. The element indices in the row and column, i,j∈[0,z], where z is the feature map. Width and height dimensions; During the backpropagation process of the deep residual network ResNet, the feature map of the current sample is calculated. Its predicted category y c gradient gradient Perform global average pooling to obtain the importance value of the feature map of the sample in the k-th channel. The specific formula is as follows: By importance value By linearly combining the values ​​in different channels of the feature map, a preliminary representation of the attention map (AM) is obtained. pre The specific formula is as follows: The final representation AM of the attention map is obtained by linearly combining the nonlinear activation weights of the ReLU function.

4. The smart meter visual fault detection method according to claim 1, characterized in that, The equivalent granularity group G equivalent Filtering out granularity g outside the boundary outlier The steps include: Calculate the equivalent granularity group G equivalent The upper quartile index q3 and the lower quartile index q1 are used to obtain the upper quartile Q3 and lower quartile Q1, and the interquartile range IQR is calculated. The equivalent granularity group G is determined based on Q3, Q1, and IQR. equivalent Upper bound B of outliers upper and the lower bound B lower The equivalent granularity group G equivalent Sort the granularity from smallest to largest, and group the equivalent granularity groups G. equivalent A certain particle size g l Its affiliation is determined by comparing it with the upper and lower bounds; the specific formula is as follows: IQR = Q3 - Q1 round() is a rounding function, where m is the total number of feature contours calculated after traversing all samples.

5. The visual fault detection method for smart meters according to claim 1, characterized in that, The steps of calculating the feature position distribution based on the binary image obtained from the attention map transformation, adaptively occluding the multi-granularity mosaic according to the feature position distribution rules, and randomly shuffling it to obtain a multi-granularity masked obfuscated mosaic include: The binary matrix Binary is obtained based on the attention map transformation. Based on the binary matrix Binary, calculate the recession probability matrix M. fading ; According to the recession probability matrix M fading The probability values ​​at different positions in the puzzle are used to occlude the multi-granularity puzzle pieces at the corresponding positions; As the training cycle increases, the probability of occlusion is continuously reduced, and the positions of the multi-granularity puzzle pieces are randomly shuffled to obtain the multi-granularity mask obfuscation puzzle piece.

6. The smart meter visual fault detection method according to claim 5, characterized in that, The decay probability matrix M is calculated based on the binary matrix Binary. fading The steps include: The minEnclosingCircle algorithm in OpenCV is used to calculate the minimum circumcircle (cir) of the feature contour for each sample's binary matrix in the smart meter image data. l The center coordinates (x) l ,y l ); After traversing all samples of the smart meter image data for training, a series of feature contour center positions P = [(x1,y1),(x2,y2)...(x...] are obtained. m ,y m )], m is the total number of feature contours calculated after traversing all samples in the training set, which is consistent with the number of equivalent granularities; Using the x and y coordinates as two dimensions, the hist2d library function in matplotlib.pyplot is used to perform two-dimensional histogram statistics on P, resulting in a two-dimensional histogram matrix M. hist Two-dimensional histogram matrix M hist The number of rows and columns are both set to the discriminative granularity group G. discriminative The granularity values ​​in the matrix are used to obtain multiple two-dimensional histogram matrices based on different granularity values; For each two-dimensional histogram matrix M hist To perform max-min normalization, we obtain the normalized matrix M. normal ; Take the normalized matrix M normal The mean of Ts is used as the probability threshold. probability The normalized matrix M normal The probability threshold Ts is greater than or equal to the probability threshold. probability Setting the elements to 1 yields the discriminative feature probability matrix M. discriminative ; For the discriminative feature probability matrix M discriminative By inverting the probabilities, we obtain the auxiliary feature probability matrix M. complementary ; Using the decay function F fading (x) Processing the auxiliary feature probability matrix M complementary The probability of auxiliary feature puzzle pieces being occluded is gradually reduced as the number of training rounds increases. The specific formula is as follows: Ts probability =mean(M normal ) M complementary =1-M discriminative M fading =F fading (M complementary ) Where δ is the probability offset coefficient, β fading is the initial decay coefficient, and epoch is the current training epoch of the model.

7. The visual fault detection method for smart meters according to claim 1, characterized in that, The steps of progressively training the smart meter visual fault detection model by using multi-granularity masking obfuscation mosaic and the original meter image as input to the detection model to obtain the trained smart meter visual fault detection model include: A smart meter visual fault detection model is constructed using the finest-grained puzzle input ResNet from the multi-granularity mask obfuscation puzzle. The smart meter visual fault detection model is driven to prioritize fine-grained feature information in the mosaic, freeze the weight parameters of the last two feature extraction layers of the five feature extraction layers in ResNet, and let the first three feature extraction layers complete the feature information learning of the current granularity mosaic. After completion, the parameters of the three feature extraction layers are passed to the next stage through weight sharing, and the weight parameters of the next feature extraction layer are unfrozen and updated. Input a coarser-grained mosaic from the multi-granularity mask obfuscation mosaic into the smart meter visual fault detection model, guide the smart meter visual fault detection model to gradually learn the coarse-grained feature information in the mosaic, unfreeze the weight parameter update of the last feature extraction layer, and pass the parameters of all previous feature extraction layers to the last stage. In the final training phase, the original meter image is input into the smart meter visual fault detection model to guide the model to learn the global structure of the original meter image. The features output from the last three feature extraction layers of the original meter image in the deep residual network are fused into global features for classification to obtain the final predicted category.

8. The method for visual fault detection of smart meters according to claim 7, characterized in that, The loss function of the smart meter visual fault detection model at different training stages adopts cross-entropy loss L. CEs : Where t is the true label distribution, y s Predict probability distributions for different feature learning stages, where s is the current feature learning stage, C is the total number of categories, and c is the category ID. Let y be the probability distribution s The element in the distribution t represents the predicted probability that the classifier will predict the input image as class c in this stage. When the true class of the image is c, the c-th element t in the true label distribution t is... c It is 1 if it is true, otherwise it is 0.

9. The visual fault detection method for smart meters according to claim 1, characterized in that, The step of inputting the image data of the meter under test into the trained smart meter visual fault detection model to complete the fault category detection includes: The image data of the meter under test is input into the trained smart meter visual fault detection model. After feature extraction and classification at different granularity feature learning stages, the predicted probability distribution is obtained respectively. In the final training phase, the features output from the last three feature extraction layers in the deep residual network are fused into global features for classification to obtain the predicted probability distribution. This yields the fault detection result for the meter image data under test. The predicted probability distribution is then represented by y. con The final prediction results are as follows: The argmax() function retrieves the index of the maximum element in a vector, which is the index of the predicted probability distribution y. s It contains the predicted probability for each category, and the index of the highest predicted probability is encoded as the final predicted category.

10. A smart meter visual fault detection system implementing the method as described in claim 1, characterized in that, It includes a data processing module, a model training module, and a recognition module; The data processing module is used to preprocess smart meter image data to obtain an attention map corresponding to the smart meter image data. The data processing module is also used to calculate the equivalent granularity based on the feature contour size of the attention map, and obtain the discriminative granularity by cluster mining after traversing the smart meter image data, so as to guide each meter image in the smart meter image data to be adaptively divided into multi-granularity mosaics. The data processing module is further configured to calculate the feature position distribution based on the binary image obtained by the attention map conversion, and adaptively occlude the multi-granularity mosaic according to the feature position distribution pattern to obtain a multi-granularity masking and obfuscating mosaic. The model training module is used to progressively train the smart meter visual fault detection model by using multi-granularity mask obfuscation mosaic and the original meter image as input to the detection model, so as to obtain a trained smart meter visual fault detection model. The recognition module is used to input the image data of the meter under test into the trained smart meter visual fault detection model to complete the detection of fault categories.

11. An electronic device, comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 10.

12. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 10.

Images