Substation meter dial defect detection method based on multi-label classification

Abstract

The invention relates to the technical field of computer vision, and provides a transformer substation meter dial defect detection method based on multi-label classification. The problems of labeling redundancy, high omission ratio, inter-class confusion and low sample utilization rate in the prior art are solved. According to the main scheme, the method comprises the steps that a sample data set containing normal dial plates and defect dial plates is established, each dial plate is marked with a single rectangular frame, and defect attributes are stored in an attribute list; training a dial plate detection model based on a YOLOv8m architecture, and optimizing model parameters by using all samples in the sample data set; a multi-label classification model based on residual attention is constructed, the attention score of each category is calculated, and category specific residual attention features are generated; outputting a multi-label prediction result through the classifier; training the multi-label classification model to obtain a dial defect classification model; and integrating the dial plate detection model and the dial plate defect classification model, and outputting a defect type by the classification model after the dial plate is positioned by the detection model.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision and provides a method for detecting defects in substation meter dials based on multi-label classification. Background Art

[0002] With the advancement of smart grid construction, the demand for intelligent inspection of substation equipment is becoming increasingly urgent. As a key component of substation equipment status monitoring, meter dial defect detection (including blurry, damaged, and dirty) directly impacts the accuracy of equipment operating status assessments. Traditional manual visual inspection methods suffer from low efficiency and high subjectivity, while automated inspection technologies based on computer vision are becoming a mainstream solution.

[0003] Early meter defect detection relied primarily on traditional image processing techniques (such as edge detection and morphological operations) combined with machine learning methods that used hand-crafted features (such as HOG and LBP). While these methods can perform basic detection in simple scenarios, their feature representation capabilities are limited, and they are less adaptable to complex backgrounds, varying lighting conditions, and multiple types of defects. In recent years, deep learning technology has significantly improved the generalization capabilities of image recognition through end-to-end feature learning mechanisms. Object detection algorithms based on convolutional neural networks (CNNs) (such as Faster R-CNN and the YOLO series) and Transformer-based detection frameworks (such as DETR) have been widely used in the field of industrial defect detection.

[0004] In existing technical solutions, meter dial defect detection usually adopts a one-stage target detection architecture. Its technical process includes the following steps: first, the collected dial image is annotated. For the various defects that may exist on the same dial (such as blur, damage, and dirt), multiple rectangular boxes need to be annotated separately and bound to the corresponding defect type labels; then, the annotated data is used to train the target detection model, and multiple prediction boxes of the same dial are fused through post-processing algorithms such as non-maximum suppression (NMS), and finally the defect detection results are output.

[0005] However, the existing technology has the following significant defects:

[0006] Labeling redundancy: The need to label multiple defects on the same dial with separate rectangular boxes results in an exponential increase in labeling workload. Repeated labeling of the same dial area creates data redundancy, significantly increasing labeling costs.

[0007] Insufficient data utilization: Existing methods typically train only on defective samples, whereas normal watch faces predominate in real-world scenarios. This underutilization of normal samples results in insufficient model discrimination for normal watch faces, leading to missed or false detections.

[0008] Weak fine-grained recognition capabilities: Dial defect types (such as blur and dirt) vary widely within a class but only slightly between them. General-purpose object detection models lack the ability to model fine-grained features, resulting in a high rate of defect type confusion. For example, the edge features of a broken defect can easily misidentify with the texture features of a dirty area, affecting detection accuracy.

[0009] Furthermore, the existing single-stage detection architecture couples meter dial location and defect classification within the same model, causing the two tasks to interfere with each other during training and making it difficult to simultaneously achieve optimal performance. These issues severely restrict the practical application of meter dial defect detection technology in intelligent substation inspections.

[0010] In response to the above technical bottlenecks, there is an urgent need for an innovative method that can reduce labeling costs, make full use of sample data and improve fine-grained recognition capabilities to promote the further development of substation equipment defect detection technology. Summary of the Invention

[0011] The present invention proposes a two-stage detection method based on multi-label classification and residual attention mechanism. By separating the dial detection and defect classification tasks, combined with multi-label attribute annotation optimization and fine-grained recognition model, it solves the problems of redundant annotation, high missed detection rate, inter-class confusion and low sample utilization in the existing technology.

[0012] In order to achieve the above-mentioned purpose, the present invention adopts the following technical means:

[0013] The present invention provides a method for detecting defects in substation meter dials based on multi-label classification, comprising the following steps:

[0014] Step S1: Construct a sample dataset containing normal dials and defective dials, mark each dial with a single rectangular box and store the defect attributes in an attribute list;

[0015] Step S2: training a dial detection model based on the YOLOv8m architecture, and optimizing model parameters using all samples in the sample dataset;

[0016] Step s3: Build a multi-label classification model based on residual attention, including:

[0017] The backbone network extracts dial image features and generates original feature vectors;

[0018] Based on the residual attention mechanism of multiple attention heads, the attention score of each category is calculated to generate category-specific residual attention features;

[0019] Output multi-label prediction results through the classifier;

[0020] Step 4: training the multi-label classification model to obtain a dial defect classification model;

[0021] Step 5: Integrate the dial detection model and the dial defect classification model. After the dial is located by the detection model, the classification model outputs the defect type.

[0022] In the above scheme, the labeling process in step 1 includes:

[0023] Mark the meter image after deduplication with a single rectangular frame and record the position and size of the dial;

[0024] The defect attributes are associated with the attribute list of the rectangular box in the form of multiple labels, allowing the same dial to be labeled with multiple defect attributes.

[0025] In the above scheme, the object detection model described in step 2 uses the YOLOv8m architecture, and the training hyperparameters include: input image size 960×960, learning rate 0.001, training rounds 300, and test set mAP 50 The highest model.

[0026] In the above scheme, the implementation of the residual attention mechanism described in step 3 includes:

[0027] Calculate the attention score of category i at spatial position j The formula is:

[0028]

[0029] Where K is the temperature, x j is the eigenvector, m i is the classifier parameter of category i;

[0030] Generate residual attention feature f for category i i =g+λa i , where g is the global average feature, a i is the weighted feature, and λ is a hyperparameter.

[0031] In the above scheme, the multi-attention head includes 4 independent single attention heads, and the temperature values ​​of each head are set to K1=1, K2=2, K3=3, K4=∞, and the final original prediction value of all categories is for:

[0032]

[0033] Where T h is the temperature value of the h-th single attention classification head, and the final prediction score vector s∈R for all categories 4 for:

[0034]

[0035] The first element of s corresponds to "blurred", the second element corresponds to "damaged", the third element corresponds to "dirty", and the fourth element corresponds to "normal". The categories corresponding to the elements in s that are greater than the threshold of 0.5 are used as the recognition results of dial defects.

[0036] In the above solution, step 4 specifically includes:

[0037] Step 4.1: Convert the dial defect detection dataset into a classification dataset in Labelme format;

[0038] Step 4.2: Train the multi-label classification model in PyTorch using the binary cross entropy loss function. The training hyperparameters include image size 224 pixels, 60 training epochs, batch size 48, and initial learning rate 0.004.

[0039] Step 4.3: Use the LinearLR learning rate scheduler and SGD optimizer to adjust the training process;

[0040] Step 4.4: After each training round, calculate the overall fl-score based on the test set as the evaluation metric, and select the model with the highest fl-score as the final dial defect classification model.

[0041] In the above solution, the integration described in step 5 includes:

[0042] Step 5.1: Process the input original image through the dial detection model and output the center point position (x, y) and size (w, h) of the dial;

[0043] Step 5.2: Based on the center point position and size, the dial is cropped from the original image to obtain a dial image. The cropping formula is:

[0044]

[0045] Step 5.3: Input the dial image into the dial defect classification model, and output a prediction result of the dial defect, wherein the prediction result includes one or more combinations of blurry, damaged, dirty, or normal.

[0046] The technical solution of this invention achieves the following significant advantages in meter dial defect detection through the innovative design of multi-label classification and two-stage detection architecture:

[0047] 1. Improved fine-grained recognition capabilities

[0048] By building a multi-label classification model based on residual attention and introducing a category-specific attention mechanism and a multi-attention head fusion strategy, the model's ability to capture fine-grained features of defects such as blurry, damaged, and dirty dials is enhanced. By balancing global and local features through residual connections, the risk of confusion between similar defect categories is effectively reduced, significantly improving classification accuracy.

[0049] 2. Model training efficiency and performance optimization

[0050] Decomposing the inspection task into two independent models for dial positioning and defect classification resolves the optimization conflicts caused by the coupling of detection and classification tasks in traditional single-stage models. The two-stage model allows for targeted training for positioning accuracy and classification capabilities, reducing model training complexity while achieving superior performance for both tasks.

[0051] 3. Enhanced data utilization efficiency

[0052] During the dial detection phase, full data training, including normal samples, is fully utilized. By expanding the sample coverage, the model's generalization ability for dial position and size is improved, reducing the missed detection rate. During the defect classification phase, multi-label attribute annotation is used to avoid redundant annotation, enabling the model to more efficiently learn feature expressions in scenarios where multiple defects coexist.

[0053] 4. Labeling costs are significantly reduced

[0054] By separating labels from attributes, multiple types of defects on the same dial can be annotated using a single rectangular box with a list of attributes, eliminating the traditional method of repeatedly annotating multiple defect boxes. This design simplifies the annotation process, reduces manual annotation time and costs, and reduces the possibility of data annotation errors.

[0055] 5. Improved algorithm robustness

[0056] By introducing a multi-attention head mechanism with adjustable temperature values, the model can adaptively fuse feature responses of different granularities (from local details to global semantics), enhance the ability to distinguish dial defects with large intra-class differences and small inter-class differences, and improve detection stability in complex backgrounds and lighting changing scenarios.

[0057] In summary, the present invention optimizes model architecture, data labeling, training strategy and other dimensions, resulting in substantial improvements in defect detection accuracy, algorithm efficiency, labeling cost and generalization capability, providing efficient and reliable technical support for the intelligent operation and maintenance of substation meters. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 This is the overall implementation process of the present invention;

[0059] Figure 2The overall architecture of the multi-label classification model;

[0060] Figure 3 This is the data processing flow of the meter dial defect detection model. DETAILED DESCRIPTION

[0061] The following is a detailed description of the embodiments of the present invention. Although the present invention will be described and illustrated in conjunction with certain specific embodiments, it should be noted that the present invention is not limited to these embodiments. On the contrary, modifications or equivalent substitutions of the present invention are intended to fall within the scope of the claims of the present invention.

[0062] In addition, in order to better illustrate the present invention, numerous specific details are given in the following detailed description. It will be understood by those skilled in the art that the present invention can also be implemented without these specific details.

[0063] In order to solve the problems of the existing methods, the present invention proposes a meter dial defect detection method based on multi-label classification, the implementation steps (see Appendix Figure 1 )as follows:

[0064] Step 1: Clean and label the meter raw data to obtain the dial defect detection dataset.

[0065] Step 2: Use the watch face defect detection dataset and the Ultralytics framework to train the object detection model YOLOv8m to obtain the watch face detection model.

[0066] Step 3: Design and build a residual attention based multi-label classification model.

[0067] Step 4: Use the dial defect detection dataset and the PyTorch framework to train the multi-label classification model to obtain the dial defect classification model.

[0068] Step 5: Integrate the dial detection model and the dial defect classification model to obtain the meter dial defect detection model.

[0069] Step 1 specifically includes the following steps:

[0070] Step 1.1: Use the image deduplication tool imagededup to process 171,645 original meter sample images, remove duplicate samples, and obtain 154,892 unique high-quality meter sample images.

[0071] Step 1.2: Using the annotation platform cvat, we annotated 154,892 deduplicated meter sample images to generate a dial defect detection dataset, including a training set and a test set. The training set contains 123,913 samples, and the test set contains 30,979 samples. During annotation, attributes such as blur, damage, and dirt are stored separately in a list of rectangular box attributes. Each dial only needs to be annotated with one rectangular box. Each sample image corresponds to a label file, which includes information such as the dial's center point location (x, y), size (w, h), category cls, and attribute attr. Attributes are combinations of one or more elements from {blur, crack, dirty, normal}.

[0072] Step 2 specifically includes the following steps:

[0073] Step 2.1: Use the format conversion tool of the annotation platform cvat to convert the dial defect detection dataset to obtain the dial detection dataset in Ultralytics YOLO format, including a training set and a test set.

[0074] Step 2.2: Select YOLOv8m as the watch face detection model architecture. The hyperparameters for training the watch face detection model are as follows:

[0075] Parameter name imgsz epochs batch lr0 cos_lr degrees shear Parameter value 960 300 32 0.001 true 30 30

[0076] Step 2.3: Use the Ultralytics framework and the dial detection training set to train the YOLOv8m model; use the dial detection test set to evaluate the intermediate model output of each epoch training, and the evaluation indicator is the overall mAP 50 . The overall mAP 50 The highest intermediate model is used as the final output dial detection model.

[0077] Step 3 specifically includes the following steps:

[0078] Step 3.1: Use the layers before the fully connected ResNet-101 fc as the backbone network B to extract the features of the 224×224 pixel dial image and obtain the original features x j

[0079] (j=1, 2, ..., 49; x j ∈R 2048 ).

[0080] Step 3.2: Calculate the attention score of category i (i=1, 2, 3, 4) at spatial position j

[0081] Where K is a temperature value greater than 0, which controls the sharpness of the score; mi ∈R 2048 is the classifier for category i.

[0082] Step 3.3: Calculate the class-specific feature vector a for class i i :

[0083]

[0084] Step 3.4: Calculate the class-independent feature vector g:

[0085]

[0086] Step 3.5: Calculate the category-dependent residual attention feature f for category i i :

[0087] f i =g+λa i

[0088] Here, λ is a hyperparameter with a value of 0.5.

[0089] Step 3.6: Calculate the original predicted values ​​for all categories

[0090]

[0091] The computational graph from step 3.2 to step 3.6 constitutes a single attention head SAH, where the temperature value K is an adjustable hyperparameter.

[0092] Step 3.7: Construct a multi-attention head MAH consisting of 4 single attention heads, with the temperature value K of each head being: K1=1, K2=2, K3=3, K4=∞, and the final raw prediction value of all categories for:

[0093]

[0094] Where T h is the temperature value of the hth single attention classification head. The final prediction score vector s∈R for all categories 4 for:

[0095]

[0096] The first element of s corresponds to blur, the second element corresponds to crack, the third element corresponds to dirty, and the fourth element corresponds to normal. The categories corresponding to the elements in s that are greater than a threshold of 0.5 are considered the dial defect recognition results.

[0097] Step 3.8: The multi-label classification model is composed of the backbone network B and the multi-attention head MAH. The overall architecture is shown in the attached Figure 2 .

[0098] Step 4 specifically includes the following steps:

[0099] Step 4.1: Use the format conversion tool of the annotation platform cvat to convert the dial defect detection dataset to obtain a dial defect classification dataset in Labelme format, including a training set and a test set.

[0100] Step 4.2: Use the dial defect classification training set to train a multi-label classification model in the PyTorch framework. The loss function of the training process is BCE:

[0101]

[0102] Where N is the batch size during training, and y∈{0, 1} is the label value. n (n=1, 2, ..., N) is:

[0103]

[0104] The hyperparameters for training the dial defect classification model are as follows:

[0105] Parameter name imgsz epochs batch lr0 scheduler optimizer Parameter value 224 60 48 0.004 LinearLR SGD

[0106] During the training process, the dial classification test set is used to evaluate the intermediate model output of each epoch training. The evaluation indicator is the overall fl-score:

[0107]

[0108] Where precision is the search accuracy and recall is the search recall:

[0109]

[0110] TP is a true positive, FP is a false positive, and FN is a false negative. The intermediate model with the highest overall fl-score is used as the final output dial defect classification model.

[0111] Step 5 specifically includes the following steps:

[0112] Step 5.1: Process the input original image I through the dial detection model to obtain the center point position (x, y) and size (w, h) of the dial.

[0113] Step 5.2: Based on the position and size of the dial, crop the dial from the original image to obtain the dial image J. The cropping formula is:

[0114]

[0115] Step 5.3: Process the dial image J through the dial defect detection model to obtain the predicted result of the dial defect. The calculation graph composed of steps 5.1 to 5.3 constitutes the meter dial defect detection model. The data processing flow is shown in the attached Figure 3 .

Claims

1. A method for detecting defects in substation meter dials based on multi-label classification, characterized in that: The following steps are involved: Step S1: Construct a sample dataset containing normal dials and defective dials, mark each dial with a single rectangular box and store the defect attributes in an attribute list; Step s2: training a dial detection model based on the YOLOv8m architecture, and optimizing model parameters using all samples in the sample dataset; Step S3: Construct a multi-label classification model based on residual attention, including: The backbone network extracts dial image features and generates original feature vectors; Based on the residual attention mechanism of multiple attention heads, the attention score of each category is calculated to generate category-specific residual attention features; Output multi-label prediction results through the classifier; Step 4: training the multi-label classification model to obtain a dial defect classification model; Step 5: Integrate the dial detection model and the dial defect classification model. After the dial is located by the detection model, the classification model outputs the defect type.

2. The method according to claim 1, characterized in that The annotation process described in step 1 includes: Mark the meter image after deduplication with a single rectangular frame and record the position and size of the dial; The defect attributes are associated with the attribute list of the rectangular box in the form of multiple labels, allowing the same dial to be labeled with multiple defect attributes.

3. The method according to claim 1, characterized in that The object detection model described in step 2 uses the YOLOv8m architecture. The training hyperparameters include: input image size 960×960, learning rate 0.001, training epochs 300, and test set mAP 50 The highest model.

4. The method according to claim 1, wherein The implementation of the residual attention mechanism described in step 3 includes: Calculate the attention score of category i at spatial position j The formula is: Where K is the temperature, x j is the eigenvector, m i is the classifier parameter of category i; Generate residual attention feature f for category i i =g+λa i , where g is the global average feature, a i is the weighted feature, and λ is a hyperparameter.

5. The method according to claim 4, characterized in that The multi-attention head contains 4 independent single attention heads, and the temperature values ​​of each head are set to K1=1, K2=2, K3=3, K4=∞, and the final original prediction value of all categories is for: Where T h is the temperature value of the h-th single attention classification head, and the final prediction score vector s∈R for all categories 4 for: The first element of s corresponds to "blurred", the second element corresponds to "damaged", the third element corresponds to "dirty", and the fourth element corresponds to "normal". The categories corresponding to the elements in s that are greater than the threshold of 0.5 are used as the recognition results of dial defects.

6. The method according to claim 1, characterized in that The step 4 specifically includes: Step 4.1: Convert the dial defect detection dataset into a classification dataset in Label1me format; Step 4.2: Train the multi-label classification model in PyTorch using the binary cross entropy loss function. The training hyperparameters include image size 224 pixels, 60 training epochs, batch size 48, and initial learning rate 0.

004. Step 4.3: Use the LinearLR learning rate scheduler and SGD optimizer to adjust the training process; Step 4.4: After each training round, calculate the overall f1-score based on the test set as the evaluation metric, and select the model with the highest f1-score as the final dial defect classification model.

7. The method according to claim 1, characterized in that The integration described in step 5 includes: Step 5.1: Process the input original image through the dial detection model and output the center point position (x, y) and size (w, h) of the dial; Step 5.2: Based on the center point position and size, the dial is cropped from the original image to obtain a dial image. The cropping formula is: Step 5.3: Input the dial image into the dial defect classification model, and output a prediction result of the dial defect, wherein the prediction result includes one or more combinations of blurry, damaged, dirty, or normal.

Images