Insulator thermogram defect detection method based on improved YOLOv7

Abstract

The invention discloses an insulator thermogram defect detection method based on improved YOLOv7. The method comprises the following steps: S1, obtaining an initial insulator thermogram; s2, supplementing the insulator thermogram data set through a data enhancement technology; s3, marking the thermal image of the insulator to form a data set, and dividing the data set into a test set and a training set; s4, performing improvement and modeling on a YOLOv7 network model, adding a large kernel attention mechanism LKA in a backbone feature extraction layer (backbone) of the YOLOv7 network model to optimize the network, introducing an adaptive dynamic learning activation function Meta-ACON to replace an activation function (Silu) of the original YOLOv7, and in the aspect of the model, adopting a lightweight ghosting shuffling convolution module GSConv to replace a conventional convolution Conv in a head module of the YOLOv7; and S5, carrying out training and performance evaluation on the improved network model to obtain a defect detection result. Compared with an original YOLOv7 network model, the method has the advantage that the feature extraction of the insulator thermogram can be better realized.

Description

Technical Field

[0001] The present invention relates to the field of image processing and power equipment defect detection, and in particular to an insulator thermal image defect detection method based on improved YOLOv7. Background Art

[0002] As one of the most important components in the transmission line, insulators can not only achieve insulation between transmission lines, between towers, and between lines and the ground, but also provide mechanical fixation of the conductive line and creepage distance between high-potential conductors and zero-potential bodies. In order to ensure normal power transmission and avoid risks and losses, regular maintenance of insulators is of great practical significance. Accurate and rapid detection of insulator defects is of great significance to avoid large-scale power outages and reduce power grid losses. It can provide guarantees for ensuring safe and stable power transmission and lay the foundation for subsequent engineering applications. At present, deep learning algorithms are widely used in the identification and positioning of high-voltage insulators and their defects in transmission lines. Object detection algorithms based on deep learning have been widely used in power systems due to their good generalization ability and ability to extract features from complex backgrounds.

[0003] Since the subtle external defects of insulators are not easy to be found due to the observation angle or dirt coverage, it is difficult to find such defects in the photos of insulators taken during the day. However, when the surface of the insulator is hot due to dirt defects, its heat power will rise rapidly and will be presented by the infrared thermal image of the night inspection. Since the thermal imaging method is suitable for the identification of defects that produce local heating, the defect state of the insulator can be identified by observing this heating state, so a method for detecting defects in insulator thermal images is needed. Thermal imaging detection has become a recommended method for the power industry. Since inspections such as drones will produce a large number of thermal imaging images, the accuracy and efficiency of image processing have become a difficulty and research hotspot of this technology. There are many artificial intelligence target detection algorithms that are applied to the image processing of power insulators, among which the advantages of the YOLO series are relatively consistent. Combined with the update of technology, the current YOLOv7 method is a relatively new algorithm, but it also needs to be improved to adapt to the scene requirements where the defect characteristics of insulator thermal images are not easy to distinguish.

[0004] Therefore, the present invention provides an insulator thermal image defect detection method based on improved YOLOv7, which can realize the detection of weak characteristic defects such as insulator contamination in transmission lines, improve the accuracy and efficiency of image processing, and provide a new idea for the intelligent inspection of transmission lines.

[0005] Patent CN 116824341A discloses an improved insulator abnormal temperature rise detection method based on YOLOv7. This method replaces the backbone network of the YOLOv7 network with the EfficientNetV2 backbone network and refers to the BiFormer attention mechanism and Focal-EIOU loss function. This method can detect the abnormal temperature rise of insulators to a certain extent. The model of this method is relatively complex, the amount of calculation is large, and the detection processing speed is long.

[0006] Patent CN 115932500A discloses a composite insulator point temperature rise defect diagnosis method and equipment based on temperature rise curve. This method determines whether the insulator has defects by detecting the curve of the insulator surface temperature change over time. This method requires controlling the drone to patrol the tower multiple times, and then transmitting the data to the computer to perform fault diagnosis by comparing the point temperature rise, which consumes considerable manpower, material resources and other costs. The core of the disclosed technical content is how to establish a standard temperature curve that can be benchmarked, which is a relatively cumbersome problem. Summary of the invention

[0007] The present invention proposes a method for detecting defects in insulator thermal images based on an improved real-time target detection algorithm YOLOv7 (YOLO, You Only LookOnce), which realizes the rapid diagnosis of insulator thermal image faults, improves the speed and accuracy of image recognition and feature extraction, and provides guidance for subsequent transmission line maintenance. To achieve the above-mentioned purpose of the invention, the technical solution adopted by the present invention is:

[0008] The present invention provides an insulator thermal image defect detection method based on improved YOLOv7, comprising the following steps:

[0009] S1. Obtaining an initial insulator thermal image;

[0010] S2, supplement the insulator thermal image dataset through data enhancement technology;

[0011] S3, annotate the thermal images of the insulators to form a data set and divide it into a test set and a training set;

[0012] S4. Improve and model the YOLOv7 network model;

[0013] S5. Train and evaluate the performance of the improved network model to obtain defect detection results.

[0014] Specifically, in step S1, an infrared thermal imager is used to capture a thermal image of the insulator, and various online data set public resources are combined to form an initial data set required by the present invention.

[0015] Specifically, in step S2, offline enhancement techniques such as Gaussian blur, random noise addition, and color jitter are used to expand the data set; Mosaic online enhancement technology is used to generalize and expand the sample data. This data enhancement technology can splice four different images in the training set together to form a new image. The insulator thermal image data set is supplemented by combining these two online and offline enhancement data enhancement technologies.

[0016] Specifically, in step S3, the labeling tool Labelimg is used to label the defective insulator thermal imaging .PNG file to obtain an .xml file, which is then converted into a .txt file adapted for the YOLO network. After the labeling of the image data is completed, the required data set is formed and divided into a test set and a training set according to the proportion.

[0017] Specifically, in step S4, the large kernel attention mechanism module LKA (Large Kernel Attention) is added to the backbone feature extraction layer (backbone) of the YOLOv7 network model to optimize the network, and a new detection layer is added. The idea of ​​the large kernel attention mechanism module LKA is to decompose a large convolution kernel into three parts: 1 deep convolution, 1 deep dilated convolution and 1 convolution. The specific process expression is: k attention =Conv 1×1 (DW_D_Conv(DW_Conv(F)))

[0018] Where F is the input feature of this layer; DW_Conv, DW_D_Conv and Conv 1×1 Respectively represent depthwise convolution, depthwise dilated convolution, and convolution; k attention To calculate the attention parameter kernel, each parameter represents the importance of each feature of F.

[0019] Specifically, in step S4, the adaptive dynamic learning activation function Meta-ACON is referenced to replace the activation function (Silu) in the original YOLOv7 network model.

[0020] The adaptive dynamic learning activation function Meta-ACON activation function is an improvement of Relu and Swish activation functions. It controls whether neuron β is activated by setting the value generated by two layers of 1×1 convolution. If β is 0, it means it is not activated. Therefore, it is necessary to construct an adaptive function for calculating β in the three design spaces of the adaptive function layer-wise, channel-wise, and pixel-wise for Meta-ACON. The adaptive function of β is:

[0021] Where σ is the sigmoid function; H is the height of the feature map; W is the width of the feature map; x c,h,w represents the input feature vector with c channels, h height and w width; W1∈R C×C / r and W2∈R C / r×C To save parameters, the scaling factor r is set to 16.

[0022] Specifically, in step S4, the lightweight ghost shuffle convolution module GSConv (Ghost-shuffleconvolution) is referenced to replace the conventional convolution Conv in the head module of the original YOLOv7 network model.

[0023] The lightweight ghost shuffle convolution module GSConv discards some conventional convolutions and uses linear transformation for feature extraction. The input feature map undergoes standard convolution, depth convolution, splicing, and channel shuffling to obtain the output feature map. The general feature map is extracted using standard convolution, and then the features of each channel are linearly operated to generate a GS feature map with the same number of channels. Finally, the basic feature map and the GS feature map are stacked as output.

[0024] Compared with the prior art, the present invention has the following beneficial effects:

[0025] The present invention provides an insulator thermal image defect detection method based on improved YOLOv7, which collects a large number of insulator thermal images, supplements the insulator thermal image data set through a data enhancement technology combining online enhancement and offline enhancement, and annotates the image data to form a data set and divides it into a test set and a training set. Then, the YOLOv7 network model is used to detect the insulator thermal image, and it is proposed to add a large kernel attention mechanism module LKA (Large Kernel Attention) to the backbone feature extraction layer of the YOLOv7 network model to optimize the network, and quote the adaptive dynamic learning activation function Meta-ACON to replace the original activation function. In order to reduce the computational cost of insulator fault identification, a lightweight ghost shuffle convolution module GSConv (Ghost-shuffleconvolution) is used to replace the standard convolution.

[0026] Finally, the extracted features are used for defect detection of insulator thermal images. The experimental results show that the invention realizes the rapid diagnosis of insulator thermal image faults and can provide a reference for subsequent transmission line maintenance. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only preferred embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0028] Figure 1 The figure is a schematic flow chart of the insulator thermal image defect detection method of the present invention.

[0029] Figure 2 This is a schematic diagram of the structure of the large kernel attention mechanism module LKA (Large Kernel Attention) introduced in the present invention. 2-1: deep convolution; 2-2: deep dilated convolution; 2-3: convolution.

[0030] Figure 3 This is a schematic diagram of the adaptive dynamic learning activation function Meta-ACON activation function introduced in the present invention (black indicates activation, white indicates inactivation, and the depth of black indicates the different degrees of activation). 3-1: input layer; 3-2: convolutional layer; 3-3: BN layer (normalization); 3-4: Meta-ACON activation.

[0031] Figure 4 The structure diagram of the lightweight ghost shuffle convolution module GSConv (Ghost-shuffle convolution) introduced in the present invention. 4-1: input; 4-2: standard convolution; 4-3: depth convolution; 4-4: splicing operation; 4-5: channel shuffle; 4-6: output).

[0032] Figure 5 It is a schematic diagram of the structure of the improved YOLOv7 network model and module of the present invention. DETAILED DESCRIPTION

[0033] Combination Figure 1 , a schematic diagram of an insulator thermal image defect detection method based on improved YOLOv7 provided by the present invention, the method comprising the following steps:

[0034] Step S1, obtaining an initial insulator thermal image;

[0035] As an implementable manner, an infrared image of the insulator is collected by using an infrared thermal imager, and the initial data set required by the present invention is formed by combining various online data set public resources.

[0036] Step S2, supplementing the insulator thermal image data set by using a data enhancement technology combining online enhancement and offline enhancement;

[0037] Based on step S1, the initial data set is further supplemented by data enhancement technology. Data enhancement is divided into two types of processing: online augmentation and offline augmentation. The images in the initial data set are flipped, the brightness is randomly adjusted, the images are randomly cropped, and the images are rotated to increase the diversity of the samples, so as to increase the number of samples. Before entering the network model, the samples are flipped, the brightness is adjusted, Mosaic, and randomly cropped. By combining the two enhancement technologies, the samples of the data set can be expanded, the generalization ability of the network can be improved, and the robust performance of the network model can be improved.

[0038] Step S3, annotating the insulator thermal image to form a data set and dividing it into a test set and a training set;

[0039] For normal insulators and defective insulators in the thermal image, the labeling software labeling tool Labelimg is used to label the defective insulator thermal imaging .PNG file to obtain an .xml file, which is then converted into a .txt file adapted for the YOLO network. After the labeling of the image data is completed, the required data set is formed and divided into a test set and a training set according to the proportion.

[0040] Step S4, improving and modeling the YOLOv7 network model;

[0041] After evaluating the mainstream target detection models at this stage, the present invention selects the YOLOv7 network model as the benchmark model, optimizes the backbone feature extraction layer of the model, introduces the large kernel attention mechanism module LKA (Large Kernel Attention) to enhance the model's feature extraction capability for small targets, which helps the model to better locate and identify insulator targets; uses the adaptive dynamic learning activation function Meta-ACON to replace the original activation function to improve the performance of the model while making the model training more stable; introduces the lightweight ghost shuffle convolution module GSConv (Ghost-shuffle convolution) to reduce the number of model parameters and balance the accuracy and speed of the model.

[0042] This example introduces the large kernel attention mechanism module LKA (Large Kernel Attention) in the backbone feature extraction layer of the YOLOv7 network model, and the large kernel attention mechanism module LKA is connected to the SPPCSPC network. The specific location is the last layer of the backbone network, that is, the 31st layer of the entire initial YOLOv7 network model.

[0043] See also Figure 2The large kernel attention mechanism module LKA (Large Kernel Attention) decomposes a large convolution into a combination of a deep convolution layer, a deep dilated convolution layer and a convolution layer to enhance the model's receptive field range and attention to local areas of the image.

[0044] This example uses the adaptive dynamic learning activation function Meta-ACON to replace the original activation function. It replaces the SiLU activation function in the CBS module in the initial YOLOv7 network model.

[0045] See also Figure 3 The adaptive dynamic learning activation function Meta-ACON combines the Relu and Swish activation functions to improve the activation function. It controls whether the neuron β is activated by setting the value generated by two layers of 1×1 convolution. If β is 0, it means it is not activated ( Figure 3 Black indicates activation, white indicates inactivation, and the depth of black indicates different degrees of activation). Therefore, it is necessary to construct an adaptive function for calculating β for Meta-ACON in the three design spaces of adaptive function: layer-wise, channel-wise, and pixel-wise. The adaptive function of β is:

[0046] Where σ is the sigmoid function; H is the height of the feature map; W is the width of the feature map; x c,h,w represents the input feature vector with c channels, h height and w width; W1∈R C×C / r and W2∈R C / r×C To save parameters, the scaling factor r is set to 16.

[0047] See also Figure 4 The lightweight ghost shuffle convolution module GSConv (Ghost-shuffle convolution) abandons some conventional convolutions and uses linear transformation for feature extraction. The input feature map undergoes standard convolution, depth convolution, splicing, and channel shuffling to obtain the output feature map. The general feature map is extracted using standard convolution, and then the features of each channel are linearly operated to generate a GS feature map with the same number of channels. Finally, the basic feature map and the GS feature map are stacked as output.

[0048] This example introduces a lightweight ghost shuffle convolution module GSConv (Ghost-shuffle convolution) to replace Conv, which retains the links between channels to the greatest extent, and can ensure the accuracy of the model while reducing the computational cost. Since all convolutions in the network use GSConv, the number of layers will be deepened and the reasoning speed will be slowed down, and the dimensions of some channels in the backbone feature extraction layer have become larger and the width and height have become smaller, so using GSConv in the head module can improve the reasoning speed of the network while improving the accuracy of the network. While reducing the computational cost, keep the effect of ordinary convolution as much as possible, which can improve the reasoning speed of the network while improving the accuracy of the network.

[0049] See also Figure 5 The improved YOLOv7 network model includes: a CBS module, a UP module (UPSample), an MP module (MP-1, MP-2), an ELAN module, an ELAN-H module, a SPPCSPC module, a REP module, the large core attention mechanism module LKA and the lightweight ghost shuffle convolution module GSConv. The CBS module consists of a Conv convolution layer, a BN layer and a SiLU activation function; the UP module (UPSample) is an upsampling module, and the upsampling method used is the nearest neighbor interpolation; the MP module includes two downsampling modules, MP-1 and MP-2, the difference between which is that the number of channels of the feature map will not change after passing through the MP1 module, while the MP2 module will double the number of channels of the feature map; the ELAN module enhances the learning ability of the network by controlling the longest and shortest gradient paths; the difference between the ELAN-H module and the ELAN module is that the number of spliced ​​CBS modules is different; the SPPCSPC module increases the receptive field of the model by setting pooling kernels of different sizes to extract more features; the REP module is used to introduce multi-scale feature information and improve the receptive field of the model, thereby improving the performance of target detection.

[0050] Step S5: training and performance evaluation of the improved network model to obtain defect detection results;

[0051] This example is experimentally verified based on the above steps, and compares the improved YOLOv7 of the present invention with other mainstream target detection models in terms of precision P (Precision), recall R (Recall), average precision (meanAverage Precision, mAP@0.5) when the IoU value is 0.5, and the frame processing speed of the representative model per second.

[0052] The calculation formulas for precision P and recall R are as follows:

[0053] TP (True Positive) represents a correctly predicted positive sample, that is, the sample is detected and the prediction result is correct; FP (False Positive) represents an incorrectly predicted positive sample, that is, the sample is detected but the prediction result is wrong, also known as a false positive; FN (False Negative) represents an incorrectly predicted negative sample, that is, the sample is not detected, but there should be a sample at this position, also known as a missed detection.

[0054] IoU is generally used to compare the overlap between the predicted box and the real box. In some target detection tasks, the threshold is preset. The calculation formula is as follows:

[0055] The mean average precision (mAP@0.5) is usually preset for the threshold, and the prediction results are sorted in descending order according to the IoU value. After changing the threshold value, repeat this process to obtain the PR curve, thereby obtaining the average precision AP (Average Precision) value. The calculation formula is shown below:

[0056] In some optional embodiments, other mainstream target detection models, the YOLOv7 network model, and the improved YOLOv7 network model provided by the present invention are used for comparative experiments, and the experimental results are shown in Table 1: Table 1 Comparison of performance results of various target detection models Model P(％) R(％) mAP@0.5 Detection speed (FPS) SSD 87.4～87.6 56.4～56.9 82.7～82.9 20.7～21.2 Faster R-CNN 62.1～62.5 84.2～84.6 84.0～84.3 25.4～25.7 YOLOv5 88.6～88.8 86.4～86.7 85.3～85.5 61.2～61.4 YOLOv7 89.1～89.4 88.4～88.6 87.0～87.2 65.0～65.3 OURS 91.7～92.0 90.5～90.8 90.8～91.2 64.0～64.4

[0057] In Table 1, the average detection accuracy of the network model of the present invention is greatly improved compared with the SSD, Faster R-CNN, YOLOv5 and YOLOv7 algorithm models, which further verifies the effectiveness of the method of the present invention and can better meet the rapid detection of insulator thermal image defects.

[0058] The preferred embodiments of the present invention are described above. It should be understood that the present invention is not limited to the above-mentioned specific embodiments, and the devices and structures that are not described in detail should be understood to be implemented in a common manner in the art. Any technician familiar with the art can use the above-disclosed methods and contents to make many possible changes and modifications to the technical solution of the present invention without departing from the scope of the technical solution of the present invention, or modify it into an equivalent embodiment of equivalent changes, which does not affect the essential content of the present invention. Therefore, any simple modification, equivalent change and modification made to the above embodiments according to the technical essence of the present invention without departing from the content of the technical solution of the present invention still falls within the scope of protection of the technical solution of the present invention.

Claims

1. An insulator thermal image defect detection method based on improved YOLOv7, characterized in that: The following steps are involved: S1. Obtain the required initial insulator thermal image; S2 supplements the insulator thermal image dataset by using a data enhancement technique that combines online enhancement and offline enhancement; S3, annotate the thermal images of the insulators to form a data set and divide it into a test set and a training set; S4. Introducing the large kernel attention mechanism LKA (Large Kernel Attention), the adaptive dynamic learning activation function Meta-ACON and the lightweight ghost shuffle convolution module GSConv (Ghost-shuffle convolution) into the YOLOv7 network model; obtaining the improved YOLOv7 network model; S5. Use the improved network model to perform defect detection and performance evaluation on the image data set to be detected to obtain detection results.

2. The insulator thermal imaging defect detection method based on improved YOLOv7 according to claim 1 is characterized in that: The invention introduces the attention mechanism module LKA (Large Kernel Attention), the adaptive dynamic learning activation function Meta-ACON and the lightweight ghost shuffle convolution module GSConv (Ghost-shuffle convolution) into the YOLOv7 network model; The LKA attention mechanism is added to the backbone feature extraction layer (backbone) network of the YOLOv7 network model; the original SiLU activation function in the YOLOv7 network model is replaced by the adaptive dynamic learning activation function Meta-ACON; the lightweight ghost shuffle convolution module GSConv replaces the conventional convolution Conv in the head module of the YOLOv7 network model.

3. The insulator thermal imaging defect detection method based on improved YOLOv7 according to claim 1 is characterized in that The insulator thermal image is subjected to data preprocessing, including: The insulator thermal imaging image data set is expanded using data enhancement technology, and the insulator thermal imaging image is annotated using annotation tools to obtain the required file. After the annotated image data is completed, the required data set is formed and divided into a test set and a training set according to the proportion.

4. The data enhancement technique according to claim 3, characterized in that The dataset is expanded by using offline enhancement techniques such as Gaussian blur, random noise addition and color jitter, and the sample data is generalized and expanded by using Mosaic online enhancement technology. The insulator thermal image dataset is supplemented by a data enhancement technology combining the online enhancement and the offline enhancement.

5. The insulator thermal imaging defect detection method based on improved YOLOv7 according to claim 1 is characterized in that: After training, the performance evaluation of the improved YOLOv7 network model is tested. The model evaluation indicators include precision P (Precision), recall R (Recall), average precision (mAP@0.5, meanAveragePrecision) when the IoU value is 0.5, and the frame processing speed of the representative model per second as the performance evaluation indicators of the improved YOLOv7 model.

6. The insulator thermal imaging defect detection method based on improved YOLOv7 according to claim 4 is characterized in that: The IoU is generally used to compare the overlap between the predicted box and the real box. In some target detection tasks, a threshold is preset. The calculation formula is as follows: The average precision mAP@0.5 when the IoU value is 0.5 is usually a preset threshold, and the prediction results are sorted in descending order according to the IoU value. After changing the threshold value, repeat this process to obtain the PR curve, thereby obtaining the average precision AP (Average Precision) value. The calculation formula is shown below: 。

Images