An insulator defect detection method based on improved YOLOv3 convolutional neural network
By improving the YOLOv3 convolutional neural network, the attention mechanism and weighted densely connected pyramid structure are introduced, and the problems of inefficiency of traditional insulator defect detection methods and missed detection of small insulators are solved, achieving more efficient, safe and high-precision detection effects.
Patent Information
- Application Number
- CN202111402706.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-11-24
- Publication Date
- 2025-03-18
- Estimated Expiration
- 2041-11-24
AI Technical Summary
Traditional insulator defect detection methods are inefficient and have high risk coefficients. The detection based on YOLOv3 has serious problems with small insulator miss detection, resulting in low detection accuracy and recall.
Improve the YOLOv3 convolutional neural network, and by introducing the convolution kernel attention module SKNet, weighted and densely connected pyramid structure and dual-branch cross-layer attention module, the network's feature extraction and fusion capabilities are enhanced, and the impact of complex environmental backgrounds, multi-scale targets and small targets on detection accuracy is reduced.
It significantly improves the accuracy and recall rate of insulator defect detection, reduces the false detection and missed detection rates, making the detection more efficient, safe and labor-saving.
Smart Images
Figure HDA0003371304690000011 
Figure HDA0003371304690000021 
Figure HDA0003371304690000031
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of power system transmission line detection, and relates to an insulator defect detection method based on an improved YOLOv3 convolutional neural network. Background Art
[0002] Electricity is an important renewable energy source in my country. With the rapid development of the national economy, the entire society's demand for electricity is also increasing. The safety of the power grid is crucial to the development of the entire country and the national economy. Insulators are an indispensable component of transmission lines. They have the function of supporting wires and blocking current. Their integrity plays an important role in the safety of the entire transmission line and in ensuring the stability of power transmission. However, insulators are exposed to harsh environments for a long time and are easily eroded by rain, snow and lightning strikes. They are inevitably damaged, exploded and cracked. Once an insulator is damaged, it will cause power outages at the least and paralyze the power grid at the worst. Therefore, it is crucial to identify and detect the integrity of insulators.
[0003] Traditional insulator defect detection is mainly manual detection, which is inefficient, time-consuming and has a high risk factor. On the one hand, insulators are in the air. The traditional method of taking pictures by drones and then relying on human eyes to observe the pictures to judge the quality of insulators is inefficient and long-term eye fatigue can easily lead to false detection and missed detection; on the other hand, the environment where insulators are located is relatively harsh, such as high voltage or deep mountain environments. When encountering rain, snow and storms, the risk factor is even greater. In recent years, with the development of deep learning and the widespread application of convolutional neural networks in computer vision, new technical support has been provided for the detection and recognition of aerial insulator images. This technology uses the characteristics of convolutional neural networks (CNN) to extract image features, and through training, the model has the ability to autonomously identify targets. At present, there are two mainstream target detection algorithms based on deep learning, one is two-stage and the other is one-stage; Faster-RCNN is a classic algorithm for two-stage target detection networks, while SSD and YOLOv3 are classic algorithms for one-stage target detection networks. However, when Faster-RCNN was used to detect insulator defects in transmission lines, it was found that the detection accuracy did not achieve a good effect, and there were disadvantages such as a large false detection rate and an inference speed that could not reach real-time performance. When SSD was used to detect insulator defects in transmission lines, it was found that there were problems such as poor detection accuracy and a high missed detection rate. When YOLOv3 was used to detect insulator defects in transmission lines, although the detection accuracy was higher than that of Faster-RCNN and SSD, there was still room for improvement, and there was also the problem of difficulty in detecting small insulators.
[0004] Therefore, in view of the various problems existing in the insulator defect detection of transmission lines mentioned above, the present invention proposes an insulator defect detection method based on an improved YOLOv3 convolutional neural network, which can not only change the traditional manual inspection detection method, making the transmission line insulator defect detection more efficient, safer and more labor-saving, but also solve the problem of serious missed detection of small insulators in the insulator defect detection of transmission lines using the original YOLOv3, thereby improving the detection recall rate. Summary of the invention
[0005] The present invention proposes an insulator defect detection method based on an improved YOLOv3 convolutional neural network, improves the network structure of YOLOv3 from the two aspects of reducing false detections and missed detections and improving detection accuracy, and proposes an improved YOLOv3 convolutional neural network based on an attention mechanism, a weighted densely connected pyramid structure and a dual-branch cross-layer attention module, which can reduce the influence of complex environmental background, multi-scale targets and small targets on the detection accuracy, and greatly improve the detection accuracy.
[0006] In order to solve the above technical problems, the present invention adopts the following technical solution: an insulator defect detection method based on an improved YOLOv3 convolutional neural network, comprising the following steps:
[0007] Step 1: Construct an insulator defect dataset: Based on the insulator defect dataset collected by drones and the public insulator defect dataset, remove unclear and ambiguous images, increase the number of dataset images through data enhancement technology, and construct a complete insulator defect dataset;
[0008] Step 2: Establish an image label library for the data set: Use the LabelImg image labeling tool to label the insulators and defects on the insulators;
[0009] Step 3: Divide the labeled data set into training set and test set;
[0010] Step 4, construct a convolutional neural network based on the improved YOLOv3: first, introduce the convolution kernel attention module SKNet. The YOLOv3 backbone network has five residual blocks, namely res1, res2, res8, res8 and res4. An output branch is derived from the second residual block res2, the third residual block res8, the fourth residual block res8 and the fifth residual block res4 of the backbone network, for a total of four branches. A convolution kernel attention module SKNet is added after each branch. Then, a weighted dense connection pyramid module is constructed to replace the feature fusion Neck module of the original YOLOv3. The output feature map of each branch that passes through the convolution kernel attention module SKNet is used as the input of the weighted dense connection pyramid module. Next, a dual-branch cross-layer attention module is constructed, and the feature map output by the weighted dense connection pyramid module is input into the dual-branch cross-layer attention module. Finally, the feature map output by the dual-branch cross-layer attention module is input into the YOLOv3 convolutional neural network detection head.
[0011] Step 5: Use the training set in the data set to train and adjust the parameters of the improved YOLOv3 convolutional neural network, and finally obtain the optimal convolutional neural network model;
[0012] Step 6: Input the test set in the data set into the trained improved YOLOv3 convolutional neural network model, output the insulator detection results and target location information, and realize the transmission line insulator defect detection.
[0013] In the preferred solution, the implementation of step 1 includes: first, using a drone to photograph the insulators on the transmission line towers in the field, removing unclear and ambiguous pictures to obtain the inspected insulator picture data, and then combining it with the publicly released data set on the Internet, and then enhancing the data set pictures by rotating, mirroring, adding noise and adjusting brightness, and finally obtaining a data set containing 3440 pictures;
[0014] In the preferred solution, the implementation of step 2 includes: using the image annotation tool LabelImg to manually annotate the target to be detected on each image, different target objects to be detected have different label categories, and after each image is annotated, an xml file containing all target category annotation information of the image will be generated;
[0015] In the preferred solution, the implementation of step 3 includes: using a random allocation method to randomly divide the 3440 data set images into a ratio of 4:1, including 2752 training sets and 688 test sets;
[0016] In the preferred scheme, the implementation of introducing the convolution kernel attention module SKNet in step 4 includes: first, an output branch is respectively derived from the second residual block res2, the third residual block res8, the fourth residual block res8 and the fifth residual block res4 of the YOLOv3 backbone network, and then a convolution kernel attention module SKNet is connected to each branch.
[0017] In the preferred solution, the implementation of constructing a weighted densely connected pyramid module in step 4 to replace the feature fusion Neck module of the original YOLOv3 includes:
[0018] First, an output branch is derived from the second residual block res2, the third residual block res8, the fourth residual block res8 and the fifth residual block res4 of the YOLOv3 backbone network respectively, and a convolution kernel attention module SKNet is connected to each branch, and the output feature maps of each branch after the convolution kernel attention module SKNet are used as feature Figure 1 ,feature Figure 2 ,feature Figure 3 and Features Figure 4 ;
[0019] Then, the features Figure 1 ,feature Figure 2 ,feature Figure 3 and Features Figure 4 The number of channels and size of the feature map are adjusted through convolution operations, neighboring interpolation and pooling operations, and feature fusion operations are performed from top to bottom, bottom to top, lateral connection and cross-layer connection;
[0020] Finally, each feature map used for feature fusion is multiplied by a trainable weight parameter to form a weighted densely connected pyramid module to replace the original YOLOv3 feature fusion Neck module.
[0021] In a preferred solution, the implementation of building a dual-branch cross-layer attention module in step 4 includes:
[0022] First, four outputs are derived from the weighted densely connected pyramid module and arranged from large to small according to their feature maps as feature map a, feature map b, feature map c, and feature map d;
[0023] Then, feature map a is used as two inputs of three dual-branch cross-layer attention modules together with feature map b, feature map c and feature map d. The input feature map a is first downsampled to be the same size as feature map b, feature map c and feature map d.
[0024] Finally, two feature maps of the same size are subjected to convolution operation, global average pooling, fully connected layer and activation function Sigmoid operation respectively. The feature map obtained by the activation function Sigmoid operation is multiplied with the input feature map of the other branch, and then the two feature maps after the multiplication operation are added to obtain the final feature map.
[0025] In the preferred solution, the implementation of step 5 includes: training the improved YOLOv3 convolutional neural network using the training set in the data set; the network model is trained 34,000 times in total, the image input size is 416×416 pixels, 8 samples are trained in one batch, the initial learning rate of the model is set to 0.001, the momentum is 0.94, the stochastic gradient descent algorithm is used, the weight decay is 0.0005, and finally the parameters are tuned according to the training results.
[0026] In the preferred solution, the implementation of step 6 includes: obtaining the training weights of the optimal model and the improved YOLOv3 network model through training, and loading the training weights and network structure to obtain the improved YOLOv3 network model during the test process, inputting the pictures of the test set into the network model, using different colors of the detection box to represent normal insulators and defective insulators, and displaying the identified target category and the located coordinate information on the original image.
[0027] Compared with the prior art, the present invention has the following advantages and effects:
[0028] 1. Aiming at the problems of model misdetection, high missed detection rate and complex background, the method of the present invention solves this problem by adding a convolution kernel attention module SKNet, thereby improving the detection accuracy and recall rate;
[0029] 2. In response to the problems of multi-scale and small targets, the method of the present invention solves these two problems by constructing a weighted densely connected pyramid module to replace the feature fusion Neck module of the original YOLOv3 and constructing a dual-branch cross-layer attention module, ultimately improving the detection accuracy of multi-scale targets. BRIEF DESCRIPTION OF THE DRAWINGS
[0030] Figure 1 It is a flow chart of an insulator defect detection method based on an improved YOLOv3 convolutional neural network of the present invention.
[0031] Figure 2 1 is a dataset image of some insulator defects according to an embodiment of the present invention.
[0032] Figure 3 It is a structural diagram of the improved YOLOv3 convolutional neural network model used in the embodiment of the present invention.
[0033] Figure 4 SKNet is a model structure diagram of the convolution kernel attention module used in the embodiment of the present invention.
[0034] Figure 5 It is a model structure diagram of the weighted densely connected pyramid network used in the embodiment of the present invention.
[0035] Figure 6 2 is a model structure diagram of a dual-branch cross-layer attention module used in an embodiment of the present invention.
[0036] Figure 7 It is a picture of test results on a part of the test set of an embodiment of the present invention. DETAILED DESCRIPTION
[0037] An insulator defect detection method based on an improved YOLOv3 convolutional neural network proposed in the present invention is described in detail as follows in conjunction with the accompanying drawings:
[0038] Figure 1 The present invention is a flow chart of an insulator defect detection method based on an improved YOLOv3 convolutional neural network, comprising the following steps:
[0039] Step 1: Construct an insulator defect dataset;
[0040] A defective insulator dataset collected by drones and a public insulator defect dataset are combined into a complete dataset. The drone inspection collects 200 pictures and the public insulator dataset collects 248 pictures. The unclear and ambiguous pictures are removed, and the number of pictures in the dataset is increased to 3440 through enhancement techniques such as rotation, mirroring, adding noise and brightness adjustment. Figure 2 1 is a dataset image of some insulator defects according to an embodiment of the present invention.
[0041] Step 2: Establish an image label library for the data set;
[0042] Use the image annotation tool LabelImg to manually annotate the targets to be detected in each image. Different target objects to be detected have different label categories. After each image is annotated, an xml file containing the annotation information of all target categories in the image will be generated.
[0043] Step 3: Dataset division: Use the random allocation method to randomly divide the 3440 dataset images in a ratio of 4:1, including 2752 training sets and 688 test sets.
[0044] Step 4: Construct an improved YOLOv3 convolutional neural network:
[0045] Figure 3The improved YOLOv3 convolutional neural network model structure diagram used in the embodiment of the present invention. First, the original YOLOv3 backbone network is kept unchanged. The backbone network of YOLOv3 is composed of the convolution block DBL in darknet and the residual block Resbock_body composed of the residual unit Res_unit, which is Darknet-53. The convolution block is composed of convolution conv, batch normalization BN and activation function Leaky relu. The residual unit Res_unit is composed of two DBL sequentially connected, a residual connection and an addition operation. The residual block Resbock_body is filled with a zero. padding, a DBL and n residual units are connected in sequence, where n represents the number of residual units; then the convolution kernel attention module SKNet is introduced, and an output branch is derived from the second residual block res2, the third residual block res8, the fourth residual block res8 and the fifth residual block res4 of the YOLOv3 backbone network, and a convolution kernel attention module SKNet is connected behind each branch. Then, a weighted dense connection pyramid module is constructed to replace the original YOLOv3 feature fusion Neck module, and the output feature maps of each branch of the convolution kernel attention module SKNet are used as the input feature maps of the weighted dense connection pyramid module. Then, the four feature maps of different sizes output by the weighted dense connection pyramid module are input into the three dual-branch cross-layer attention modules respectively. Finally, the three feature maps of different sizes output by the three dual-branch cross-layer attention modules are input into the original detection head YOLOv3 YOLO_head.
[0046] Step 5: Use the training set in the data set to train and adjust the parameters of the improved YOLOv3 convolutional neural network, and finally obtain the optimal convolutional neural network model:
[0047] First, the pre-trained weights of the backbone network are loaded using the transfer learning method, and then the training set data is loaded to train the network model. A total of 34,000 training times are performed, the image input size is 416×416 pixels, 8 samples are trained in one batch, the initial learning rate of the model is set to 0.001, the momentum is 0.94, the stochastic gradient descent algorithm is used, and the weight decay is 0.0005. Finally, the parameters are tuned according to the training results until the optimal training model result is obtained, and the optimal model weight result is saved.
[0048] Step 6: Input the test set in the data set into the trained improved YOLOv3 convolutional neural network model, output the insulator detection results and target location information, and realize the transmission line insulator defect detection:
[0049] The optimal model training weights and improved YOLOv3 network model are obtained through training. The training weights and network structure are loaded during the test process to obtain the improved YOLOv3 network model. The test set images are input into the network model. Different colors of the detection box are used to represent normal insulators and defective insulators. The identified target category and the located coordinate information are displayed on the original image. The final detection result is as follows: Figure 7 shown.
[0050] Figure 4 SKNet is a model structure diagram of the convolution kernel attention module used in the embodiment of the present invention.
[0051] First, the input feature map U of size C×H×W is subjected to a 5×5 convolution operation and a 3×3 convolution operation respectively, and two feature maps U1 and U2 of size C×H×W are obtained respectively. Then, the two feature maps are element-wise added to obtain a feature map U3 of size C×H×W. Then, the fused feature map U3 is subjected to a global average pooling operation, and the feature map is averaged in the two dimensions H and W to obtain a one-dimensional vector C×1×1, which represents the importance of the information of each channel of the feature map. Then, a fully connected layer is input to obtain a one-dimensional feature vector of size C×1×1, and then the one-dimensional feature vector is normalized using the activation function softmax. Finally, the one-dimensional feature vector after the activation function softmax normalization operation is divided into two parts, and the two feature maps U1 and U2 are element-wise multiplied respectively, and then the two feature maps U4 and U5 after the multiplication are element-wise summed, and finally the feature map U6 of the same size as the original input feature map is obtained, and it is input into the weighted dense connection pyramid module as input.
[0052] Figure 5 It is a model structure diagram of the weighted densely connected pyramid network used in the embodiment of the present invention.
[0053] First, an output branch is derived from the second residual block res2, the third residual block res8, the fourth residual block res8 and the fifth residual block res4 of the YOLOv3 backbone network respectively, and a convolution kernel attention module SKNet is connected to each branch, and the output feature maps of each branch after the convolution kernel attention module SKNet are used as feature Figure 1 ,feature Figure 2 ,feature Figure 3 and Features Figure 4 ;
[0054] Then, the features Figure 1 ,feature Figure 2 ,feature Figure 3 and Features Figure 4The number of channels and size of the feature map are adjusted through convolution, upsampling, and downsampling, and feature fusion operations are performed from top to bottom, bottom to top, lateral connection, and cross-layer connection;
[0055] Finally, each feature map used for feature fusion is multiplied by a trainable weight parameter w, where w1, w2…w23 represent different weight parameters, forming a weighted densely connected pyramid module to replace the original YOLOv3 feature fusion Neck module.
[0056] Figure 6 2 is a model structure diagram of a dual-branch cross-layer attention module used in an embodiment of the present invention.
[0057] First, four outputs are derived from the weighted densely connected pyramid module and arranged from large to small according to their feature maps as feature map a, feature map b, feature map c, and feature map d;
[0058] Then, feature map a is used together with feature map b, feature map c and feature map d as two inputs X and Y of three dual-branch cross-layer attention modules, where feature map X represents a larger feature map at the bottom layer, and feature map Y represents a smaller feature map at the top layer. The input feature map X is first downsampled to be adjusted to feature map X1 of the same size as feature map Y.
[0059] Finally, the feature map Y first undergoes a global average pooling, then two fully connected layers, and then the activation function Sigmoid operation, and finally multiplies it with the feature map X1 to obtain the feature map X2. The feature map X1 first undergoes two 1×1 convolutions, then the activation function Sigmoid operation, and finally multiplies it with the feature map Y to obtain the feature map Y1. The obtained feature map X2 and feature map Y1 are added to obtain the final feature map Z.
[0060] Figure 7 It is a test result picture on a part of the test set of the embodiment of the present invention, wherein different colors of the detection box are used to represent the normal insulator and the defective insulator, and the identified target category and the located coordinate information are displayed on the original image.
Claims
1. An insulator defect detection method based on an improved YOLOv3 convolutional neural network, characterized in that: The following steps are involved: Step 1: Construct an insulator defect dataset: Based on the insulator defect dataset collected by drones and the public insulator defect dataset, remove unclear and ambiguous images, increase the number of dataset images through data enhancement technology, and construct a complete insulator defect dataset; Step 2: Establish an image label library for the data set: Use the LabelImg image labeling tool to label the insulators and defects on the insulators; Step 3: Divide the labeled data set into training set and test set; Step 4. Construct a convolutional neural network based on the improved YOLOv3: First, introduce the convolution kernel attention module SKNet. The YOLOv3 backbone network has five residual blocks, namely res1, res2, res8, res8 and res4. An output branch is derived from the second residual block res2, the third residual block res8, the fourth residual block res8 and the fifth residual block res4 of the backbone network, for a total of four branches. Add a convolution kernel attention module SKNet after each branch, and use the output feature maps of each branch after the convolution kernel attention module SKNet as feature map 1, feature map 2, feature map 3 and feature map 4 respectively. Then, the feature map 1. Feature maps 2, 3 and 4 adjust the number of channels and sizes of feature maps through convolution operations, neighbor interpolation and pooling operations, and perform feature fusion operations from top to bottom, bottom to top, lateral connection and cross-layer connection; finally, each feature map used for feature fusion is multiplied by a trainable weight parameter to form a weighted dense connection pyramid module to replace the original YOLOv3 feature fusion Neck module; then, a dual-branch cross-layer attention module is constructed, and the feature map output by the weighted dense connection pyramid module is input into the dual-branch cross-layer attention module. Finally, the feature map output by the dual-branch cross-layer attention module is input into the YOLOv3 convolutional neural network detection head; Step 5: Use the training set in the data set to train and adjust the parameters of the improved YOLOv3 convolutional neural network, and finally obtain the optimal convolutional neural network model; Step 6: Input the test set in the data set into the trained improved YOLOv3 convolutional neural network model, output the insulator detection results and target location information, and realize the transmission line insulator defect detection.
2. According to claim 1, the insulator defect detection method based on the improved YOLOv3 convolutional neural network is characterized in that: The implementation of step 1 includes: first, using drones to photograph insulators on transmission line towers in the wild, removing unclear and ambiguous images to obtain inspected insulator image data, and then combining it with the publicly released online datasets. Then, the dataset images are enhanced through rotation, mirroring, noise addition, and brightness adjustment techniques, and finally a dataset containing 3,440 images is obtained.
3. According to claim 1, the insulator defect detection method based on the improved YOLOv3 convolutional neural network is characterized in that: The implementation of step 2 includes: using the image annotation tool LabelImg to manually annotate the targets to be detected in each image. Different target objects to be detected have different label categories. After each image is annotated, an xml file containing the annotation information of all target categories in the image will be generated.
4. According to claim 1, the insulator defect detection method based on the improved YOLOv3 convolutional neural network is characterized in that: The implementation of step 3 includes: using the random allocation method to randomly divide the 3440 data set images in a ratio of 4:1, including 2752 training sets and 688 test sets.
5. The insulator defect detection method based on the improved YOLOv3 convolutional neural network according to claim 1 is characterized in that: The implementation of building a two-branch cross-layer attention module in step 4 includes: First, four outputs are derived from the weighted densely connected pyramid module and arranged from large to small according to their feature maps as feature map a, feature map b, feature map c, and feature map d; Then, feature map a is used together with feature map b, feature map c and feature map d as two inputs X and Y of three dual-branch cross-layer attention modules, where feature map X represents a larger feature map at the bottom layer, and feature map Y represents a smaller feature map at the top layer. The input feature map X is first downsampled to be adjusted to feature map X1 of the same size as feature map Y. Finally, the feature map Y first undergoes a global average pooling, then two fully connected layers, and then the activation function Sigmoid operation, and finally multiplies it with the feature map X1 to obtain the feature map X2. The feature map X1 first undergoes two 1×1 convolutions, then the activation function Sigmoid operation, and finally multiplies it with the feature map Y to obtain the feature map Y1. The obtained feature map X2 and feature map Y1 are added to obtain the final feature map Z.
6. The insulator defect detection method based on the improved YOLOv3 convolutional neural network according to claim 1 is characterized in that: The implementation of step 5 includes: using the training set in the dataset to train the improved YOLOv3 convolutional neural network; the network model is trained 34,000 times in total, the image input size is 416×416 pixels, 8 samples are trained in one batch, the initial learning rate of the model is set to 0.001, the momentum is 0.94, the stochastic gradient descent algorithm is used, the weight decay is 0.0005, and finally the parameters are tuned according to the training results.
7. The insulator defect detection method based on the improved YOLOv3 convolutional neural network according to claim 1 is characterized in that: The implementation of step 6 includes: obtaining the training weights of the optimal model and the improved YOLOv3 network model through training, and loading the training weights and network structure to obtain the improved YOLOv3 network model during the test process, inputting the pictures of the test set into the network model, using different colors of the detection box to represent normal insulators and defective insulators, and displaying the identified target category and the located coordinate information on the original image.