An insulator defect detection method based on improved YOLOv4-tiny and YOLOv4

Abstract

An insulator defect detection method based on improved YOLOv4‑tiny and YOLOv4 is characterized by: step 1, collecting image data containing insulators; step 2, expanding the number of defective insulator images; step 3, marking insulator strings and defective insulators; step 4, improving the YOLOv4‑tiny model target detection frame selection method; step 5, using the improved YOLOv4‑tiny model to detect the overall target of the insulator string; step 6, cropping the detected insulator string image; step 7, improving the YOLOv4 model backbone network; step 8, improving the YOLOv4 feature fusion network; step 9, based on improved YOLOv4 insulator defect detection; step 10, based on improved YOLOv4‑tiny and YOLOv4 insulator defect detection model training; step 11, insulator defect detection model online application. Its detection accuracy is less affected by background and has higher accuracy.

Description

Technical Field

[0001] The invention belongs to the technical field of insulator defect detection and relates to an insulator defect detection method based on improved YOLOv4-tiny and YOLOv4. Background Art

[0002] The insulator string is composed of multiple insulators and is an important part of the transmission line. It plays a vital role in supporting and fixing the transmission wires, forming good insulation between the wires and the ground, and preventing the current from returning to the ground. Once the insulator is defective, it will affect the insulation and mechanical properties of the insulator string, which can easily lead to contact between the transmission wires or between the transmission wires and the transmission towers, causing short circuits and transmission defects. In severe cases, it may even cause large-scale power outages, affecting the safe and stable operation of the power system. Insulators are exposed to the natural environment for a long time. They not only bear normal mechanical loads and power loads, but also have to withstand extreme weather such as lightning strikes, strong winds, freezing, extreme cold and extreme heat, resulting in frequent insulator defects. Therefore, it is necessary to regularly inspect the transmission lines, promptly detect insulator defects, and notify the operation and inspection department for processing.

[0003] Compared with the traditional manual inspection method, the transmission line inspection method based on drones is highly efficient and has become one of the important methods in daily inspection. First, drones are used to collect images of transmission lines, and then videos are manually viewed or machine vision methods are used to analyze video images to find defects in insulators and other transmission equipment. The use of manual video viewing for insulator defect identification is inefficient, greatly affected by human factors, and wastes a lot of human resources. The insulator defect detection method based on machine vision mainly uses machine learning methods to analyze video images and identify defective insulators. The traditional machine vision method mainly uses feature extraction methods (such as gradient directional histograms) to extract features from insulator images, and then uses classification methods (such as support vector machines and correlation vector machines) to classify the extracted features to achieve insulator defect detection. This type of method requires artificially designed feature extraction methods, has poor generalization ability, and low accuracy. In recent years, people have proposed to apply deep learning-based target detection methods to insulator defect detection, such as: R-CNN (region-based convolutional neural network) series target detection methods, YOLO (You Only Look Once) series target detection methods and SSD (Single Shot MultiBox Detector) series detection methods. The R-CNN series methods have high accuracy, but the network model is complex and cannot be deployed on computing devices that can be carried by drones, and the real-time performance is poor. The YOLO and SSD series methods have simple network models and fast detection speeds. The detection accuracy is slightly lower than that of the R-CNN class methods, but they can be deployed on computing devices carried by drones, and the real-time performance is good. They are currently widely used in autonomous transmission line inspections by drones. Due to the shooting angle, position and distance of drone inspections, the insulator background in the image is complex and diverse, and some insulators account for a small proportion of the image, resulting in the low accuracy of YOLO class methods in identifying insulator defects, which affects their practical application. Therefore, it is necessary to study high-accuracy insulator defect detection methods to improve the real-time detection performance of insulator defects based on autonomous drone inspections. Summary of the invention

[0004] The technical problem to be solved by the present invention is to overcome the shortcomings of the prior art and provide an insulator defect detection method based on improved YOLOv4-tiny and YOLOv4 to improve the accuracy of real-time detection of insulator defects.

[0005] The solution to the technical problem of the present invention is: a method for detecting insulator defects based on improved YOLOv4-tiny and YOLOv4, characterized in that the specific steps are as follows:

[0006] Step 1, constructing an image dataset: from the transmission line images taken by the drone, select images containing defective insulators and images of normal insulators, and establish an image dataset consisting of images containing defective insulators and images of normal insulators for training the insulator defect detection model;

[0007] Step 2, expanding the number of samples of images containing defective insulators: First, according to the characteristics of defective insulators in the image, Photoshop software is used to construct new defective insulators on other normal insulators in the insulator string. By constructing new defective insulators on different normal insulators, the number of samples of images containing defective insulators is increased; then, Photoshop software is used to process the above images containing defective insulators to obtain an image of an insulator string containing defective insulators with a transparent background; finally, Photoshop software is used to fuse the insulator string with a transparent background with the inspection images with different backgrounds to obtain defective insulator images with different backgrounds, so as to further expand the number of samples of defective insulator images and improve the generalization ability of the model;

[0008] Step 3: Label the insulator strings in the image data set and the defective insulators in the insulator strings using Labeling software: the insulator strings are labeled as insulators, and the defective insulators are labeled as defects;

[0009] Step 4: Improve the YOLOv4-tiny model target detection box optimization method: propose a dynamic non-maximum suppression method, which determines the dynamic threshold based on the statistical characteristics of the detection box around the target, and is used to replace the fixed threshold in the non-maximum suppression method to reduce the probability of missed detection of insulator strings.

[0010] Step 5: Use the improved YOLOv4-tiny model, i.e., the lightweight YOLOv4 model, to detect the overall target of the insulator string;

[0011] Step 6, cropping the detected insulator string image: cropping the insulator string according to the insulator string detection frame coordinate information obtained in step 5 to obtain an insulator string image;

[0012] Step 7, improved YOLOv4 model backbone network: construct a dual-branch channel attention module, and add the attention module after the cross-stage local network in the YOLOv4 backbone network. Its input is the output of the cross-stage local network, so that features with high correlation with insulator defects are given greater weight;

[0013] Step 8, improved YOLOv4 feature fusion network: construct an hourglass-shaped feature fusion network to achieve the fusion of shallow features and deep features of the backbone network;

[0014] Step 9, insulator defect detection based on improved YOLOv4: using the improved YOLOv4 model obtained in steps 7 and 8 to detect defective insulators from the cropped insulator string graph obtained in step 6.

[0015] Step 10, training an insulator defect detection model based on improved YOLOv4-tiny and YOLOv4: using the labeled image dataset in step 3 to train the improved YOLOV4-tiny model and the improved YOLOv4 model to obtain an insulator defect detection model;

[0016] Step 11, model application: apply the detection model obtained in step 10 to insulator defect detection, input the transmission line image data collected by the drone into the trained model, and output the insulator defect detection results.

[0017] Furthermore, in step 4, the YOLOv4-tiny model target detection box optimization method is improved, and the dynamic non-maximum suppression method is expressed as:

[0018] 1) Let the set of candidate detection frames of the insulator string be B, and the corresponding detection score set be S;

[0019] First, sort the candidate detection frames in descending order according to the scores, select the candidate detection frame with the highest score, record it as M, and move it from set B to set C;

[0020] Then, calculate the detection box M and each detection box b in the set B i The intersection-over-union ratio between them, i.e., the IoU value, is used to approximate the density of the detection boxes around the candidate detection box M using the statistical characteristics of the obtained IoU value; the mean expression of the IoU value is as follows:

[0021]

[0022] Where n is the number of candidate detection boxes in set B; the mean IoU value is N M Represents the density of detection boxes around the candidate detection box M;

[0023] Finally, the mean IoU value N M With a fixed threshold N t Compare and select the maximum value as the final dynamic suppression threshold N D , its mathematical expression is as follows:

[0024] N D =max(N M ,N t ) (2)

[0025] 2) If the candidate detection box M and the candidate detection box bi The IoU value between them is less than the dynamic threshold N D , b i The score of the candidate detection box b is retained. i is retained, otherwise b i The score is set to 0, s i The mathematical expression is as follows:

[0026]

[0027] In the formula, s i is the detection box b i If the score of the box is 0, it means that the detection box will be suppressed; repeat steps 1) and 2) until the candidate detection box set B is empty, and finally select the best candidate detection box as the optimal detection box for the detection target.

[0028] Furthermore, the improved YOLOv4 model backbone network in step 7 is to construct a dual-branch channel attention module; introduce the attention module into the YOLOv4 backbone network; the feature map in the backbone network is output through the constructed dual-branch channel attention module, and the specific process is as follows:

[0029] First, define the output feature map of the cross-stage local network module in the YOLOv4 backbone network as X, which passes through a convolution layer with a convolution kernel size of 3*3, a step size of 1, and a padding of 1, and a convolution layer with a convolution kernel size of 5*5, a step size of 1, and a padding of 2. The output feature maps are X1 and X2, respectively, forming two branches; the output feature maps X1 and X2 are added to obtain:

[0030] Y=X1+X2 (4)

[0031] Using channel global average pooling to operate on feature map Y can obtain channel-level global statistical information ω c is the cth element of ω, and its expression is:

[0032]

[0033] Where Y c is the feature map of the cth channel, H and W are the feature maps Y c After obtaining the global information, one-dimensional convolution is used to realize cross-channel interaction. The channel interaction range is determined by the size of the one-dimensional convolution kernel. The size of the channel convolution kernel is determined by the common method of the channel attention mechanism, that is,

[0034]

[0035] Where C is the number of channels, α and β are preset parameters, Indicates distance The nearest odd number; the convolution kernel size determined by formula (6) is the number of channels participating in channel interaction. The global information obtained by formula (5) is subjected to a one-dimensional convolution with a convolution kernel of k to achieve the interaction of k channel information. The output is:

[0036] v=σ(C1D k (ω)) (7)

[0037] Where C1D k is a one-dimensional convolution with kernel k, and σ(·) is a sigmoid function. In order to obtain the weights a=[a1,a2,L,a c ] and b=[b1,b2,L,b c ], we divide the channel interaction information sequence obtained by equation (7) into two parts of equal size, v1 and v2, and calculate the weight by the following equation:

[0038]

[0039]

[0040] In the formula, a i and b i are the i-th components of the weights of the two branches X1 and X2, respectively, and v 1i and v 2i are the i-th components of v1 and v2 respectively. exp() represents the exponential operation. The obtained weight is multiplied by the feature map of the corresponding branch, and then the results are added to obtain the output feature map:

[0041] O i =a i ·X 1i +b i ·X 2i (10)

[0042] In the formula, O i is the output feature map of the i-th channel, X 1i is the input feature map of the i-th channel of branch X1, X 2i It is the input feature map of the i-th channel of branch X2.

[0043] Further, the improved YOLOv4 feature fusion network in step 8 is expressed as:

[0044] First, the shallow feature map is downsampled using maximum pooling, and the feature map obtained after downsampling is fused with the feature map of the middle layer; secondly, the fused middle layer feature map is further downsampled using maximum pooling, and the feature map obtained after downsampling is fused with the feature map of the deep layer; then, on the one hand, the number of channels of the fused deep feature map is adjusted, and it is used as the final output feature map of the deep network, and on the other hand, the fused deep feature map is upsampled and fused with the fused middle layer feature map obtained in the downsampling process to obtain a new middle layer fused feature map; finally, on the one hand, the number of channels of the newly fused middle layer feature map is adjusted, and it is used as the final output feature map of the middle layer network, and on the other hand, the newly fused middle layer feature map is upsampled and fused with the shallow feature map to obtain the final output feature map of the shallow network.

[0045] Further, the insulator defect detection model training process based on the improved YOLOv4-tiny and YOLOv4 in step 10 is expressed as:

[0046] Basic parameter settings: the size of each input image is 416×416, the initial learning rate is 0.001, a step-type learning rate strategy is adopted, the network parameters are adjusted using a stochastic gradient descent optimizer with a momentum of 0.9, the weight decay is set to 0.0005, and 8 images are trained per batch;

[0047] Model training: First, the improved YOLOv4-tiny is trained only using the constructed dataset images to achieve accurate insulator detection based on the YOLOv4-tiny model; then, the trained YOLOv4-tiny model parameters are frozen, and the complete insulator defect detection model composed of the improved YOLOv4-tiny and YOLOv4 models is trained to achieve insulation string defect detection based on YOLOv4; finally, the trained YOLO-tiny model parameters are unfrozen, and based on the obtained training model parameters, the insulator defect detection model based on the improved YOLO-tiny and YOLOv4 is finally trained to obtain a network model that can detect insulator defects.

[0048] The beneficial effects of the present invention are:

[0049] (1) The method of the present invention first uses the improved YOLOv4 lightweight model YOLOv4-tiny to detect the insulator string, then cuts the detected insulator string according to the obtained detection frame information, and finally uses the improved YOLOv4 model with higher accuracy to perform defect detection on the cut insulator string; compared with the method of directly detecting insulation defects, the method is less affected by the background and has higher detection accuracy;

[0050] (2) The present invention constructs a dual-branch channel attention module and introduces the model into the YOLOv4 backbone network, assigning a larger weight to the feature channel with a higher correlation with insulator defect recognition and a smaller weight to the feature channel with a lower correlation with insulator defect recognition, thereby improving the backbone network's ability to extract insulator defect features;

[0051] (3) The present invention constructs an hourglass-shaped feature fusion network, fuses the shallow features and deep features extracted by the YOLOv4 backbone network, realizes the deep fusion of the positioning detail features of the insulator defects and the semantic features of the insulator defects, and transmits more fusion information to the feature map where the large-scale detection branch is located, which is beneficial to the recognition of smaller defects in the insulator and improves the accuracy of insulator defect recognition.

[0052] (4) The present invention determines a dynamic threshold based on the statistical characteristics of the detection frame around the insulator string target to replace the fixed threshold in the non-maximum suppression method, and proposes a dynamic non-maximum suppression method to reduce missed detections caused by the close distance between some insulator strings and the high repetition of candidate detection frames, thereby improving the accuracy of insulator string detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 It is a flowchart of the present invention;

[0054] Figure 2 It is a schematic diagram of the insulator defect detection process;

[0055] Figure 3 This is the structure diagram of the dual-branch channel attention module;

[0056] Figure 4 This is the structure diagram of the feature fusion module. DETAILED DESCRIPTION

[0057] The present invention will be further described below using the accompanying drawings and specific implementation methods.

[0058] See also Figure 1-Figure 4 Embodiment 1, this embodiment is an insulator defect detection method based on improved YOLOv4-tiny and YOLOv4, which is used to improve the accuracy of real-time detection of insulator defects. The insulator string is first detected using the improved YOLOv4-tiny model, and then the detected insulator string is trimmed. Finally, the trimmed insulator string is tested for insulator defects using the improved YOLOv4 model with higher accuracy, and finally the insulator defect detection is realized with high accuracy. The detection process is as follows: Figure 1 and Figure 2 As shown, the specific steps are as follows:

[0059] Step 1: Construct the image dataset required for insulator defect detection model training: The image data consists of defective insulator images and normal insulator images. From the transmission line images taken by drones, select images of defective insulators and normal insulators, and crop the images into 416×416 images to establish an image dataset for training the insulator defect detection model.

[0060] Step 2: Expand the number of image samples containing defective insulators: The specific process is as follows:

[0061] Firstly, according to the characteristics of defective insulators in the image, Photoshop software is used to construct new defective insulators on other normal insulators in the insulator string. By constructing new defective insulators on different normal insulators, the number of samples of defective insulator images is increased. Then, the image of the insulator string containing defective insulators is cropped, and the above image containing defective insulators is processed by Photoshop software to obtain an image of the insulator string containing defective insulators with a transparent background. Finally, Photoshop software is used to fuse the insulator string with a transparent background with the inspection images with different backgrounds to obtain defective insulator images under different backgrounds, so as to further expand the number of samples of insulator defect images and improve the generalization ability of the model.

[0062] Step 3: Label the insulator strings in the image data set and the defective insulators in the insulator strings using Labeling software: the insulator strings are labeled as insulators, and the defective insulators are labeled as defects;

[0063] Step 4: Improve the target detection frame selection method of the YOLOv4-tiny model: When two insulator string targets partially overlap or are very close to each other, the non-maximum suppression method will suppress the insulator string detection frame with a lower score, resulting in missed detection of insulator targets and affecting the detection performance of insulator strings. To solve this problem, a dynamic non-maximum suppression method is used to reduce the probability of missed detection of insulator strings: a dynamic threshold is determined based on the statistical characteristics of the detection frame around the target to replace the fixed threshold in the non-maximum suppression method.

[0064] In step 4, the target detection box selection method of the improved YOLOv4-tiny model is expressed as follows:

[0065] Assume that the set of candidate detection frames of the insulator string is B, and the corresponding detection score set is S; first, sort the candidate detection frames in descending order by score, select the candidate detection frame with the highest score as M, and move it from set B to set C. Then, calculate the detection frame M and each detection frame b in set B. iThe intersection-over-union (IoU) value between the candidate detection box M and the candidate detection box M is approximated by the statistical characteristics of the obtained IoU value. The mean expression of the IoU value is as follows:

[0066]

[0067] Where n is the number of candidate detection boxes in set B. The larger the mean of the IoU value, the greater the overlap between the detection box M and the entire set B. i The closer to the candidate detection box M, therefore, this embodiment uses the mean IoU value N M To approximate the density of the detection boxes around the candidate detection box M. Finally, the mean IoU value N M With a fixed threshold N t Compare and select the maximum value as the final dynamic suppression threshold N D , its mathematical expression is as follows:

[0068] N D =max(N M ,N t ) (2)

[0069] If the candidate detection box M is consistent with the candidate detection box b i The IoU value between them is less than the dynamic threshold N D , b i The score of the candidate detection box b is retained. i is retained, otherwise b i The score is set to 0, s i The mathematical expression is as follows:

[0070]

[0071] In the formula, s i is the detection box b i If the score of the box is 0, it means that the detection box will be suppressed; repeat the above process until the candidate detection box set B is empty, and finally select the best candidate detection box as the optimal detection box for the detection target.

[0072] Step 5, using the improved YOLOv4-tiny model (lightweight YOLOv4 model) to detect the insulator string as a whole: YOLOv4-tiny has the advantages of simple structure and faster detection speed compared with the YOLOv4 model; the insulator defect image is relatively small and is greatly affected by the background, and the accuracy of directly detecting the insulator defect is poor; the insulator string as a whole is large, the features are relatively obvious, and the detection is relatively easy. The detection method with a relatively simple model structure can also achieve good detection accuracy; therefore, this embodiment uses the improved YOLOv4-tiny model to first detect the insulator string in the image, so as to facilitate the subsequent detection of insulator defects from the detected insulator string image using a model with higher accuracy;

[0073] Step 6, cropping the detected insulator string image: cropping the insulator string according to the insulator string detection frame coordinate information obtained in step 5 to obtain an insulator string image, thereby increasing the proportion of the insulator image and reducing the influence of the background on insulator defect detection;

[0074] Step 7, improved YOLOv4 model backbone network: To address the problem that the insulator defect image is small and greatly affected by the background, a dual-branch channel attention module is constructed and introduced into the YOLOv4 backbone network, so that features with a high degree of correlation with insulator defects are given a greater weight, thereby improving the backbone network's ability to extract insulator defect features;

[0075] The improved YOLOv4 model backbone network in step 7 is expressed as:

[0076] This example constructs a dual-branch channel attention module (see Figure 3 As shown in the figure, the attention module is added to the back of the cross-stage local network in the YOLOv4 backbone network. Its input is the output of the cross-stage local network, which improves the backbone network's ability to extract insulator defect features. The feature map in the backbone network is output through the constructed dual-branch channel attention module. The specific process is as follows:

[0077] First, define the output feature map of the cross-stage local network module in the YOLOv4 backbone network as X, which passes through a convolution layer with a convolution kernel size of 3*3, a step size of 1, and a padding of 1, and a convolution layer with a convolution kernel size of 5*5, a step size of 1, and a padding of 2. The output feature maps are X1 and X2, respectively, forming two branches; add the feature maps X1 and X2 to get:

[0078] Y=X1+X2 (4)

[0079] Using channel global average pooling to operate on feature map Y can obtain channel-level global statistical information ωc is the cth element of ω, and its expression is:

[0080]

[0081] Where Y c is the feature map of the cth channel, H and W are the feature maps Y c After obtaining the global information, we use one-dimensional convolution to achieve cross-channel interaction to strengthen the information interaction between different channels. The channel interaction range is determined by the size of the one-dimensional convolution kernel. The size of the channel convolution kernel is determined by the common method of the channel attention mechanism, that is,

[0082]

[0083] Where C is the number of channels, α and β are preset parameters, Indicates distance The convolution kernel size determined by formula (6) is the number of channels participating in channel interaction. The global statistical information obtained by formula (5) can be subjected to a one-dimensional convolution with a convolution kernel of k to achieve the interaction of k channel information. The output is:

[0084] v=σ(C1D k (ω)) (7)

[0085] Where C1D k is a one-dimensional convolution with kernel k, and σ(·) is a sigmoid function. In order to obtain the weights a=[a1,a2,L,a c ] and b=[b1,b2,L,b c ], we divide the channel interaction information sequence obtained by equation (7) into two parts of equal size, v1 and v2, and calculate the weight by the following equation:

[0086]

[0087]

[0088] In the formula, a i and b i are the i-th components of the weights of the two branches X1 and X2, respectively, and v 1i and v 2i are the i-th components of v1 and v2 respectively. exp() represents the exponential operation. The obtained weight is multiplied by the feature map of the corresponding branch, and then the results are added to obtain the output feature map:

[0089] O i =a i ·X 1i+b i ·X 2i (10)

[0090] In the formula, O i is the output feature map of the i-th channel, X 1i is the input feature map of the i-th channel of branch X1, X 2i is the input feature map of the ith channel of branch X2. The more features related to insulator defects are contained in the channel, the greater the weight will be. The output feature map obtained by formula (10) contains more features related to insulator defects.

[0091] Step 8, improved YOLOv4 feature fusion network: Insulator defect targets are relatively small, and large-scale feature maps play an important role in small target detection. For this reason, an hourglass-shaped feature fusion network is constructed to achieve the fusion of shallow features and deep features of the backbone network, while transferring more fused features to the large-scale feature map to improve the insulator defect detection capability;

[0092] The improved YOLOv4 feature fusion network in step 8 is expressed as:

[0093] The constructed feature fusion network is as follows Figure 4 As shown in the figure (assuming the input image size is 416×416), the shallow feature map (feature map with a larger scale, the scale is 52×52) contains more detailed features of the insulator defect location, and the deep feature map (feature map with a smaller scale, the scale is 13×13) contains more semantic features of the insulator defect. It is proposed to first use the maximum pooling to downsample the shallow feature map, and fuse the feature map obtained after the downsampling with the feature map of the middle layer; secondly, the fused feature map of the middle layer is further downsampled by the maximum pooling, and the feature map obtained after the downsampling is fused with the feature map of the deep layer; then, on the one hand, the fused deep feature map is subjected to the On the one hand, the number of channels of the fused deep feature map is adjusted and used as the final output feature map of the deep network. On the other hand, the fused deep feature map is upsampled and fused with the fused intermediate layer feature map obtained in the downsampling process to obtain a new intermediate layer fused feature map. Finally, on the one hand, the number of channels of the newly fused intermediate layer feature map is adjusted and used as the final output feature map of the intermediate layer network. On the other hand, the newly fused intermediate layer feature map is upsampled and fused with the shallow feature map to obtain the final output feature map of the shallow network. After the above process, the feature map with a large scale will be integrated with more features, which is more conducive to the detection of insulator defects.

[0094] Step 9, insulator defect detection based on improved YOLOv4: using the improved YOLOv4 model obtained in steps 7 and 8 to detect defective insulators from the cropped insulator string graph obtained in step 6.

[0095] Step 10, training an insulator defect detection model based on improved YOLOv4-tiny and YOLOv4: using the labeled image dataset in step 3 to train the YOLOV4-tiny model and the improved YOLOv4 model to obtain a high-accuracy insulator defect detection model;

[0096] The training process of the insulator defect detection model based on the improved YOLOv4-tiny and YOLOv4 in step 10 is expressed as:

[0097] Basic parameter settings: The size of each input image is 416×416, the initial learning rate is 0.001, a step-type learning rate strategy is adopted, and the network parameters are adjusted using a stochastic gradient descent optimizer with a momentum of 0.9. The weight decay is set to 0.0005 to prevent the model from overfitting. 8 images are trained in each batch;

[0098] Model training: First, the improved YOLOv4-tiny is trained only using the constructed dataset images to achieve accurate insulator detection based on the YOLOv4-tiny model; then, the trained YOLOv4-tiny model parameters are frozen, and the complete insulator defect detection model composed of the improved YOLOv4-tiny and YOLOv4 models is trained to achieve insulation string defect detection based on YOLOv4; finally, the trained YOLO-tiny model parameters are unfrozen, and based on the obtained training model parameters, the insulator defect detection model based on the improved YOLO-tiny and YOLOv4 is finally trained to obtain a network model that can accurately detect insulator defects;

[0099] Step 11, model application, applies the detection model obtained in step 10 to insulator defect detection, inputs the transmission line image data collected by the drone into the trained model, and outputs the insulator defect detection results.

[0100] The above description is only a preferred embodiment of the present invention and does not constitute any other form of limitation to the present invention. Any modification or equivalent change made based on the technical essence of the present invention still falls within the scope of protection required by the present invention.

Claims

1. A method for detecting insulator defects based on improved YOLOv4-tiny and YOLOv4, characterized in that: The specific steps are as follows: Step 1, constructing an image dataset: from the transmission line images taken by the drone, select images containing defective insulators and images of normal insulators, and establish an image dataset consisting of images containing defective insulators and images of normal insulators for training the insulator defect detection model; Step 2, expanding the number of samples of images containing defective insulators: First, according to the characteristics of defective insulators in the image, Photoshop software is used to construct new defective insulators on other normal insulators in the insulator string. By constructing new defective insulators on different normal insulators, the number of samples of images containing defective insulators is increased; then, Photoshop software is used to process the above images containing defective insulators to obtain an image of an insulator string containing defective insulators with a transparent background; finally, Photoshop software is used to fuse the insulator string with a transparent background with the inspection images with different backgrounds to obtain defective insulator images with different backgrounds, so as to further expand the number of samples of defective insulator images and improve the generalization ability of the model; Step 3: Label the insulator strings in the image data set and the defective insulators in the insulator strings using Labeling software: the insulator strings are labeled as insulators, and the defective insulators are labeled as defects; Step 4: Improve the YOLOv4-tiny model target detection box optimization method: propose a dynamic non-maximum suppression method, which determines the dynamic threshold based on the statistical characteristics of the detection box around the target, and is used to replace the fixed threshold in the non-maximum suppression method to reduce the probability of missed detection of insulator strings. Step 5: Use the improved YOLOv4-tiny model, i.e., the lightweight YOLOv4 model, to detect the overall target of the insulator string; Step 6, cropping the detected insulator string image: cropping the insulator string according to the insulator string detection frame coordinate information obtained in step 5 to obtain an insulator string image; Step 7, improved YOLOv4 model backbone network: construct a dual-branch channel attention module and add the attention module after the cross-stage local network in the YOLOv4 backbone network. Its input is the output of the cross-stage local network, so that features with high correlation with insulator defects are given greater weights; Step 8, improved YOLOv4 feature fusion network: construct an hourglass-shaped feature fusion network to achieve the fusion of shallow features and deep features of the backbone network; Step 9, insulator defect detection based on improved YOLOv4: using the improved YOLOv4 model obtained in steps 7 and 8 to detect defective insulators from the cropped insulator string graph obtained in step 6; Step 10, training an insulator defect detection model based on improved YOLOv4-tiny and YOLOv4: using the labeled image dataset in step 3 to train the improved YOLOV4-tiny model and the improved YOLOv4 model to obtain an insulator defect detection model; Step 11, model application: apply the detection model obtained in step 10 to insulator defect detection, input the transmission line image data collected by the drone into the trained model, and output the insulator defect detection results.

2. The insulator defect detection method based on improved YOLOv4-tiny and YOLOv4 according to claim 1, characterized in that: In step 4, the target detection frame optimization method of the improved YOLOv4-tiny model is expressed as follows: 1) Let the set of candidate detection frames of the insulator string be B, and the corresponding detection score set be S; First, sort the candidate detection frames in descending order according to the scores, select the candidate detection frame with the highest score, record it as M, and move it from set B to set C; Then, calculate the detection box M and each detection box b in the set B i The intersection-over-union ratio between them, i.e., the IoU value, is used to approximate the density of the detection boxes around the candidate detection box M using the statistical characteristics of the obtained IoU value; the mean expression of the IoU value is as follows: Where n is the number of candidate detection boxes in set B; the mean IoU value is N M Represents the density of detection boxes around the candidate detection box M; Finally, the mean IoU value N M With a fixed threshold N t Compare and select the maximum value as the final dynamic suppression threshold N D , its mathematical expression is as follows: N D =max(N M ,N t ) (2) 2) If the candidate detection box M and the candidate detection box b i The IoU value between them is less than the dynamic threshold N D , b i The score of the candidate detection box b is retained. i is retained, otherwise b i The score is set to 0, s i The mathematical expression is as follows: In the formula, s i is the detection box b i If the score of the box is 0, it means that the detection box will be suppressed; repeat steps 1) and 2) until the candidate detection box set B is empty, and finally select the best candidate detection box as the optimal detection box for the detection target.

3. The insulator defect detection method based on improved YOLOv4-tiny and YOLOv4 according to claim 1, characterized in that: The improved YOLOv4 model backbone network in step 7 is to construct a dual-branch channel attention module; introduce the attention module into the YOLOv4 backbone network; the feature map in the backbone network is output through the constructed dual-branch channel attention module, and the specific process is as follows: First, define the output feature map of the cross-stage local network module in the YOLOv4 backbone network as X, which passes through a convolution layer with a convolution kernel size of 3*3, a step size of 1, and a padding of 1, and a convolution layer with a convolution kernel size of 5*5, a step size of 1, and a padding of 2. The output feature maps are X1 and X2, respectively, forming two branches; the output feature maps X1 and X2 are added to obtain: Y=X1+X2 (4) Use channel global average pooling to operate on feature map Y to obtain channel-level global statistical information ω c is the cth element of ω, and its expression is: Where Y c is the feature map of the cth channel, H and W are the feature maps Y c After obtaining the global information, one-dimensional convolution is used to realize cross-channel interaction. The channel interaction range is determined by the size of the one-dimensional convolution kernel. The size of the channel convolution kernel is determined by the common method of the channel attention mechanism, that is, Where C is the number of channels, α and β are preset parameters, Indicates distance The nearest odd number; the convolution kernel size determined by formula (6) is the number of channels participating in channel interaction. The global information obtained by formula (5) is subjected to a one-dimensional convolution with a convolution kernel of k to achieve the interaction of k channel information. The output is: v=σ(C1D k (oh)) (7) Where, C1D k is a one-dimensional convolution with kernel k, σ(·) is a sigmoid function; in order to obtain the weights a=[a1,a2,L,a of the two branches X1 and X2 from the channel interaction information c ] and b=[b1,b2,L,b c ], we divide the channel interaction information sequence obtained by (7) into two parts of equal size, v1 and v2, and calculate the weight by the following formula: In the formula, a i and b i are the i-th components of the weights of the two branches X1 and X2, respectively, and v 1i and v 2i are the i-th components of v1 and v2 respectively. exp() represents the exponential operation. The obtained weight is multiplied by the feature map of the corresponding branch, and then the results are added to obtain the output feature map: O i =a i ·X 1i +b i ·X 2i (10) In the formula, O i is the output feature map of the i-th channel, X 1i is the input feature map of the i-th channel of branch X1, X 2i It is the input feature map of the i-th channel of branch X2.

4. The insulator defect detection method based on improved YOLOv4-tiny and YOLOv4 according to claim 1, characterized in that: The improved YOLOv4 feature fusion network in step 8 is expressed as: First, the shallow feature map is downsampled using the maximum pooling, and the feature map obtained after the downsampling is fused with the feature map of the middle layer; secondly, the fused feature map of the middle layer is further downsampled using the maximum pooling, and the feature map obtained after the downsampling is fused with the feature map of the deep layer; Then, on the one hand, the number of channels of the fused deep feature map is adjusted and used as the final output feature map of the deep network, and on the other hand, the fused deep feature map is upsampled and fused with the fused intermediate layer feature map obtained in the downsampling process to obtain a new intermediate layer fused feature map; finally, on the one hand, the number of channels of the newly fused intermediate layer feature map is adjusted and used as the final output feature map of the intermediate layer network, and on the other hand, the newly fused intermediate layer feature map is upsampled and fused with the shallow feature map to obtain the final output feature map of the shallow network.

5. The insulator defect detection method based on improved YOLOv4-tiny and YOLOv4 according to claim 1, characterized in that: The insulator defect detection model training process based on the improved YOLOv4-tiny and YOLOv4 in step 10 is expressed as follows: Basic parameter settings: the size of each input image is 416×416, the initial learning rate is 0.001, a step-type learning rate strategy is adopted, the network parameters are adjusted using a stochastic gradient descent optimizer with a momentum of 0.9, the weight decay is set to 0.0005, and 8 images are trained per batch; Model training: First, the improved YOLOv4-tiny is trained only using the constructed dataset images to achieve accurate insulator detection based on the YOLOv4-tiny model; then, the trained YOLOv4-tiny model parameters are frozen, and the complete insulator defect detection model composed of the improved YOLOv4-tiny and YOLOv4 models is trained to achieve insulation string defect detection based on YOLOv4; finally, the trained YOLO-tiny model parameters are unfrozen, and based on the obtained training model parameters, the insulator defect detection model based on the improved YOLO-tiny and YOLOv4 is finally trained to obtain a network model that can detect insulator defects.

Images