Insulator defect detection method based on FPGA preprocessing and improved YOLOv5

Abstract

This invention discloses an insulator defect detection method based on FPGA preprocessing and improved YOLOv5. The method comprises the following steps: obtaining a training set and a test set of insulator samples; constructing an improved YOLOv5 insulator defect detection model; preprocessing the insulator image to be detected; training the insulator defect detection model to obtain detection results of the insulator defect image to be detected; inputting the weights trained by the insulator defect detection model into the insulator defect detection model; and then inputting the preprocessed insulator image to be detected into the insulator defect detection model to obtain detection results of the insulator defect image. This method utilizes the high-speed parallel data processing capabilities of FPGAs to improve the efficiency of image transmission and communication. It also implements a software denoising algorithm on the FPGA to rapidly reduce noise interference on insulator fault detection. An improved YOLOv5 model is then implemented using an efficient feature fusion network with self-learning weights to achieve accurate and rapid detection of insulator defects.

Description

Technical Field

[0001] The present invention relates to the technical field of image detection of high-voltage transmission line power equipment, and in particular to an insulator defect detection method based on FPGA preprocessing and improved YOLOv5. Background Art

[0002] The safety and reliability of transmission lines directly impact the stability of power transmission. As a crucial component in high-voltage transmission lines, insulator strings play a vital role in electrical insulation and mechanical support. Operating in harsh environments such as strong electric fields, cryogenic temperatures, freezing rain, and snow, insulators are prone to failures such as material aging, breakage, and chip loss. Data indicates that 81.3% of transmission line accidents are caused by insulator defects. Therefore, regular insulator inspections to identify faults and potential hazards are crucial for the safe and stable operation of transmission lines.

[0003] Traditional insulator inspection methods generally use manual inspection, but the labor intensity is high, the work efficiency is extremely low, and there are certain risks. Summary of the Invention

[0004] The present invention aims to at least partially resolve one of the technical problems in the related art. To this end, one purpose of the present invention is to propose an insulator defect detection method based on FPGA preprocessing and improved YOLOv5. This method utilizes the FPGA's high-speed parallel data processing capabilities to improve image transmission and communication efficiency, while simultaneously porting a software denoising algorithm to the FPGA for implementation to rapidly reduce noise interference with insulator fault detection. An improved YOLOv5 model is then implemented using an efficient self-learning weight feature fusion network. Weights are self-learned for insulator features of varying sizes in the input image, enhancing the model's feature fusion for small target insulators. A multi-scale self-attention mechanism is introduced into the YOLOv5 backbone network CSP architecture, enabling better acquisition of semantic information about insulator defects in the foreground as the network deepens. This enables accurate and rapid detection of insulator defects.

[0005] According to the present invention, an insulator defect detection method based on FPGA preprocessing and improved YOLOv5 is proposed, and the method steps are as follows:

[0006] S1: Obtain an insulator sample training set and a test set, obtain insulator images, randomly select a predetermined number of insulator images from the insulator images to prepare insulator images as samples, and divide the samples into a training set and a test set;

[0007] S2: Build an improved YOLOv5 insulator defect detection model. Replace YOLOv5's feature fusion network PANet with a feature fusion network BiFPN with self-learning weights. Combined with dense connections Dense and self-attention mechanism, build an insulator defect detection model.

[0008] S3: Preprocessing the image of the insulator to be detected. The image of the insulator to be detected is first preprocessed by using the FPGA to perform noise reduction algorithm;

[0009] S4: training the insulator defect detection model by inputting the insulator training samples into the insulator defect detection model for training;

[0010] S5: Obtain the detection result of the insulator defect image to be detected, input the weight trained by the insulator defect detection model into the insulator defect detection model, and then input the preprocessed insulator image to be detected into the insulator defect detection model to obtain the insulator defect image detection result.

[0011] Preferably, the method steps for constructing the improved YOLOv5 insulator defect detection model in step S2 are as follows:

[0012] S21: Replace the feature layer fusion network PANet of YOLOv5 with the BiFPN feature fusion network with self-learning weights. Use BiFPN to establish bidirectional connections between feature maps of different scales and fuse the feature information of feature maps of different scales.

[0013] The single-layer multi-scale feature list formula is:

[0014] P i P i+1 ...P i+n =f(C i C i+1 ...C i+n ) (1)

[0015] Among them, n represents the number of features used for feature fusion, C i Represents the input features of a layer, P i represents the output features of a layer, and f represents the function of the feature fusion process;

[0016] The formula for multi-layer and multi-scale feature list is:

[0017] P i P i+1 ...P i+n =f j f j+1 ...f j+m (C i C i+1 …C i+n) (2)

[0018] Where m represents the number of repeated stacking of BiFPN blocks;

[0019] In order to learn the importance of different input features, BiFPN distinguishes and fuses different input features. It is a weighted fusion mechanism. For this, BiFPN uses fast normalization processing, and its expression is as follows:

[0020]

[0021] Among them, w i Represents the weight, and uses the activation function ReLu to shrink the learnable weight to [0,1]. ε=0.00001 to avoid unstable output values. i Represents the input features, Out represents the fusion result of weighted features;

[0022] S22: Combine the densely connected Dense network to improve the BiFPN module and obtain the Dense-BiFPN network. Each node of the Dense-BiFPN network adds weights to each input feature, and upsamples the fifth layer input P5. IN and the fourth layer input P4 IN Perform weighted fusion to the fourth layer intermediate node output P4 TD , the fusion formula is as follows:

[0023]

[0024] Among them, Resize means upsampling or downsampling the input, in this case it is an upsampling operation, Conv is a convolution operation, ω4 is the learnable weight of the fourth layer output, and ω5 is the learnable weight of the fifth layer output;

[0025]

[0026] Among them, P4 out is the output of the last node of the fourth layer. Resize means upsampling or downsampling the input. Here it is a downsampling operation. ω4, ω6, and ω7 are input P4 respectively. IN 、P4 TD 、P3 out The learnable weights of

[0027] P2 out With P4 out and P5 out Dense connection, P3 out and P5 out Perform dense connection and the output expression is as follows:

[0028]

[0029] Among them, f 1x1 and f 3x3 Represents 1×1 and 3×3 convolution, U2(P2 out ) represents the feature map P2 out It is downsampled by 2 times and combined with the feature layers of BiFPN at different scales to output the fourth layer N4. out and the fifth layer N5 out The expression is as follows:

[0030]

[0031] Among them, N2 dense and N3 dense are the output of dense connections across the second layer of nodes and the output of dense connections across the third layer of nodes, respectively. out t and P5 out t are the outputs of the fourth-layer BiFPN and the fifth-layer BiFPN respectively;

[0032] S23: Add the C3TR model to improve the CSP structure of the YOLOv5 backbone network. The acquired features are aggregated and optimized through the multi-head self-attention mechanism and multi-scale feature fusion of C3TR. The calculation expression of the output vector σ of the multi-head self-attention mechanism is as follows:

[0033]

[0034]

[0035] Among them, d θ Represents the dimension of the last dimension of the query θ, Λ ζ j , Λ η j , Λ θ j , are the weight vectors for each set of values ​​ζ, keys η, and queries θ, respectively.

[0036] Preferably, in step S3, the method steps of pre-processing the insulator image to be detected using the FPGA with a noise reduction algorithm are as follows:

[0037] S31: FPGA performs fast noise reduction preprocessing on the image. The hardware algorithm implements the median filter algorithm expression as follows:

[0038] g(x,y)=Mcd{f(xl,yl),(k,l∈W)} (10)

[0039] Where f(x,y) is the pixel of the original image, g(x,y) is the pixel of the filtered image, and W is the two-dimensional model;

[0040] S32: Select a 3*3 two-dimensional model and sort to get the median;

[0041] S33: Insulator image edge enhancement: Use a 3x3 Soble edge enhancement operator to extract edge information, and then superimpose the edge information on the original image to achieve image edge enhancement.

[0042] S34: Convert the image to the HIS color space. Only the brightness channel needs to be enhanced. The color gamut and saturation channels do not need to be changed. Two sets of 3*3 templates are used as horizontal and vertical convolution kernels. For each pixel in the image, the brightness channel value is calculated, and the convolution of its neighboring pixel value and the convolution kernel is calculated. The expression of the horizontal gradient is as follows:

[0043]

[0044] The expression for the vertical gradient is as follows:

[0045]

[0046] Where A is a 3*3 pixel matrix, and its root mean square is used as the actual gradient:

[0047]

[0048] Compare the actual gradient with the set threshold. If the result is greater than the threshold, increase the brightness of the point by 50. Otherwise, the value of the point remains unchanged and then the channels are merged and converted to RGB space.

[0049] Preferably, in step S32, a 3*3 two-dimensional model is selected and the steps for selecting and sorting to obtain the median are as follows:

[0050] S321: Sort the three rows of pixels in the window respectively;

[0051] S322: Sort the three maximum values, three middle values, and three minimum values ​​of the three rows of pixels respectively;

[0052] S323: Sort the minimum value among the maximum values, the middle value among the middle values, and the maximum value of the minimum values ​​to obtain the median value.

[0053] Preferably, the method steps for training the insulator defect detection model in step S4 are as follows:

[0054] S41: Label the insulator images to create an insulator image training dataset. The dataset is divided into a training set and a test set, which are input into the insulator defect detection model for training.

[0055] S42: Set the training parameters. During the training phase, use the SGD optimizer, set the initial learning rate to 0.01, the momentum to 0.97, the final learning rate to 0.00001, the weight decay to 0.005, and use the CosinAnnealingLR method to decay the learning rate. The number of training epochs is set to 200, and the batch size of each batch is 32.

[0056] S43: Setting the size of the adaptively scaled insulator defect image to 640×640, and implementing adaptive scaling of the insulator training set and test set images based on the input image size set by the network;

[0057] S44: Comparing the effects of the improved insulator defect detection model using model training evaluation indicators. Preferably, the training evaluation indicators of the insulator defect detection model in step S44 are:

[0058] For the target detection task, the evaluation indicators include precision P, recall rate R, and average precision mAP. The mAP value is calculated according to the intersection-over-union ratio IOU = 0.5. The calculation formula is as follows:

[0059]

[0060]

[0061]

[0062] Where TP, FP, and FN represent true positive, false positive, and false negative, respectively.

[0063] The beneficial effects of the present invention are:

[0064] (1) The drone is equipped with an FPGA chip, which uses the data parallel processing capability of the FPGA to quickly reduce the noise interference in the insulator image, compress the useless information in the image, highlight the key points, reduce bandwidth usage, and improve the speed of data transmission and the accuracy of insulator recognition.

[0065] (2) A feature fusion network (BiFPN) with self-learning weights combined with dense connections (Dense) is used to improve the fusion layer of YOLOv5, further improving the fusion of high-level feature information, better balancing feature information of different scales, and performing weight self-learning on insulator features of different scales in the input image, thereby strengthening the model's feature fusion of small target insulators.

[0066] (3) The multi-scale self-attention mechanism is introduced into the CSP architecture of the YOLOv5 backbone network, which enables the network to better obtain the semantic information of the insulator defects in the foreground when deepening, and can effectively solve the problem of insufficient accuracy in insulator defect detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] In the attached figure:

[0068] Figure 1 This is a flow chart of the insulator defect detection method based on FPGA preprocessing and improved YOLOv5 proposed in the present invention;

[0069] Figure 2 This is the improved network model diagram proposed by the present invention;

[0070] Figure 3 This is a schematic diagram of the weighted feature fusion combined with dense connection module proposed in the present invention;

[0071] Figure 4 This is a densely connected network model diagram proposed by the present invention;

[0072] Figure 5 Schematic diagram of the C3TR attention module proposed in this invention;

[0073] Figure 6 Schematic diagram of the system detection platform combined with FPGA preprocessing proposed by the present invention;

[0074] Figure 7 This is a schematic diagram of the median filtering algorithm proposed in the present invention;

[0075] Figure 8 This is the FPGA preprocessing effect diagram proposed by the present invention;

[0076] Figure 9 This is a graph of the loss function proposed by the present invention;

[0077] Figure 10 This is a graph showing the average detection accuracy proposed by the present invention;

[0078] Figure 11 This is the insulator defect detection diagram proposed by the present invention. DETAILED DESCRIPTION

[0079] Reference Figure 1 , an insulator defect detection method based on FPGA preprocessing and improved YOLOv5, the method comprising the following steps:

[0080] (1) Obtain insulator images through drone aerial photography: The obtained insulator images are used to generate insulator image samples, which are divided into a training set and a test set;

[0081] (2) Build an improved YOLOv5 insulator defect detection model: replace the feature fusion network PANet of YOLOv5 with an efficient feature fusion network (BiFPN) with self-learning weights, and combine dense connection (Dense) and self-attention mechanism to build an insulator defect detection model;

[0082] (3) The insulator image to be detected is pre-processed using FPGA for noise reduction algorithm;

[0083] (4) Training the insulator defect detection model: Input the insulator training samples into the insulator defect detection model for training.

[0084] (5) Obtaining the detection result of the insulator defect image to be detected: inputting the weights trained by the insulator defect detection model into the insulator defect detection model, and then inputting the preprocessed insulator image to be detected into the insulator defect detection model to obtain the insulator defect image detection result.

[0085] The step (2) of constructing an improved YOLOv5 insulator defect detection model specifically refers to:

[0086] In order to make the YOLOv5 (you only look once v5) target detection model more suitable for insulator defect fault detection tasks, a multi-head attention (Transformer Layer) and weighted bidirectional feature pyramid (Dense-BiFPN) are proposed to improve the YOLOv5 model. The improved network model is as follows Figure 2 As shown in the figure, a highly efficient self-learning weighted feature fusion network (BiFPN) combined with dense connections (Dense) is used as an alternative to the PANet module in YOLOv5. Compared with the original PANet module in YOLOv5, BiFPN reduces the number of YOLOv5 parameters by reducing nodes with less influence in the network. Furthermore, combined with dense connections, the model's semantic information transmission capability is improved through a skip-connection approach. Weights are self-learned for insulator features of different scales in the input image, enhancing the model's feature fusion capabilities for small target insulator defects. A multi-scale self-attention mechanism is added to the top-level CSP architecture of the YOLOv5 backbone network to improve the retention of effective target information and suppress useless background information, thereby maximizing the preservation of insulator feature information. This improves YOLOv5's performance in insulator fault detection.

[0087] The neck PANet structure in the original YOLOv5 structure is a simple bidirectional fusion in the feature pyramid. Although it can achieve the fusion of shallow positioning information and deep strong semantic information of high-level feature maps, it has no relevant weighting design and lacks hierarchical evaluation of importance.

[0088] In order to solve this problem, the present invention introduces the Dense-BiFPN module, whose structure is as follows Figure 3 As shown. First, BiFPN is a simple and efficient weighted bidirectional feature pyramid network. It introduces learnable weights to learn the importance of different input features. At the same time, it repeatedly applies bottom-up and top-down multi-scale feature fusion methods to establish bidirectional connections between feature maps of different scales. To a certain extent, it solves the problem of feature information loss and can better fuse the feature information of feature maps of different scales. Replace the feature layer fusion network PANet of YOLOv5 with the BiFPN feature fusion network with self-learning weights: BiFPN can establish bidirectional connections between feature maps of different scales and fuse the feature information of feature maps of different scales. The single-layer multi-scale feature list formula is: The step (2) of constructing the improved YOLOv5 insulator defect detection model specifically refers to:

[0089] In order to make the YOLOv5 model more suitable for insulator defect fault detection tasks, multi-head attention (Transformer Layer) and weighted bidirectional feature pyramid (Dense-BiFPN) are proposed to improve the YOLOv5 model. The improved network model is as follows Figure 2 As shown in the figure, a highly efficient self-learning weighted feature fusion network (BiFPN) combined with dense connections (Dense) is used as an alternative to the PANet module in YOLOv5. Compared with the original PANet module in YOLOv5, BiFPN reduces the number of YOLOv5 parameters by reducing nodes with less influence in the network. Furthermore, combined with dense connections, the model's semantic information transmission capability is improved through a skip-connection approach. Weights are self-learned for insulator features of different scales in the input image, enhancing the model's feature fusion capabilities for small target insulator defects. A multi-scale self-attention mechanism is added to the top-level CSP architecture of the YOLOv5 backbone network to improve the retention of effective target information and suppress useless background information, thereby maximizing the preservation of insulator feature information. This improves YOLOv5's performance in insulator fault detection.

[0090] The neck PANet structure in the original YOLOv5 structure is a simple bidirectional fusion in the feature pyramid. Although it can achieve the fusion of shallow positioning information and deep strong semantic information of high-level feature maps, it has no relevant weighting design and lacks hierarchical evaluation of importance.

[0091] In order to solve this problem, the present invention introduces the Dense-BiFPN module, whose structure is as follows Figure 3As shown. First of all, BiFPN is a simple and efficient weighted bidirectional feature pyramid network. It introduces learnable weights to learn the importance of different input features. At the same time, it repeatedly applies bottom-up and top-down multi-scale feature fusion methods to establish bidirectional connections between feature maps of different scales. To a certain extent, it solves the problem of feature information loss and can better fuse the feature information of feature maps of different scales. Replace the feature layer fusion network PANet of YOLOv5 with the BiFPN feature fusion network with self-learning weights: BiFPN can establish bidirectional connections between feature maps of different scales and fuse the feature information of feature maps of different scales. The single-layer multi-scale feature list formula is:

[0092] P i P i+1 ...P i+n =f(C i C i+1 ...C i+n ) (1)

[0093] Among them, n represents the number of features used for feature fusion, C i Represents the input features of a layer, P i Represents the output features of a layer, and f represents the function of the feature fusion process.

[0094] The formula for multi-layer and multi-scale feature list is:

[0095] P i P i+1 ...P i+n =f j f j+1 ...f j+m (C i C i+1 ...C i+n ) (2)

[0096] Where m represents the number of repeated stacking of BiFPN blocks.

[0097] In order to learn the importance of different input features, BiFPN distinguishes and fuses different input features. It is a weighted fusion mechanism. For this, BiFPN uses fast normalization processing, and its expression is as follows:

[0098]

[0099] w i Represents the weight, and uses the activation function ReLu to shrink the learnable weight to [0,1]. ε=0.00001 to avoid unstable output values. i Represents the input features, and Out represents the fusion result of weighted features.

[0100] The Dense network structure is combined with the BiFPN module to improve the network feature fusion capability. The input image of the YOLOv5 model is (640, 640, 3), and the five input layers of BiFPN are P1 IN =(320,320,32), P2 IN =(160,160,64),P3 IN =(80,80,128),P4 IN =(40,40,256) and P5 IN =(20,20,512), each node of the Dense-BiFPN network adds weights to each input feature and trains these weights using a fast normalization method.

[0101] Upsampling inputs the fifth layer into P5 IN and the fourth layer input P4 IN Perform weighted fusion to the fourth layer intermediate node output P4 TD , the fusion formula is as follows:

[0102]

[0103] Among them, Resize means upsampling or downsampling the input, which is an upsampling operation here, Conv is a convolution operation, ω4 is the learnable weight of the fourth layer output, and ω5 is the learnable weight of the fifth layer output.

[0104]

[0105] Among them, P4 out is the output of the last node of the fourth layer. Resize means upsampling or downsampling the input. Here it is a downsampling operation. ω4, ω6, and ω7 are input P4 respectively. IN 、P4 TD 、P3 out The learnable weights.

[0106] Dense connection processing Figure 4 , fused feature map P i IN Different levels and deeper features P i-1 out , first downsample to keep the same i IN The same resolution is obtained, and then these features are element-wise added and merged, and finally the merged feature layer is smoothed by a 3×3 convolution layer. In order not to increase the complexity of the BiFPN structure and improve feature fusion, in the process of BiFPN downsampling, P2 out With P4out and P5 out Dense connection, P3 out and P5 out Perform dense connection. The output expression is as follows:

[0107]

[0108] where f 1x1 and f 3x3 Represents 1x1 and 3x3 convolution, U2(P2 out ) represents the feature map P2 out It is downsampled by 2 times and combined with the feature layers of BiFPN at different scales to output the fourth layer N4. out and the fifth layer N4 out The expression is as follows:

[0109]

[0110] Among them N2 dense is the output of the dense connection across the second layer of nodes, P4 out is the output of the fourth layer of BiFPN.

[0111] The C3TR model is a combination of the C3 (CSP residual network) structure in the original YOLOv5 backbone network and the TransformerEncoder (self-attention mechanism). It is an improvement on the C3 model in the original YOLOv5 network. Its structure is as follows Figure 5 As shown in the figure, the C3 structure is combined with the Transformer Encoder, and the acquired features are aggregated and optimized using the Transformer self-attention mechanism and multi-scale feature fusion. After adding the multi-head attention module, the number of feature extraction operations and model complexity are slightly increased, the weight of retaining effective target information is increased, and useless background information is suppressed, thus maximizing the retention of effective feature information. The calculation expression of the output vector σ of the multi-head self-attention mechanism is as follows:

[0112]

[0113]

[0114] Among them, d θ Represents the dimension of the last dimension of the query θ, Λ ζ j , Λ η j , Λ θ j , are the weight vectors for each set of values ​​ζ, keys η, and queries θ, respectively.

[0115] The step (3) is to pre-process the insulator image to be detected using an FPGA with a noise reduction algorithm, specifically comprising the following steps:

[0116] like Figure 6 The figure shows an aerial insulator image detection system with FPGA preprocessing. FPGAs (Field Programmable Gate Arrays) offer advantages such as short development cycles, fast computational speed, and high flexibility. They are widely used in high-precision, fast computing applications. FPGAs, combined with CPUs, further integrate hardware design with software requirements, making them a popular choice for industrial production and design. Images are captured using a high-definition camera mounted on a drone. The drone's FPGA performs rapid noise reduction preprocessing on the images. To reduce the impact of the denoising algorithm on the insulator edges, an edge enhancement algorithm is applied to enhance the edge information. Finally, the processed images are transmitted to the image processing module.

[0117] FPGA performs fast noise reduction preprocessing on the image, and the hardware algorithm implements the median filter algorithm expression as follows:

[0118] g(x,y)=Mcd{f(xl,yl),(k,l∈W)} (10)

[0119] Where f(x,y) is the pixel of the original image, g(x,y) is the pixel of the filtered image, and W is the two-dimensional model.

[0120] like Figure 7 As shown in the figure, it is the principle diagram of the median filtering algorithm. This improved method uses a 3*3 two-dimensional model.

[0121] The first step is to sort the three rows of pixels in the window respectively.

[0122] In the second step, the 3 maximum values, 3 middle values, and 3 minimum values ​​in the three rows of pixels are sorted respectively.

[0123] The third step is to sort the minimum value among the maximum values, the middle value among the middle values, and the maximum value of the minimum value to obtain the median value.

[0124] Insulator image edge enhancement uses a 3x3 Soble (an image processing technique primarily used to obtain the first-order gradient of an image, commonly used in edge detection) edge enhancement operator to extract edge information. This edge information is then superimposed on the original image to achieve image edge enhancement.

[0125] Convert the image to the HSI color space (HSI uses three parameters, H, S, and I, to describe color characteristics. H defines the color frequency, called hue; S represents the color depth, called saturation; and I represents intensity or brightness). Only the luminance channel needs to be enhanced; the color gamut and saturation channels do not need to be changed. Using two sets of 3*3 templates as horizontal and vertical convolution kernels, for each pixel in the image, the convolution of its neighboring pixel values ​​and the convolution kernel is calculated. The expression for the horizontal gradient is as follows:

[0126]

[0127] The expression for the vertical gradient is as follows:

[0128]

[0129] Where A is a 3*3 pixel matrix, and its root mean square is used as the actual gradient:

[0130]

[0131] The actual gradient is compared with the set threshold. If the result is greater than the threshold, the brightness value of the point is increased by 50. Otherwise, the value of the point remains unchanged. After enhancement, the channels are merged and converted into RGB space to achieve edge information enhancement. Figure 8 This is the effect diagram of denoising preprocessing of insulator images based on FPGA. The noise information is effectively removed and the edge information of the insulator is retained through the edge enhancement algorithm, which is conducive to the subsequent detection of insulator defects.

[0132] The step (4) of training the insulator defect detection model specifically includes the following steps:

[0133] To train the improved insulator defect model, first, insulator images are annotated, with labels set as normal insulators (insulator) and defective insulators (defect). An insulator image dataset is created and divided into a training set and a test set. The dataset is then input into the insulator defect detection model for training.

[0134] Set training parameters: During the training phase, we use the SGD optimizer (stochastic gradient descent optimizer, which performs gradient descent to update model parameters), with an initial learning rate of 0.01, a momentum of 0.97, a final learning rate of 0.00001, and a weight decay of 0.005. We also use the CosinAnnealingLR method (cosine annealing algorithm, when the loss function is small, the learning rate needs to be reduced, and the cosine function can be used to reduce the learning rate) to decay the learning rate. The number of training epochs is set to 200, and the batch size of each batch is 32;

[0135] Set the size of the adaptively scaled insulator defect image to 640×640, and implement adaptive scaling of the insulator training and test set images based on the input image size set by the network;

[0136] Training evaluation metrics for insulator defect detection models;

[0137] For target detection tasks, the main evaluation indicators include precision (P), recall (R), and average precision (mAP). The mAP value is generally calculated based on the intersection-over-union (IOU) = 0.5. The calculation formula is as follows:

[0138]

[0139]

[0140]

[0141] Where TP, FP, and FN represent true positive, false positive, and false negative, respectively.

[0142] like Figure 9 and 10 As shown in Figure 2, in order to verify the improvement effect of the improved module on the insulator segmentation performance, the above insulator dataset is used to train the model a (YOLOv5s), model b (YOLOv5s+Dense-BiFPN) and the model c (YOLOv5s+Dense-BiFPN+C3TR) of the present invention. Figure 8 As shown in Figure 2, after 200 iterations of training, the training results are basically stable at about 1%, and the training results can all converge. Comparing the loss curves of the three models, the improved model of the present invention has a faster convergence speed and a smaller convergence value. Figure 9 As shown in the figure, by introducing learnable weights to learn different input features, the mAP reached 98.9, an increase of 2.87% compared to the original model, which improved the network's detection accuracy for insulator defects and achieved better detection results.

[0143] In order to further verify the detection effect of the proposed model on real aerial insulator defects, the insulator defect images obtained by field drone aerial photography were tested. Figure 11 As shown in the figure, the improved model in this paper is effective in detecting chip-drop defects in glass insulators, achieving improved detection accuracy compared to other models in training. Wireless image transmission requires processing large amounts of image data, balancing distance and power consumption. This requires bandwidth compression while optimizing battery life. The FPGA's parallel data processing capabilities increase data transmission speed. A denoising algorithm then compresses useless image information, highlighting key points and reducing bandwidth usage, thereby improving the real-time performance of the system's insulator defect detection.

[0144] In summary, the present invention provides an insulator defect detection method based on FPGA preprocessing and improved YOLOv5.

[0145] 1) Aerial images of insulators are prone to electromagnetic, vibration, and other noise interference. Wireless communication requires processing a large amount of image data. This paper uses the FPGA chip onboard the drone to quickly reduce the noise of insulator images and compress the bandwidth, thereby improving communication efficiency and effectively enhancing the real-time performance of the entire target detection system.

[0146] 2) The insulator sizes in the network input images vary greatly, and the insulator background is complex. By introducing learnable weights through BiFPN to learn the importance of different input features, and combining it with Dense for multi-scale feature fusion, the insulator detection accuracy is improved, and the AP value reaches 99.7%.

[0147] 3) The insulator fault is very small relative to the entire input image. By improving the C3 network within the YOLOv5 backbone network and adding a Transformer module to the C3 network, the focus of feature information acquisition is placed on the insulator foreground. Combined with BiFPN multi-scale feature fusion, the probability of false detection and missed detection of faulty insulators is effectively reduced. Compared with the original model, all evaluation indicators are significantly improved, with the mAP value increasing by 2.87% to 98.9%. This invention can provide a reference for drone aerial image detection systems in high-voltage transmission lines.

Claims

1. An insulator defect detection method based on FPGA preprocessing and improved YOLOv5, characterized in that: The method steps are as follows: S1: Obtain an insulator sample training set and a test set, obtain insulator images, randomly select a predetermined number of insulator images from the insulator images to prepare insulator images as samples, and divide the samples into a training set and a test set; S2: Build an improved YOLOv5 insulator defect detection model. Replace YOLOv5's feature fusion network PANet with a feature fusion network BiFPN with self-learning weights. Combined with dense connections Dense and self-attention mechanism, build an insulator defect detection model. Combined with the densely connected Dense network to improve the BiFPN module, we get the Dense-BiFPN network. Each node of the Dense-BiFPN network adds weights to each input feature, and upsampling the fifth layer input P5 IN and the fourth layer input P4 IN Perform weighted fusion to the fourth layer intermediate node output P4 TD ; P2 out With P4 out and P5 out Dense connection, P3 out and P5 out Perform dense connection and the output expression is as follows: Among them, f 1x1 and f 3x3 Represents 1×1 and 3×3 convolution, U2(P2 out ) represents the feature map P2 out It is downsampled by 2 times and combined with the feature layers of BiFPN at different scales to output the fourth layer N4. out and the fifth layer N5 out The expression is as follows: Among them, N2 dense and N3 dense are the output of dense connections across the second layer of nodes and the output of dense connections across the third layer of nodes, P4 out and P5 out They are the output of the fourth layer BiFPN and the output of the fifth layer BiFPN respectively; The C3TR model is added to improve the CSP structure of the YOLOv5 backbone network. The acquired features are aggregated and optimized through C3TR's multi-head self-attention mechanism and multi-scale feature fusion. S3: Preprocessing the image of the insulator to be detected. The image of the insulator to be detected is first preprocessed by using the FPGA to perform noise reduction algorithm; S4: training the insulator defect detection model by inputting the insulator training samples into the insulator defect detection model for training; S5: Obtain the detection result of the insulator defect image to be detected, input the weight trained by the insulator defect detection model into the insulator defect detection model, and then input the preprocessed insulator image to be detected into the insulator defect detection model to obtain the insulator defect image detection result.

2. The insulator defect detection method based on FPGA preprocessing and improved YOLOv5 according to claim 1 is characterized in that: The method steps for constructing the improved YOLOv5 insulator defect detection model in step S2 are as follows: S21: Replace the feature layer fusion network PANet of YOLOv5 with the BiFPN feature fusion network with self-learning weights. Use BiFPN to establish bidirectional connections between feature maps of different scales and fuse the feature information of feature maps of different scales. The single-layer multi-scale feature list formula is: P i P i+1 ...P i+n =f(C i C i+1 ...C i+n ) (1) Among them, n represents the number of features used for feature fusion, C i Represents the input features of a layer, P i represents the output features of a layer, and f represents the function of the feature fusion process; The formula for multi-layer and multi-scale feature list is: P i P i+1 ...P i+n =f j f j+1 ...f j+m (C i C i+1 ...C i+n ) (2) Where m represents the number of repeated stacking of BiFPN blocks; In order to learn the importance of different input features, BiFPN distinguishes and fuses different input features. It is a weighted fusion mechanism. For this, BiFPN uses fast normalization processing, and its expression is as follows: Among them, w i Represents the weight, and uses the activation function ReLu to shrink the learnable weight to [0,1]. ε=0.00001 to avoid unstable output values. i Represents the input features, Out represents the fusion result of weighted features; S22: The fusion formula is as follows: Among them, Resize means upsampling or downsampling the input, in this case it is an upsampling operation, Conv is a convolution operation, ω4 is the learnable weight of the fourth layer output, and ω5 is the learnable weight of the fifth layer output; Among them, P4 out is the output of the last node of the fourth layer. Resize means upsampling or downsampling the input. Here it is a downsampling operation. ω4, ω6, and ω7 are input P4 respectively. IN 、P4 TD 、P3 out The learnable weights of S23: The calculation expression of the output vector σ of the multi-head self-attention mechanism is as follows: Among them, d θ Represents the dimension of the last dimension of the query θ, are the weight vectors for each set of values ​​ζ, key η, and query θ respectively.

3. The insulator defect detection method based on FPGA preprocessing and improved YOLOv5 according to claim 1 is characterized in that: In step S3, the method steps for pre-processing the insulator image to be detected using the FPGA noise reduction algorithm are as follows: S31: FPGA performs fast noise reduction preprocessing on the image. The hardware algorithm implements the median filter algorithm expression as follows: g(x,y)=Mcd{f(xl,yl),(k,l∈W)} (10) Where f(x,y) is the pixel of the original image, g(x,y) is the pixel of the filtered image, and W is the two-dimensional model; S32: Select a 3*3 two-dimensional model and sort to get the median; S33: Insulator image edge enhancement: Use a 3x3 Soble edge enhancement operator to extract edge information, and then superimpose the edge information on the original image to achieve image edge enhancement. S34: Convert the image to the HSI color space. Only the brightness channel needs to be enhanced. The color gamut and saturation channels do not need to be changed. Two sets of 3*3 templates are used as horizontal and vertical convolution kernels. For each pixel in the image, the brightness channel value is calculated, and the convolution of its neighboring pixel value and the convolution kernel is calculated. The expression of the horizontal gradient is as follows: The expression for the vertical gradient is as follows: Where A is a 3*3 pixel matrix, and its root mean square is used as the actual gradient: Compare the actual gradient with the set threshold. If the result is greater than the threshold, increase the brightness of the point by 50. Otherwise, the value of the point remains unchanged and then the channels are merged and converted to RGB space.

4. The insulator defect detection method based on FPGA preprocessing and improved YOLOv5 according to claim 3 is characterized in that: In step S32, a 3*3 two-dimensional model is selected and sorted to obtain the median value as follows: S321: Sort the three rows of pixels in the window respectively; S322: Sort the three maximum values, three middle values, and three minimum values ​​of the three rows of pixels respectively; S323: Sort the minimum value among the maximum values, the middle value among the middle values, and the maximum value of the minimum values ​​to obtain the median value.

5. The insulator defect detection method based on FPGA preprocessing and improved YOLOv5 according to claim 1, characterized in that: The method steps for training the insulator defect detection model in step S4 are as follows: S41: Label the insulator images to create an insulator image training dataset. The dataset is divided into a training set and a test set, which are input into the insulator defect detection model for training. S42: Set the training parameters. During the training phase, use the SGD optimizer, set the initial learning rate to 0.01, the momentum to 0.97, the final learning rate to 0.00001, the weight decay to 0.005, and use the CosinAnnealingLR method to decay the learning rate. The number of training epochs is set to 200, and the batch size of each batch is 32. S43: Setting the size of the adaptively scaled insulator defect image to 640×640, and implementing adaptive scaling of the insulator training set and test set images based on the input image size set by the network; S44: Compare the effects of the improved insulator defect detection model through model training evaluation indicators.

6. The insulator defect detection method based on FPGA preprocessing and improved YOLOv5 according to claim 1, characterized in that: The training evaluation index of the insulator defect detection model in step S44 is: For the target detection task, the evaluation indicators include precision P, recall rate R, and average precision mAP. The mAP value is calculated according to the intersection-over-union ratio IOU = 0.

5. The calculation formula is as follows: Where TP, FP, and FN represent true positive, false positive, and false negative, respectively.

Images