Power transmission line tower defect detection method and system based on improved Yolov5

Abstract

The invention relates to an improved Yov5-based power transmission line tower defect detection method and system, and the method comprises the steps: firstly constructing a data set based on the collection of the image data of a to-be-detected power transmission tower, and then carrying out the detection of the defects of the to-be-detected power transmission tower in a backbone network of a Yov5 algorithm; using deformable convolution to replace conventional convolution, simplifying a spatial pyramid pool module SimSPPF and fusing with an ECA attention mechanism, and using an SIoU loss function to define a penalty mechanism; through the above means, the Yolov5 algorithm is improved, a target detection model is constructed, and finally the model is trained and optimized for defect detection. According to the method, a larger receptive field can be obtained through deformable convolution, so that defect features of the power transmission line tower can be better captured, a spatial pyramid pool module is simplified and fused with an ECA attention mechanism, the calculation burden is relieved, the defect features captured by the deformable convolution can be further screened and strengthened, and the accuracy of the defect features is improved. And then a punishment mechanism is redefined by using an SIoU loss function, so that the precision of defect detection of the transmission tower under a complex background is effectively improved.

Description

Technical Field

[0001] The invention relates to a defect detection method, belongs to the technical field of transmission line tower detection, and particularly relates to a transmission line tower defect detection method and system based on improved Yolov5. Background Art

[0002] As an important part of the power system, transmission line towers are responsible for the transmission and distribution of electric energy and play an important role in the power system. However, since transmission line towers are outdoors and are easily affected by external environmental factors, various components may have some hidden faults. Once problems occur, they will cause great economic losses. Therefore, in order to ensure the normal operation of transmission lines, it is of great significance to study the defect detection method of transmission line towers.

[0003] In the existing technical literature, there are already many applications of defect detection using neural networks; for example: using multi-scale features and context information based on deep neural networks to identify bolt defects in transmission tower images, automatically detecting bird's nests on transmission line towers based on Faster-RCNN convolutional neural networks, etc. However, the above methods often focus on detecting defects in a single position or type, and have poor generalization capabilities and low inspection accuracy. However, in actual applications, the defects of transmission line towers are diverse. Therefore, there is an urgent need for a method that can comprehensively detect multiple defects of transmission line towers and meet the requirements of speed and accuracy of transmission line tower inspections. Summary of the invention

[0004] The purpose of the present invention is to overcome the above-mentioned defects and problems existing in the prior art and to provide a comprehensive and accurate transmission line tower defect detection method and system based on improved Yolov5.

[0005] To achieve the above objectives, the technical solution of the present invention is: a method for detecting defects of transmission line towers based on improved Yolov5, comprising:

[0006] S1. Based on the UAV inspection, the image data of the transmission tower to be inspected is collected to construct a data set of the transmission line tower; the data set includes: images of the transmission line tower in a normal state and images of the transmission line tower with various defects;

[0007] S2. Improve the Yolov5 algorithm and build a target detection model based on the improved Yolov5 algorithm;

[0008] The improvement of the Yolov5 algorithm includes: in the backbone network of the Yolov5 algorithm, using deformable convolution instead of conventional convolution, simplifying the spatial pyramid pooling module SimSPPF and fusing it with the ECA attention mechanism, and using the SIoU loss function to define the penalty mechanism;

[0009] S3. Input the data set of the transmission line tower into the target detection model, train and optimize the target detection model, and perform defect detection on the transmission line tower based on the optimized target detection model.

[0010] In step S2, deformable convolution is used instead of conventional convolution, specifically referring to:

[0011] Use conventional convolution operations to extract the displacement value corresponding to the offset of the image data, and merge the displacement value into the conventional convolution operation. At the same time, optimize the adjustment mechanism of the variable convolution and output the image features.

[0012] The expression for incorporating the displacement value into the regular convolution operation is as follows:

[0013]

[0014] Among them: y(p0) is the point on the output image feature map, p0 is the center point of the convolution kernel in the conventional convolution operation, p n is the offset of the conventional convolution operation, Δp n is the offset of the variable convolution, w(·) is the weight of the corresponding offset point, x(p0+p n +Δp n ) is the input transmission tower image data to be detected at position p n The value in ;

[0015] The expression of the adjustment mechanism for optimizing variable convolution is as follows:

[0016]

[0017] Where: Δm n is the modulation scalar.

[0018] In step S2, simplifying the spatial pyramid pooling module SimSPPF specifically refers to:

[0019] The input of the spatial pyramid pooling module SPPF is divided into the first part and the second part; the first part is connected in series with three maximum pooling layers of size 5×5, and the output of the second part is connected to the output of the first part through Concat; at the same time, the ReLU activation function is used in the simplified spatial pyramid pooling module SimSPPF.

[0020] In step S2, the ECA attention mechanism specifically refers to:

[0021] After performing global average pooling on image features through the ECA attention mechanism, a one-dimensional convolution operation with k kernels is used to adaptively allocate the weight proportion of the input channels;

[0022] The size of the k-core is obtained through an adaptive function, and its expression is as follows:

[0023]

[0024] Among them: k is the size of the convolution kernel obtained, c is the number of channels, γ and b are adjustment parameters; | | odd is the odd number closest to the function value.

[0025] In step S2, the SIoU loss function includes an angle loss function, a distance loss function, a shape loss function, and an IoU loss function;

[0026] The expression of the angle loss function is as follows:

[0027]

[0028] Where: Λ is the angle loss function, c h is the height difference between the center point of the real box and the predicted box, σ is the distance between the center point of the real box and the predicted box, and π is a constant;

[0029]

[0030] Where: α is the angle, is the center coordinate of the real frame, is the center coordinate of the prediction box;

[0031] The expression of the distance loss function is as follows:

[0032]

[0033] γ=2-Λ;

[0034] Where: Δ is the distance loss function, p x It is the square of the ratio of the horizontal distance between the center point of the real box and the predicted box to the width of the minimum bounding rectangle. is the deviation of the prediction box in the horizontal direction, p y It is the square of the ratio of the vertical distance between the center point of the predicted box and the real box to the height of the minimum enclosing rectangle. is the deviation of the prediction box in the vertical direction, c w 、c h are the width and height of the rectangle enclosed by the coordinates of the center points of the real box and the predicted box, respectively, and γ is the weight for adjusting the distance loss. is the degree of deviation of the prediction box in the t direction;

[0035] The expression of the shape loss function is as follows:

[0036]

[0037] Where: Ω is the shape loss function, (w, h) is the width and height of the prediction box, (w gt ,h gt ) is the width and height of the real box, θ is used to control the degree of attention to shape loss, w w is the ratio of the absolute value of the width difference between the predicted box and the real box to the maximum value, w h It is the ratio of the absolute value of the height difference between the predicted box and the real box to the maximum value. is the width loss, For height loss;

[0038] The expression of the IoU loss function is as follows:

[0039]

[0040] Among them: IoU is the IoU loss function, intersection A is the intersection area of ​​the predicted box and the real box, and union B is the union area of ​​the predicted box and the real box;

[0041] Then the expression of the SIoU loss function is as follows:

[0042]

[0043] Among them: Loss SIoU is the SIoU loss function, Ω is the shape loss function, and Δ is the distance loss function.

[0044] A transmission line tower defect detection system based on improved Yolov5, the system comprising:

[0045] A data set construction module is used to collect image data of transmission towers to be inspected based on drone inspections to construct a data set of transmission line towers; the data set includes: images of transmission line towers in normal states and images of transmission line towers with various defects;

[0046] Algorithm improvement module, used to improve the Yolov5 algorithm and build a target detection model based on the improved Yolov5 algorithm;

[0047] The improvement of the Yolov5 algorithm includes: in the backbone network of the Yolov5 algorithm, using deformable convolution instead of conventional convolution, simplifying the spatial pyramid pooling module SimSPPF and fusing it with the ECA attention mechanism, and using the SIoU loss function to define the penalty mechanism;

[0048] The defect detection module is used to input the data set of the transmission line tower into the target detection model, train and optimize the target detection model, and perform defect detection on the transmission line tower based on the optimized target detection model.

[0049] The algorithm improvement module improves the Yolov5 algorithm according to the following method:

[0050] Use conventional convolution operations to extract the displacement value corresponding to the offset of the image data, and merge the displacement value into the conventional convolution operation. At the same time, optimize the adjustment mechanism of the variable convolution and output the image features.

[0051] The expression for incorporating the displacement value into the regular convolution operation is as follows:

[0052]

[0053] Among them: y(p0) is the point on the output image feature map, p0 is the center point of the convolution kernel in the conventional convolution operation, p n is the offset of the conventional convolution operation, Δp n is the offset of the variable convolution, w(·) is the weight of the corresponding offset point, x(p0+p n +Δp n ) is the input transmission tower image data to be detected at position p n The value in ;

[0054] The expression of the adjustment mechanism for optimizing variable convolution is as follows:

[0055]

[0056] Where: Δm n is the modulation scalar.

[0057] The input of the spatial pyramid pooling module SPPF is divided into the first part and the second part; the first part is connected in series with three maximum pooling layers of size 5×5, and the output of the second part is connected to the output of the first part through Concat; at the same time, the ReLU activation function is used in the simplified spatial pyramid pooling module SimSPPF.

[0058] After performing global average pooling on image features through the ECA attention mechanism, a one-dimensional convolution operation with k kernels is used to adaptively allocate the weight proportion of the input channels;

[0059] The size of the k-core is obtained through an adaptive function, and its expression is as follows:

[0060]

[0061] Among them: k is the size of the convolution kernel obtained, c is the number of channels, γ and b are adjustment parameters; | |odd is the odd number closest to the function value.

[0062] The SIoU loss function includes angle loss function, distance loss function, shape loss function, and IoU loss function;

[0063] The expression of the angle loss function is as follows:

[0064]

[0065] Where: Λ is the angle loss function, c h is the height difference between the center point of the real box and the predicted box, σ is the distance between the center point of the real box and the predicted box, and π is a constant;

[0066]

[0067] Where: α is the angle, is the center coordinate of the real frame, is the center coordinate of the prediction box;

[0068] The expression of the distance loss function is as follows:

[0069]

[0070] γ=2-Λ;

[0071] Where: Δ is the distance loss function, p x It is the square of the ratio of the horizontal distance between the center point of the real box and the predicted box to the width of the minimum enclosing rectangle. is the deviation of the prediction box in the horizontal direction, p y It is the square of the ratio of the vertical distance between the center point of the predicted box and the real box to the height of the minimum enclosing rectangle. is the deviation of the prediction box in the vertical direction, c w 、c h are the width and height of the rectangle enclosed by the coordinates of the center points of the real box and the predicted box, respectively, and γ is the weight for adjusting the distance loss. is the degree of deviation of the prediction box in the t direction;

[0072] The expression of the shape loss function is as follows:

[0073]

[0074]

[0075] Where: Ω is the shape loss function, (w, h) is the width and height of the prediction box, (w gt ,h gt) is the width and height of the real box, θ is used to control the degree of attention to shape loss, w w is the ratio of the absolute value of the width difference between the predicted box and the real box to the maximum value, w h It is the ratio of the absolute value of the height difference between the predicted box and the real box to the maximum value. is the width loss, For height loss;

[0076] The expression of the IoU loss function is as follows:

[0077]

[0078] Among them: IoU is the IoU loss function, intersection A is the intersection area of ​​the predicted box and the real box, and intersection B is the union area of ​​the predicted box and the real box;

[0079] Then the expression of the SIoU loss function is as follows:

[0080]

[0081] Among them: Loss SIoU is the SIoU loss function, Ω is the shape loss function, and Δ is the distance loss function.

[0082] Compared with the prior art, the present invention has the following beneficial effects:

[0083] The invention discloses a transmission line tower defect detection method and system based on improved Yolov5. The method first collects transmission line tower image data to be detected based on unmanned aerial vehicle inspection to construct a data set of transmission line towers, then uses deformable convolution to replace conventional convolution in the backbone network of Yolov5 algorithm, simplifies the spatial pyramid pool module SimSPPF and integrates it with an ECA attention mechanism, and adopts a SIoU loss function to define a penalty mechanism; the Yolov5 algorithm is improved and a target detection model is constructed by the above means, and finally the target detection model is trained and optimized to realize defect detection; in the application of the design, a larger receptive field can be obtained through deformable convolution, so that the defect characteristics of the transmission line tower can be better captured, and the spatial pyramid pool module is simplified and integrated with the ECA attention mechanism, while reducing the calculation burden, the defect characteristics captured by the deformable convolution can be further screened and strengthened, and then the SIoU loss function is used to redefine the penalty mechanism, thereby effectively improving the accuracy of transmission tower defect detection under complex backgrounds. BRIEF DESCRIPTION OF THE DRAWINGS

[0084] Figure 1 It is a flow chart of the steps of the method of the present invention.

[0085] Figure 2It is a flow chart of transmission line tower defect detection using the improved Yolov5 algorithm in the present invention.

[0086] Figure 3 It is a network model structure diagram of the improved Yolov5 algorithm in the present invention.

[0087] Figure 4 It is a schematic diagram of the influence of various improved modules in Example 1 of the present invention on the network model.

[0088] Figure 5 It is a schematic diagram comparing defect detection results based on different algorithms in Example 1 of the present invention.

[0089] Figure 6 It is a schematic diagram of the system structure of the present invention.

[0090] Figure 7 It is a schematic diagram of the device structure of the present invention.

[0091] In the figure: data set construction module 1, algorithm improvement module 2, defect detection module 3, processor 4, memory 5, computer program code 51. DETAILED DESCRIPTION

[0092] The present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0093] Embodiment 1:

[0094] See also Figure 1-Figure 2 , a transmission line tower defect detection method based on improved Yolov5, comprising:

[0095] S1. Based on the UAV inspection, the image data of the transmission tower to be inspected is collected to construct a data set of the transmission line tower; the data set includes a training set and a test set; the training set and the test set both include: images of the transmission line tower in a normal state and images of the transmission line tower with various defects;

[0096] S2. Improve the Yolov5 algorithm and build a target detection model based on the improved Yolov5 algorithm;

[0097] The improvement of the Yolov5 algorithm includes: in the backbone network of the Yolov5 algorithm, using deformable convolution instead of conventional convolution, simplifying the spatial pyramid pooling module SimSPPF and fusing it with the ECA attention mechanism, and using the SIoU loss function to define the penalty mechanism;

[0098] Furthermore, in step S2, deformable convolution is used instead of conventional convolution, specifically referring to:

[0099] When using conventional convolution for image sampling, the same receptive field is used to extract features from objects of different sizes and irregular shapes; this feature extraction method is not ideal, especially for tower defects with large size variations or irregular shapes; deformable convolution is used in this solution to address some of the limitations of conventional convolution. It can be clearly seen from the convolution operation that compared with conventional convolution, deformable convolution can obtain a larger receptive field, and the receptive field range during the sampling process can change with the change of the size of the detected target, making the feature extraction process more consistent with the actual detection object.

[0100] The input features are first passed through the convolution layer to obtain the offset field, where N represents the size of the convolution kernel, resulting in N offsets; since the image is a 2D image, there are a total of 2N offsets, which correspond to displacement values, and then these displacement values ​​are incorporated into the calculation of conventional convolution, which can achieve the purpose of expanding the receptive field, thereby ultimately completing the task of matching the size of the detected target.

[0101] The expression for incorporating the displacement value into the regular convolution operation is as follows:

[0102]

[0103] Among them: y(p0) is the point on the output image feature map, p0 is the center point of the convolution kernel in the conventional convolution operation, p n is the offset of the conventional convolution operation, Δp n is the offset of the variable convolution, w(·) is the weight of the corresponding offset point, x(p0+p n +Δp n ) is the input transmission tower image data to be detected at position p n The value in ;

[0104] R∈{(-1,-1),(-1,0),...,(0,1),(1,1)} is the receptive field of the convolution kernel, indicating the area covered by the convolution kernel;

[0105] In order to further enhance the network's ability to control the spatial support area, an adjustment mechanism is introduced. Through the adjustment mechanism, the network can decide how to adjust the feature amplitudes of different positions or regions during the learning process, thereby further adjusting the perception and response to the input features; this means that before the module outputs, the network can make more fine-grained adjustments to the input features, adjusting the feature amplitudes of different positions or regions to values ​​that are more suitable for the network task. In extreme cases, the adjustment mechanism allows the network to adjust the feature amplitudes of certain positions or regions to zero, so that the module can selectively not perceive signals from specific positions or regions, allowing the module to reduce or completely eliminate the impact of image content from certain spatial positions on the output. This flexibility enables the network module to better adapt to different scenarios and task requirements, and enhances its ability to adjust the spatial support area.

[0106] The expression of the adjustment mechanism for optimizing variable convolution is as follows:

[0107]

[0108] Where: Δm n is the modulation scalar.

[0109] Preferably, since the generated offset is usually a floating point number, bilinear interpolation is used to obtain the corresponding pixel value after the offset.

[0110] By replacing regular convolutions with deformable convolutions in the backbone network of YOLOv5, the network can more flexibly adjust its receptive field to match the actual size and shape of the tower defects; this flexibility brings a larger receptive field, allowing the network to capture richer contextual information, which helps to accurately locate and identify tiny cracks, corrosion, distortion and other defects on the towers, and provide rich feature inputs for the ECA attention module in the backbone network.

[0111] Furthermore, in step S2, simplifying the spatial pyramid pool module SimSPPF specifically refers to:

[0112] The input of the spatial pyramid pooling module SPPF is divided into the first part and the second part; the first part is connected in series with three maximum pooling layers of size 5×5, and the output of the second part is connected to the output of the first part through Concat; at the same time, the ReLU activation function is used in the simplified spatial pyramid pooling module SimSPPF.

[0113] Spatial pyramid pooling (SPP) is a pooling method that maps local features to spaces of different dimensions and fuses them. Inputting images of different sizes can generate feature vectors of fixed sizes, thereby effectively extracting feature information of different scales.

[0114] Although the SPPF module has a good performance, the number of parameters and the amount of calculation are too large. For this reason, this scheme simplifies the SPPF module and divides the input into two parts. One part is connected in series with three maximum pooling layers of size 5×5 (Max Pooling). Among them, the calculation results of connecting two 5×5 maximum pooling layers and one 9×9 maximum pooling layer are the same as those of connecting three 5×5 maximum pooling layers and one 13×13 maximum pooling layer. By connecting the maximum pooling layer in series to process the input, the amount of calculation is greatly reduced, and the speed and efficiency are improved. The other part of the input is directly connected to the output of the three pooling operations through Concat, which effectively expands the network's receptive field and enhances the perception of the tower defect characteristics, especially the recognition of targets in complex scenarios. In addition, this scheme uses the ReLU activation function in the SimSPPF module, while the SPPF module uses the SiLU activation function. Therefore, compared with SPPF, using SimSPPF can produce faster detection speed.

[0115] Furthermore, in step S2, the ECA attention mechanism specifically refers to:

[0116] After performing global average pooling on image features through the ECA attention mechanism, a one-dimensional convolution operation with k kernels is used to adaptively allocate the weight proportion of the input channels;

[0117] The size of the k-core is obtained through an adaptive function, and its expression is as follows:

[0118]

[0119] Where: k is the size of the convolution kernel obtained, c is the number of channels, γ is the adjustment parameter (preferably set to 2); b is the adjustment parameter (preferably set to 1); | | odd is the odd number closest to the function value.

[0120] The SE attention mechanism is a method that allows a neural network to automatically determine the importance of feature channels and assign different weights accordingly, aiming to achieve better detection accuracy; this scheme utilizes the ECA attention mechanism, which is an improved version of the SE attention mechanism.

[0121] In SENet, it is necessary to consider the relationship between all channels to achieve weight distribution. However, the SE attention mechanism will incur a lot of computational costs. The ECA attention mechanism solves this problem by replacing the fully connected layer with a convolutional layer. After adding the ECA attention mechanism, the total parameters of the model are significantly reduced. In contrast, the ECA attention mechanism only requires a small increase in parameters to obtain good detection results.

[0122] The ECA attention mechanism first performs global average pooling (GAP) on the feature map extracted by the deformable convolution, and then uses a one-dimensional convolution operation with a k-kernel, which can adaptively assign weights to the input channel c and has good cross-channel information acquisition capabilities. This enables the model to pay more attention to those features that are more important for detecting tower defects; the size of the convolution kernel is obtained through an adaptive function.

[0123] Combining the above two improvements, the ECA attention mechanism and the simplified pyramid pooling module are integrated in the backbone network of YOLOv5, which can reduce the computational complexity of the model while improving the ability to extract key defect features of the tower, and can provide a more accurate feature basis for the SIoU loss function, thereby making the prediction of the bounding box more accurate.

[0124] Furthermore, the penalty mechanism is redefined using the SIoU loss function, and the angle between the predicted box and the actual box is considered to improve the accuracy of transmission tower defect detection in complex backgrounds.

[0125] Traditional IoU calculation methods do not take into account the direction of mismatch between the desired true box and the predicted box. This deficiency leads to slower convergence and lower efficiency, because the predicted box may "wander around" during training and ultimately produce a worse model; this scheme proposes a new loss function SIoU, in which the penalty indicator is redefined based on the vector angle between the expected regressions.

[0126] In step S2, the SIoU loss function includes an angle loss function, a distance loss function, a shape loss function, and an IoU loss function;

[0127] The expression of the angle loss function is as follows:

[0128]

[0129] Where: Λ is the angle loss function, c h is the height difference between the center of the real box and the predicted box, σ is the distance between the center of the real box and the predicted box, and π is a constant (i.e., pi); in fact is equal to the angle α;

[0130]

[0131] Where: α is the angle, is the center coordinate of the real frame, is the center coordinate of the prediction box; in the application, it can be noted that when α is Or 0, the angle loss is 0; during the training process, if First try to minimize α, otherwise minimize

[0132] The expression of the distance loss function is as follows:

[0133]

[0134] γ=2-Λ;

[0135] Where: Δ is the distance loss function, p x It is the square of the ratio of the horizontal distance between the center point of the real box and the predicted box to the width of the minimum enclosing rectangle. is the deviation of the prediction box in the horizontal direction, p y It is the square of the ratio of the vertical distance between the center point of the predicted box and the real box to the height of the minimum enclosing rectangle. is the deviation of the prediction box in the vertical direction, c w 、c h are the width and height of the rectangle enclosed by the coordinates of the center points of the real box and the predicted box, respectively, and γ is the weight for adjusting the distance loss. is the deviation of the prediction box in the t direction (t = x, y, x is the horizontal direction, y is the vertical direction);

[0136] The expression of the shape loss function is as follows:

[0137]

[0138] Where: Ω is the shape loss function, (w, h) is the width and height of the prediction box, (w gt ,h gt ) is the width and height of the real box, θ is used to control the degree of attention to shape loss, w w is the ratio of the absolute value of the width difference between the predicted box and the real box to the maximum value, w h It is the ratio of the absolute value of the height difference between the predicted box and the real box to the maximum value. is the width loss, For height loss;

[0139] In order to avoid focusing too much on shape loss and reducing the movement of the prediction box during application, a genetic algorithm is used to calculate θ close to 4. Therefore, the parameter range of θ is [2,6].

[0140] The expression of the IoU loss function is as follows:

[0141]

[0142] Among them: IoU is the IoU loss function, intersection A is the intersection area of ​​the predicted box and the real box, and union B is the union area of ​​the predicted box and the real box;

[0143] Then the expression of the SIoU loss function is as follows:

[0144]

[0145] Among them: Loss SIoU is the SIoU loss function, Ω is the shape loss function, and Δ is the distance loss function.

[0146] In the detection network of YOLOv5, the use of SIoU loss function significantly improves the accuracy of transmission line tower defect detection. When processing the feature map fused by the neck network, the function fully considers the four key factors of angle, distance, shape and IOU loss, thereby accurately predicting the bounding box of the tower defect.

[0147] S3. Input the data set of the transmission line tower into the target detection model, train and optimize the target detection model, and perform defect detection on the transmission line tower based on the optimized target detection model.

[0148] Further, the following steps are included:

[0149] S31, collecting image data of transmission towers to be inspected based on drone inspection, constructing a data set of transmission line towers, and taking the data set as input;

[0150] The dataset includes images of normal towers and images of towers with various defects (such as cracks, corrosion, foreign matter, etc.). Each image is equipped with a corresponding annotation file in XML format, which describes the detailed information of the target object in the image, including the tower defect type, bounding box coordinates, tower defect size, and target visibility.

[0151] S32, setting the training parameters of the target detection model and initializing the network structure, and then performing model training until the loss function is judged to converge, and then outputting the optimized target detection model;

[0152] S33. The optimized target detection model detects the location and category of the tower defects and outputs the detection results.

[0153] Specifically, the processing steps of the optimized target detection model are as follows:

[0154] The data set of transmission line towers is input into the target detection model. The backbone network extracts features from the image data of the training set based on deformable convolution and ECA attention modules, and uses SimSPPF to reduce the computational burden. The features extracted by the backbone network are then fused through the neck network. Finally, the gradient of the SIOU loss function is used to update the network weights, and the optimization is continuously iterated until the optimal hyperparameters are found.

[0155] The optimal hyperparameter combination is input into the model to obtain a trained transmission line tower defect detection model, and the test set is used to verify the effectiveness of the trained model.

[0156] The transmission tower images taken by the drone are transferred to the trained target detection model to perform defect detection on the transmission line towers. Finally, the coordinates of the bounding box, the detected tower fault category, and the confidence score corresponding to the fault category are output.

[0157] In this embodiment, the improved network architecture ECA-SimSPPF-SIoU-YOLOv5 is as follows Figure 3 As shown. Several convolutions in the skeleton and neck have been replaced by deformable convolutions, the SPP structure in the backbone has been replaced by the SimSPPF structure, and two layers of SimSPPF structures have been added. The ECA attention mechanism has been incorporated into the backbone network, and the loss function has been replaced by SIoU. In the figure, the CONV module performs convolution, BN, and activation function operations on the input feature map; DCNv2 is a deformable convolution, which increases the receptive field during the sampling process; SimSPPF is a simplified SPPF, and SimSPPF uses the ReLU activation function to reduce the computational complexity; ECA is a channel attention mechanism used to enhance the network's ability to model the relationship between channels; Concat is used for the fusion of feature maps to improve the performance of target detection; Upsampling is an upsampling module used to fuse smaller feature maps with larger feature maps; Detect is a detection module; The C3 module is transformed from the BottleneckCSP (bottleneck layer) module, and its structure and function are basically the same. They are all CSP architectures, but they are different in the choice of correction units. The C3 module contains 3 standard convolutional layers and multiple Bottleneck modules.

[0158] The C3 module is a network structure module used in deep learning, especially in computer vision tasks; C3 usually refers to the abbreviation of three Cs (i.e. three convolution modules). This module is designed to improve the feature extraction capability and computational efficiency of the network while maintaining low model complexity.

[0159] In this embodiment, see Figure 4 ; In order to verify the effectiveness of each improved module in this scheme, multiple ablation experiments were carried out using the transmission line tower image dataset; each improved module was evaluated using the controlled variable method. Figure 4As shown, the left Y axis represents the size of the mAP value, and different improvement methods are represented by black dot line graphs; the right Y axis represents the size of the parameters, and different improvement methods are represented by square line graphs; it can be clearly seen from the figure that each improvement can increase the mAP of the model, and when all improvements are added, the highest mAP of 91.4% is achieved. This is significantly better than the model with only one type of improvement. In addition, compared with the YOLOv5s model, the model only increases the parameter size by less than 1%.

[0160] See also Figure 5 ,Depend on Figure 5 From the data in, we can see that the proposed algorithm has improved the detection accuracy of common defects of transmission line towers compared with the other three algorithms, among which the detection accuracy of tower crack defects is 87.1%, the detection accuracy of tower corrosion defects is 91.2%, the detection accuracy of tower deformation defects is 89.2%, and the detection accuracy of plant and animal damage defects is 98.4%. This is because the proposed method adds deformation convolution to make the model closely aligned with the shape of the defect during sampling, which helps to extract high-level features; in addition, the addition of the ECA attention mechanism effectively amplifies the importance of key information in the detection process, thereby improving the detection accuracy; finally, by substituting the SIoU loss function, this scheme takes into account the angle and effectively solves the bounding box regression problem, making the defect detection more comprehensive and the detection accuracy higher.

[0161] Embodiment 2:

[0162] See also Figure 6 , a transmission line tower defect detection system based on improved Yolov5, the system comprising:

[0163] The data set construction module 1 is used to collect the image data of the transmission tower to be inspected based on the drone inspection, and to construct a data set of the transmission line tower; the data set includes: images of the transmission line tower in a normal state and images of the transmission line tower with various defects;

[0164] Algorithm improvement module 2 is used to improve the Yolov5 algorithm and build a target detection model based on the improved Yolov5 algorithm;

[0165] The improvement of the Yolov5 algorithm includes: in the backbone network of the Yolov5 algorithm, using deformable convolution instead of conventional convolution, simplifying the spatial pyramid pooling module SimSPPF and fusing it with the ECA attention mechanism, and using the SIoU loss function to define the penalty mechanism;

[0166] Furthermore, the algorithm improvement module 2 improves the Yolov5 algorithm according to the following method:

[0167] Use conventional convolution operations to extract the displacement value corresponding to the offset of the image data, and merge the displacement value into the conventional convolution operation. At the same time, optimize the adjustment mechanism of the variable convolution and output the image features.

[0168] The expression for incorporating the displacement value into the regular convolution operation is as follows:

[0169]

[0170] Among them: y(p0) is the point on the output image feature map, p0 is the center point of the convolution kernel in the conventional convolution operation, p n is the offset of the conventional convolution operation, Δp n is the offset of the variable convolution, w(·) is the weight of the corresponding offset point, x(p0+p n +Δp n ) is the input transmission tower image data to be detected at position p n The value in ;

[0171] The expression of the adjustment mechanism for optimizing variable convolution is as follows:

[0172]

[0173] Where: Δm n is the modulation scalar.

[0174] The input of the spatial pyramid pooling module SPPF is divided into the first part and the second part; the first part is connected in series with three maximum pooling layers of size 5×5, and the output of the second part is connected to the output of the first part through Concat; at the same time, the ReLU activation function is used in the simplified spatial pyramid pooling module SimSPPF.

[0175] After performing global average pooling on image features through the ECA attention mechanism, a one-dimensional convolution operation with k kernels is used to adaptively allocate the weight proportion of the input channels;

[0176] The size of the k-core is obtained through an adaptive function, and its expression is as follows:

[0177]

[0178] Among them: k is the size of the convolution kernel obtained, c is the number of channels, γ and b are adjustment parameters; | | odd is the odd number closest to the function value.

[0179] The SIoU loss function includes angle loss function, distance loss function, shape loss function, and IoU loss function;

[0180] The expression of the angle loss function is as follows:

[0181]

[0182] Where: Λ is the angle loss function, c h is the height difference between the center point of the real box and the predicted box, σ is the distance between the center point of the real box and the predicted box, and π is a constant;

[0183]

[0184] Where: α is the angle, is the center coordinate of the real frame, is the center coordinate of the prediction box;

[0185] The expression of the distance loss function is as follows:

[0186]

[0187] γ=2-Λ;

[0188] Where: Δ is the distance loss function, p x It is the square of the ratio of the horizontal distance between the center point of the real box and the predicted box to the width of the minimum bounding rectangle. is the deviation of the prediction box in the horizontal direction, p y It is the square of the ratio of the vertical distance between the center point of the predicted box and the real box to the height of the minimum enclosing rectangle. is the deviation of the prediction box in the vertical direction, c w 、c h are the width and height of the rectangle enclosed by the coordinates of the center points of the real box and the predicted box, respectively, and γ is the weight for adjusting the distance loss. is the degree of deviation of the prediction box in the t direction;

[0189] The expression of the shape loss function is as follows:

[0190]

[0191] Where: Ω is the shape loss function, (w, h) is the width and height of the prediction box, (w gt ,h gt ) is the width and height of the real box, θ is used to control the degree of attention to shape loss, w w is the ratio of the absolute value of the width difference between the predicted box and the real box to the maximum value, w h It is the ratio of the absolute value of the height difference between the predicted box and the real box to the maximum value. is the width loss, For height loss;

[0192] The expression of the IoU loss function is as follows:

[0193]

[0194] Among them: IoU is the IoU loss function, intersection A is the intersection area of ​​the predicted box and the real box, and union B is the union area of ​​the predicted box and the real box;

[0195] Then the expression of the SIoU loss function is as follows:

[0196]

[0197] Among them: Loss SIoU is the SIoU loss function, Ω is the shape loss function, and Δ is the distance loss function.

[0198] The defect detection module 3 is used to input the data set of the transmission line tower into the target detection model, train and optimize the target detection model, and perform defect detection on the transmission line tower based on the optimized target detection model.

[0199] Embodiment 3:

[0200] See also Figure 7 , In this embodiment, a transmission line tower defect detection device based on improved Yolov5 is also included, and the device includes a processor 4 and a memory 5;

[0201] The memory 5 is used to store computer program code 51 and transmit the computer program code 51 to the processor 4;

[0202] The processor 4 is used to execute the power transmission line tower defect detection method based on improved Yolov5 described in Example 1 according to the instructions in the computer program code 51.

[0203] This embodiment also includes a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed on a computer, the power transmission line tower defect detection method based on improved Yolov5 described in Example 1 is implemented.

[0204] Generally speaking, the computer instructions for implementing the method of the present invention may be carried in any combination of one or more computer-readable storage media. Non-transitory computer-readable storage media may include any computer-readable media, except for the signal itself that is temporarily propagating.

[0205] The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EKROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, device, or device.

[0206] Computer program code for performing the operation of the present invention can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, SMalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages, in particular, Python suitable for neural network computing and platform frameworks based on TensorFlow, PyTorch, etc. can be used. The program code can be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer or to an external computer (for example, using an Internet service provider to connect via the Internet) through any type of network, including a local area network (LAN) or a wide area network (WAN).

[0207] The above-mentioned device and non-temporary computer-readable storage medium can be found in the specific description of a transmission line tower defect detection method based on improved Yolov5 and its beneficial effects, which will not be repeated here.

[0208] Although the embodiments of the present invention have been shown and described above, it should be understood that the above embodiments are exemplary and should not be construed as limitations on the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. A transmission line tower defect detection method based on improved Yolov5, characterized in that: include: S1. Collect image data of transmission towers to be inspected based on drone inspections and build a data set of transmission line towers; The data set includes: images of power transmission line towers in normal state and images of power transmission line towers with various defects; S2. Improve the Yolov5 algorithm and build a target detection model based on the improved Yolov5 algorithm; The improvement of the Yolov5 algorithm includes: in the backbone network of the Yolov5 algorithm, using deformable convolution instead of conventional convolution, simplifying the spatial pyramid pooling module SimSPPF and fusing it with the ECA attention mechanism, and using the SIoU loss function to define the penalty mechanism; S3. Input the data set of the transmission line tower into the target detection model, train and optimize the target detection model, and perform defect detection on the transmission line tower based on the optimized target detection model.

2. The power transmission line tower defect detection method based on improved Yolov5 according to claim 1, characterized in that: In step S2, deformable convolution is used instead of conventional convolution, specifically referring to: Use conventional convolution operations to extract the displacement value corresponding to the offset of the image data, and merge the displacement value into the conventional convolution operation. At the same time, optimize the adjustment mechanism of the variable convolution and output the image features. The expression for incorporating the displacement value into the regular convolution operation is as follows: Among them: y(p0) is the point on the output image feature map, p0 is the center point of the convolution kernel in the conventional convolution operation, p n is the offset of the conventional convolution operation, Δp n is the offset of the variable convolution, w(·) is the weight of the corresponding offset point, x(p0+p n +Δp n ) is the input transmission tower image data to be detected at position p n The value in ; The expression of the adjustment mechanism for optimizing variable convolution is as follows: Where: Δm n is the modulation scalar.

3. The power transmission line tower defect detection method based on improved Yolov5 according to claim 1, characterized in that: In step S2, simplifying the spatial pyramid pooling module SimSPPF specifically refers to: The input of the spatial pyramid pooling module SPPF is divided into the first part and the second part; the first part is connected in series with three maximum pooling layers of size 5×5, and the output of the second part is connected to the output of the first part through Concat; at the same time, the ReLU activation function is used in the simplified spatial pyramid pooling module SimSPPF.

4. The method for detecting defects of transmission line towers based on improved Yolov5 according to claim 1, characterized in that: In step S2, the ECA attention mechanism specifically refers to: After performing global average pooling on image features through the ECA attention mechanism, a one-dimensional convolution operation with k kernels is used to adaptively allocate the weight proportion of the input channels; The size of the k-core is obtained through an adaptive function, and its expression is as follows: Among them: k is the size of the convolution kernel obtained, c is the number of channels, γ and b are adjustment parameters; || odd is the odd number closest to the function value.

5. The power transmission line tower defect detection method based on improved Yolov5 according to claim 1, characterized in that: In step S2, the SIoU loss function includes an angle loss function, a distance loss function, a shape loss function, and an IoU loss function; The expression of the angle loss function is as follows: Where: Λ is the angle loss function, c h is the height difference between the center point of the real box and the predicted box, σ is the distance between the center point of the real box and the predicted box, and π is a constant; Where: α is the angle, is the center coordinate of the real frame, is the center coordinate of the prediction box; The expression of the distance loss function is as follows: Where: Δ is the distance loss function, p x It is the square of the ratio of the horizontal distance between the center point of the real box and the predicted box to the width of the minimum enclosing rectangle. is the deviation of the prediction box in the horizontal direction, p y It is the square of the ratio of the vertical distance between the center point of the predicted box and the real box to the height of the minimum enclosing rectangle. is the deviation of the prediction box in the vertical direction, c w 、c h are the width and height of the rectangle enclosed by the coordinates of the center points of the real box and the predicted box, respectively, and γ is the weight for adjusting the distance loss. is the degree of deviation of the prediction box in the t direction; The expression of the shape loss function is as follows: Where: Ω is the shape loss function, (w, h) is the width and height of the prediction box, (w gt ,h gt ) is the width and height of the real box, θ is used to control the degree of attention to shape loss, w w is the ratio of the absolute value of the width difference between the predicted box and the real box to the maximum value, w h It is the ratio of the absolute value of the height difference between the predicted box and the real box to the maximum value. is the width loss, For height loss; The expression of the IoU loss function is as follows: Among them: IoU is the IoU loss function, intersection A is the intersection area of ​​the predicted box and the real box, and union B is the union area of ​​the predicted box and the real box; Then the expression of the SIoU loss function is as follows: Among them: Loss SIoU is the SIoU loss function, Ω is the shape loss function, and Δ is the distance loss function.

6. A transmission line tower defect detection system based on improved Yolov5, characterized in that: The system comprises: A data set construction module (1) is used to collect image data of transmission towers to be inspected based on drone inspections to construct a data set of transmission line towers; the data set includes: images of transmission line towers in normal states and images of transmission line towers with various defects; Algorithm improvement module (2), used to improve the Yolov5 algorithm and build a target detection model based on the improved Yolov5 algorithm; The improvement of the Yolov5 algorithm includes: in the backbone network of the Yolov5 algorithm, using deformable convolution instead of conventional convolution, simplifying the spatial pyramid pooling module SimSPPF and fusing it with the ECA attention mechanism, and using the SIoU loss function to define the penalty mechanism; The defect detection module (3) is used to input the data set of the transmission line tower into the target detection model, train and optimize the target detection model, and perform defect detection on the transmission line tower based on the optimized target detection model.

7. The power transmission line tower defect detection system based on improved Yolov5 according to claim 6, characterized in that: The algorithm improvement module (2) improves the Yolov5 algorithm according to the following method: Use conventional convolution operations to extract the displacement value corresponding to the offset of the image data, and merge the displacement value into the conventional convolution operation. At the same time, optimize the adjustment mechanism of the variable convolution and output the image features. The expression for incorporating the displacement value into the regular convolution operation is as follows: Among them: y(p0) is the point on the output image feature map, p0 is the center point of the convolution kernel in the conventional convolution operation, p n is the offset of the conventional convolution operation, Δp n is the offset of the variable convolution, w(·) is the weight of the corresponding offset point, x(p0+p n +Δp n ) is the input transmission tower image data to be detected at position p n The value in ; The expression of the adjustment mechanism for optimizing variable convolution is as follows: Where: Δm n is the modulation scalar.

8. The power transmission line tower defect detection system based on improved Yolov5 according to claim 6, characterized in that: The algorithm improvement module (2) improves the Yolov5 algorithm according to the following method: The input of the spatial pyramid pooling module SPPF is divided into the first part and the second part; the first part is connected in series with three maximum pooling layers of size 5×5, and the output of the second part is connected to the output of the first part through Concat; at the same time, the ReLU activation function is used in the simplified spatial pyramid pooling module SimSPPF.

9. The power transmission line tower defect detection system based on improved Yolov5 according to claim 6, characterized in that: The algorithm improvement module (2) improves the Yolov5 algorithm according to the following method: After performing global average pooling on image features through the ECA attention mechanism, a one-dimensional convolution operation with k kernels is used to adaptively allocate the weight proportion of the input channels; The size of the k-core is obtained through an adaptive function, and its expression is as follows: Among them: k is the size of the convolution kernel obtained, c is the number of channels, γ and b are adjustment parameters; | | odd is the odd number closest to the function value.

10. The power transmission line tower defect detection system based on improved Yolov5 according to claim 6, characterized in that: The algorithm improvement module (2) improves the Yolov5 algorithm according to the following method: The SIoU loss function includes angle loss function, distance loss function, shape loss function, and IoU loss function; The expression of the angle loss function is as follows: Where: Λ is the angle loss function, c h is the height difference between the center point of the real box and the predicted box, σ is the distance between the center point of the real box and the predicted box, and π is a constant; Where: α is the angle, is the center coordinate of the real frame, is the center coordinate of the prediction box; The expression of the distance loss function is as follows: Where: Δ is the distance loss function, p x It is the square of the ratio of the horizontal distance between the center point of the real box and the predicted box to the width of the minimum enclosing rectangle. is the deviation of the prediction box in the horizontal direction, p y It is the square of the ratio of the vertical distance between the center point of the predicted box and the real box to the height of the minimum enclosing rectangle. is the deviation of the prediction box in the vertical direction, c w 、c h are the width and height of the rectangle enclosed by the coordinates of the center points of the real box and the predicted box, respectively, and γ is the weight for adjusting the distance loss. is the degree of deviation of the prediction box in the t direction; The expression of the shape loss function is as follows: Where: Ω is the shape loss function, (w, h) is the width and height of the prediction box, (w gt ,h gt ) is the width and height of the real box, θ is used to control the degree of attention to shape loss, w w is the ratio of the absolute value of the width difference between the predicted box and the real box to the maximum value, w h It is the ratio of the absolute value of the height difference between the predicted box and the real box to the maximum value. is the width loss, For height loss; The expression of the IoU loss function is as follows: Among them: IoU is the IoU loss function, intersection A is the intersection area of ​​the predicted box and the real box, and union B is the union area of ​​the predicted box and the real box; Then the expression of the SIoU loss function is as follows: Among them: Loss SIoU is the SIoU loss function, Ω is the shape loss function, and Δ is the distance loss function.

Images