Power transmission tower bolt looseness detection method based on improved YOLOv5s model

Abstract

The invention discloses a power transmission tower bolt looseness detection method based on an improved YOLOv5s model. The method comprises the steps that a to-be-detected power transmission tower bolt data set is acquired; inputting the to-be-detected power transmission tower bolt data set into an improved YOLOv5s model for detection, and outputting loose bolt data; wherein the improved YOLOv5s model is optimized based on an existing YOLOv5s model network structure and is obtained through training of a training set, and the training set comprises a loose bolt sample and a fastening bolt sample. According to the invention, accurate detection of power transmission tower bolt looseness can be realized under the conditions of complex environment interference, a large number of bolts and the like.

Description

Technical Field

[0001] The present invention relates to the field of electric power technology, and in particular to a method for detecting loose bolts of a transmission tower based on an improved YOLOv5s model. Background Art

[0002] In the transmission tower bolts, external factors may play an important role in the loss of bolt preload, including vibration and shock, temperature changes, insufficient or uneven tightening force, corrosion and oxidation. Under these circumstances, the bolts gradually loosen and become missing, seriously affecting the rigidity of the tower structure and threatening the safety of the tower. If this is not checked and eliminated in time, it is likely to cause a tower collapse accident.

[0003] Due to the complex environment of the transmission tower, bolts are easily blocked by other objects, resulting in missing important features, false detection and missed detection. In addition, as a tall structure, the transmission tower has certain requirements for the shooting equipment and shooting angle. Improper shooting angle can easily cause false detection and missed detection. There are many bolts and their distribution is uneven, which is prone to missed detection during target detection.

[0004] Therefore, it is necessary to design a high-precision detection model that can achieve accurate detection under conditions such as complex environmental interference and a large number of bolts. Summary of the invention

[0005] In order to solve the above technical problems, the present invention proposes a transmission tower bolt loosening detection method based on an improved YOLOv5s model, which can accurately detect bolt loosening under conditions of complex environmental interference and a large number of bolts.

[0006] To achieve the above object, the present invention provides a method for detecting loose bolts of a transmission tower based on an improved YOLOv5s model, comprising:

[0007] Obtain the data set of transmission tower bolts to be inspected;

[0008] Input the transmission tower bolt data set to be detected into the improved YOLOv5s model for detection, and output the loose bolt data;

[0009] The improved YOLOv5s model is optimized based on the existing YOLOv5s model network structure and obtained through training with a training set, wherein the training set includes loose bolt samples and tightened bolt samples.

[0010] Preferably, the improved YOLOv5s model includes a backbone BackBone module, a neck Slim-Neck module and a head Head module; wherein the backbone BackBone module is used to extract features of input data; the neck Slim-Neck module is used to fuse the extracted semantic features with position features; and the head Head module is used to make predictions based on the features extracted and fused by the backbone BackBone module and the neck Slim-Neck module.

[0011] Preferably, the backbone BackBone module includes:

[0012] CBS submodule: It consists of a Conv unit, a BatchNorm unit, and a SiLU unit connected in sequence, which is used to extract local spatial information in the image and give it nonlinear transformation;

[0013] C3 submodule: It consists of standard convolution and bottleneck layers, and is used to improve the computational efficiency of the algorithm. Each C3 module is connected through the Concat operation.

[0014] SPPF submodule: used to extract feature maps of different sizes from the input image;

[0015] Among them, the convolution layers in the CBS submodule and the C3 submodule are coordinate convolutions.

[0016] Preferably, the coordinate convolution is:

[0017] O(x,y)=∑∑I(i,j)*H(xi,yi)

[0018] Where O(x,y) is the pixel value of the output image O, I(i,j) is the pixel value of the input image I, H(xi,yi) is the weight of the filter H at position (xi,yi), x is the horizontal coordinate of the pixel in image O, y is the vertical coordinate of the pixel in image O, i is the horizontal coordinate of the pixel in image I, and j is the horizontal coordinate of the pixel in image I.

[0019] Preferably, the improved YOLOv5s model also includes a CBAM attention mechanism module, which is a module combining a channel attention mechanism CAM and a spatial attention mechanism SAM, and the CBAM attention mechanism module is respectively connected to the C3 submodule and the SPPF submodule.

[0020] Preferably, the processing process of the CBAM attention mechanism module is:

[0021]

[0022]

[0023] F'=MC(F)F

[0024] F"=MS(F)F'

[0025] In the formula, MC(F) represents the channel attention output weight; σ represents the activation function sigmoid; Represents the feature map in space after average pooling; represents the feature map in space after maximum pooling; W0 represents the weight matrix of the first fully connected layer; W1 represents the weight matrix of the second fully connected layer; MS(F) represents the spatial attention output weight; f 7*7 Represents a convolution kernel of size 7×7; Represents the feature map after average pooling on the channel; represents the feature map after maximum pooling on the channel; F represents the input feature map; F' represents the feature map obtained by channel attention weighting; F" represents the feature map obtained by spatial attention weighting.

[0026] Preferably, the neck Slim-Neck module includes a VoV-GSCSP unit, which is used to reduce the dimension of the input feature map by 1×1 ordinary convolution, and then input the dimension reduction result into two branches, one of which is to perform two layers of GSConv through the GSbottleneck subunit, and the other branch is to perform a Concat operation on the results of the two layers of GSConv, and finally output it through 1×1 convolution;

[0027] Among them, the convolutions in the neck Slim-Neck module are all GSConv.

[0028] Preferably, the calculation method of GSConv is:

[0029] h_v'=W*concat[(h_v,AGG(h_n1),AGG(h,_n2),...,AGG(h_nk))]

[0030] Where h_v represents the original feature vector of node v, h_n1, h,_n2, …, h_nk represent the original feature vectors of the neighboring nodes of node v respectively, h_v' represents the updated feature vector of node v, concat represents the concatenation of feature vectors, AGG represents the function of aggregating neighbor features, and W is a learnable weight matrix.

[0031] Preferably, the loss function used by the head module is a GIoU loss function, specifically:

[0032]

[0033] In the formula, A represents the predicted bounding box, B represents the real bounding box, and C represents the circumscribed rectangle of the two bounding boxes;

[0034] The algorithm used by the Head module is the Soft-NMS algorithm, specifically:

[0035]

[0036] In the formula, M represents the candidate prior box with a higher score, B i Represents the candidate prior box to be processed, N t is the set threshold, and IoU is the overlap score between the candidate prior box with a high decay score and the candidate prior box of the previous frame.

[0037] Preferably, the training set is obtained after processing based on the transmission tower bolt data set, and the improved YOLOv5s model is trained by using a cosine annealing strategy to dynamically adjust the learning rate and a stochastic gradient descent optimization algorithm.

[0038] Compared with the prior art, the present invention has the following advantages and technical effects:

[0039] The present invention improves the ability of the improved YOLOv5s model to obtain target position information by replacing the ordinary convolution in the backbone network with the coordinate convolution; by adding the CBAM attention mechanism, the improved YOLOv5s model's attention to the target area is enhanced; the Slim-Neck structure is adopted to improve the improved YOLOv5s model's ability to process and express features; the IoU loss function is replaced by the GIoU loss function, and the NMS algorithm is replaced by the Soft-NMS algorithm, which improves the target detection effect of the improved YOLOv5s model in complex scenes. The present invention can accurately detect the loosening of transmission tower bolts under conditions such as complex environmental interference and a large number of bolts. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0041] Figure 1 This is a flow chart of a method for detecting loose bolts on a transmission tower based on an improved YOLOv5s model according to an embodiment of the present invention;

[0042] Figure 2 A schematic diagram of loose and tight samples in a data set according to an embodiment of the present invention;

[0043] Figure 3 A schematic diagram of the network structure of the YOLOv5s model according to an embodiment of the present invention;

[0044] Figure 4 Schematic diagram of the structure of coordinate convolution according to an embodiment of the present invention;

[0045] Figure 5 It is a schematic diagram of the structure of the CBAM attention mechanism module in an embodiment of the present invention;

[0046] Figure 6 Schematic diagram of the structure of the Silm-Neck module in an embodiment of the present invention;

[0047] Figure 7 is a schematic diagram of the structure of GSConv in an embodiment of the present invention;

[0048] Figure 8 is a schematic structural diagram of a VoV-GSCSP unit in an embodiment of the present invention;

[0049] Fig. 9 is a schematic structural diagram of a GSBottleneck subunit in an embodiment of the present invention;

[0050] Fig.10 Schematic diagram of the principle of the GIoU loss function in an embodiment of the present invention;

[0051] Fig.11 This is a diagram showing the effect of the Soft-NMS algorithm in an embodiment of the present invention;

[0052] Fig.12 Schematic diagram of the network structure of the improved YOLOv5s model according to an embodiment of the present invention;

[0053] Fig.13 Graph showing the effectiveness of a test according to an embodiment of the present invention. DETAILED DESCRIPTION

[0054] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0055] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0056] The present invention proposes a transmission tower bolt loosening detection method based on an improved YOLOv5s model. Figure 1 ,include:

[0057] Obtain the data set of transmission tower bolts to be inspected;

[0058] Input the transmission tower bolt data set to be detected into the improved YOLOv5s model for detection, and output the loose bolt data;

[0059] Among them, the improved YOLOv5s model is optimized based on the existing YOLOv5s model network structure and obtained through training with a training set, which includes loose bolt samples and tightened bolt samples.

[0060] Specifically, the network structure diagram of the existing YOLOv5s model is as follows Figure 3 As shown. YOLOv5s is a lightweight target detection model with relatively low cost in inference calculation and training speed. The YOLOv5s model includes: BackBone module, Neck module and Head module;

[0061] The BackBone module includes: a CBS submodule, a C3 submodule and a SPPF submodule.

[0062] The CBS submodule consists of a Conv unit, a BatchNorm unit, and a SiLU unit, which are used to extract local spatial information from the image and give it nonlinear transformation; the C3 submodule consists of three standard convolutional layers and a bottleneck layer, which are used to improve the computational efficiency of the algorithm. Each C3 submodule is connected by a Concat operation; the SPPF submodule is used to extract feature maps of different sizes from the input image.

[0063] Neck module: It consists of FPN submodule and PAN submodule, which is used to cleverly integrate the extracted semantic features with the position features. At the same time, it organically integrates the backbone network layer with the detection layer to enhance the features extracted by different network layers.

[0064] Head module: It is mainly composed of Anchors submodule, Classification submodule and Regression submodule. Among them, the Anchors submodule is used to define target boxes of different sizes and aspect ratios. It is usually clustered by K-means clustering on the target boxes of the training set. It can be calculated before model training and stored in the model to generate detection boxes for prediction; the Classification submodule is used to classify each detection box to determine whether it is a target object. It usually classifies features in the form of a fully connected layer plus a Softmax function; the Regression submodule is used to regress each detection box to obtain its position and size. It usually regresses features in the form of a fully connected layer.

[0065] Furthermore, the improved YOLOv5s model includes a backbone BackBone module, a neck Slim-Neck module and a head Head module, wherein the backbone BackBone module is used to extract features of input data; the neck Slim-Neck module is used to fuse the extracted semantic features with position features; and the head Head module is used to make predictions based on the features extracted and fused by the backbone BackBone module and the neck Slim-Neck module.

[0066] Furthermore, the backbone BackBone module includes:

[0067] CBS submodule: It consists of a Conv unit, a BatchNorm unit, and a SiLU unit connected in sequence, which is used to extract local spatial information in the image and give it nonlinear transformation;

[0068] C3 submodule: It consists of standard convolution and bottleneck layers, and is used to improve the computational efficiency of the algorithm. Each C3 module is connected through the Concat operation.

[0069] SPPF submodule: used to extract feature maps of different sizes from the input image;

[0070] Among them, the convolution layers in the CBS submodule and the C3 submodule are coordinate convolutions.

[0071] Specifically, in this embodiment, the structural diagram of coordinate convolution is as follows: Figure 4 As shown in the figure, all ordinary convolutions except the SPPF submodule in the BackBone module of the YOLOv5s model are replaced with coordinate convolutions, which effectively enhances the model's ability to obtain target position information. The coordinate convolution can be expressed as the following formula:

[0072] O(x,y)=∑∑I(i,j)*H(xi,yi)

[0073] Among them, O(x,y) is the pixel value of the output image O, I(i,j) is the pixel value of the input image I, H(xi,yi) is the weight of the filter H at the position (xi,yi), x is the horizontal coordinate of the pixel of image O, y is the vertical coordinate of the pixel of image O, i is the horizontal coordinate of the pixel of image I, and j is the horizontal coordinate of the pixel of image I.

[0074] Furthermore, the improved YOLOv5s model also includes a CBAM attention mechanism module. The CBAM attention mechanism is a module that combines the channel attention mechanism CAM and the spatial attention mechanism SAM. The CBAM attention mechanism module is connected to the C3 sub-module and the SPPF sub-module respectively.

[0075] Specifically, the structural diagram of the CBAM attention mechanism module is as follows: Figure 5 As shown in the figure, a CBAM attention mechanism module is added between the C3 submodule and the SPPF submodule in the BackBone module of the backbone network. The CBAM attention mechanism module is a module that combines the channel attention mechanism (CAM) and the spatial attention mechanism (SAM), and its expression is:

[0076]

[0077]

[0078] F'=MC(F)F

[0079] F"=MS(F)F'

[0080] In the formula, MC(F) represents the channel attention output weight; σ represents the activation function sigmoid; Represents the feature map in space after average pooling; represents the feature map in space after maximum pooling; W0 represents the weight matrix of the first fully connected layer; W1 represents the weight matrix of the second fully connected layer; MS(F) represents the spatial attention output weight; f 7*7 Represents a convolution kernel of size 7×7; Represents the feature map after average pooling on the channel; represents the feature map after maximum pooling on the channel; F represents the input feature map; F' represents the feature map obtained by channel attention weighting; F" represents the feature map obtained by spatial attention weighting.

[0081] Furthermore, the neck Slim-Neck module includes a VoV-GSCSP unit, which is used to reduce the dimension of the input feature map by 1×1 ordinary convolution, and then input the dimension reduction result into two branches, one of which is to perform two layers of GSConv in the GSBottleneck subunit, and the other branch is to perform a Concat operation on the results of the two layers of GSConv, and finally output it through 1×1 convolution;

[0082] Among them, the convolutions in the neck Slim-Neck module are all GSConv.

[0083] Specifically, the Neck module in the existing YOLOv5s model is modified into a neck Slim-Neck module. The structural diagram of the neck Silm-Neck module is as follows: Figure 6As shown in Figure 1, the neck Slim-Neck module replaces all ordinary convolutions in the neck with GSConv and then introduces the VoV-GSCSP unit. The structural diagram of GSConv is shown in Figure 1. Figure 7 The schematic diagram of the structure of the VoV-GSCSP unit is shown in Figure 8 The VoV-GSCSP unit first reduces the dimension of the input feature map by 1×1 ordinary convolution, and then inputs the result into two branches at the same time, one of which is input into the GSBottleneck subunit for two layers of GSConv. The structural diagram of the GSBottleneck subunit is shown in Fig. 9 As shown in the figure, the other branch is concat operation with the results of two layers of GSConv, and finally output through 1×1 convolution.

[0084] Furthermore, the calculation method of GSConv is:

[0085] h_v'=W*concat[(h_v,AGG(h_n1),AGG(h,_n2),...,AGG(h_nk))]

[0086] Where h_v represents the original feature vector of node v, h_n1, h,_n2, …, h_nk represent the original feature vectors of the neighboring nodes of node v respectively, h_v' represents the updated feature vector of node v, concat represents the concatenation of feature vectors, AGG represents the function of aggregating neighbor features, and W is a learnable weight matrix.

[0087] Furthermore, the loss function used by the head module is the GIoU loss function, specifically:

[0088]

[0089] In the formula, A represents the predicted bounding box, B represents the real bounding box, and C represents the circumscribed rectangle of the two bounding boxes;

[0090] The algorithm used by the Head module is the Soft-NMS algorithm, specifically:

[0091]

[0092] In the formula, M represents the candidate prior box with a higher score, B i Represents the candidate prior box to be processed, N t is the set threshold, and IoU is the overlap score between the candidate prior box with a high decay score and the candidate prior box of the previous frame.

[0093] Specifically, the principle diagram of the GIoU loss function and the effect diagram of the Soft-NMS algorithm are as follows: Fig.10 , Fig.11 As shown in the figure, the IoU loss function in the Head module is replaced by the GIoU loss function, and the NMS algorithm is replaced by the Soft-NMS algorithm.

[0094] Furthermore, the training set is obtained after processing based on the transmission tower bolt data set, and the improved YOLOv5s model is trained by using a cosine annealing strategy to dynamically adjust the learning rate and a stochastic gradient descent optimization algorithm.

[0095] Specifically, based on the preprocessed data set, the optimized YOLOv5s model is trained to obtain an improved YOLOv5s model.

[0096] In this example, the transmission tower bolt dataset has a total of 7050 images, including two categories: loose and tight. These images are divided into training set and validation set in a ratio of 8:2. The training set is used to optimize the network parameters to minimize the loss function, and the validation set is used to verify the performance of the model during the training process. Figure 2 Samples of loose and tightened bolts are shown.

[0097] Improved YOLOv5s model network architecture diagram as follows Fig.12 As shown in the figure, the training parameters are set as follows: Batch size is 8; Workers size is 8; the maximum number of iterations is 200; the momentum factor is 0.937; the initial learning rate is 0.01, the weight decay coefficient is 0.0005, the cosine annealing strategy is used to dynamically adjust the learning rate, and the stochastic gradient descent optimization algorithm is used.

[0098] The performance comparison between the present invention and other existing models is shown in Table 1, which meets the requirements of accurate transmission tower bolt loosening in terms of accuracy, recall rate, average precision, and average precision mean indicators.

[0099] Table 1

[0100]

[0101] Furthermore, the improved YOLOv5s model is used to detect the target video or image, and the detected video or image is output.

[0102] The present invention is used in the effectiveness experiment of transmission tower bolt loosening detection. Fig.13 The invention can effectively identify whether the bolts are loose or tight. The invention can accurately detect the looseness of bolts under the conditions of complex environmental interference and a large number of bolts.

[0103] The above are only preferred specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for detecting loose bolts in a transmission tower based on an improved YOLOv5s model, characterized in that: include: Obtain the data set of transmission tower bolts to be inspected; Input the transmission tower bolt data set to be detected into the improved YOLOv5s model for detection, and output the loose bolt data; The improved YOLOv5s model is optimized based on the existing YOLOv5s model network structure and obtained through training with a training set, wherein the training set includes loose bolt samples and tightened bolt samples.

2. The method for detecting loose bolts of transmission towers based on the improved YOLOv5s model according to claim 1, characterized in that: The improved YOLOv5s model includes a backbone BackBone module, a neck Slim-Neck module and a head Head module; wherein the backbone BackBone module is used to extract features of input data; the neck Slim-Neck module is used to fuse the extracted semantic features with position features; and the head Head module is used to make predictions based on the features extracted and fused by the backbone BackBone module and the neck Slim-Neck module.

3. The method for detecting loose bolts of transmission towers based on the improved YOLOv5s model according to claim 2, characterized in that: The BackBone module includes: CBS submodule: It consists of a Conv unit, a BatchNorm unit, and a SiLU unit connected in sequence, which is used to extract local spatial information in the image and give it nonlinear transformation; C3 submodule: It consists of standard convolution and bottleneck layers, and is used to improve the computational efficiency of the algorithm. Each C3 module is connected through the Concat operation. SPPF submodule: used to extract feature maps of different sizes from the input image; Among them, the convolution layers in the CBS submodule and the C3 submodule are coordinate convolutions.

4. The method for detecting loose bolts of transmission towers based on the improved YOLOv5s model according to claim 3 is characterized in that: The coordinate convolution is: O(x,y)=∑∑I(i,j)*H(xi,yi) Where O(x,y) is the pixel value of the output image O, I(i,j) is the pixel value of the input image I, H(xi,yi) is the weight of the filter H at position (xi,yi), x is the horizontal coordinate of the pixel in image O, y is the vertical coordinate of the pixel in image O, i is the horizontal coordinate of the pixel in image I, and j is the horizontal coordinate of the pixel in image I.

5. The method for detecting loose bolts of transmission towers based on the improved YOLOv5s model according to claim 3, characterized in that: The improved YOLOv5s model also includes a CBAM attention mechanism module, which is a module combining a channel attention mechanism CAM and a spatial attention mechanism SAM. The CBAM attention mechanism module is connected to the C3 submodule and the SPPF submodule respectively.

6. The method for detecting loose bolts of transmission towers based on the improved YOLOv5s model according to claim 5, characterized in that: The processing process of the CBAM attention mechanism module is: F'=MC(F)F F"=MS(F)F' In the formula, MC(F) represents the channel attention output weight; σ represents the activation function sigmoid; Represents the feature map in space after average pooling; represents the feature map in space after maximum pooling; W0 represents the weight matrix of the first fully connected layer; W1 represents the weight matrix of the second fully connected layer; MS(F) represents the spatial attention output weight; f 7*7 Represents a convolution kernel of size 7×7; Represents the feature map after average pooling on the channel; represents the feature map after maximum pooling on the channel; F represents the input feature map; F' represents the feature map obtained by channel attention weighting; F" represents the feature map obtained by spatial attention weighting.

7. The method for detecting loose bolts of transmission towers based on the improved YOLOv5s model according to claim 2, characterized in that: The neck Slim-Neck module includes a VoV-GSCSP unit, which is used to reduce the dimension of the input feature map by 1×1 ordinary convolution, and then input the dimension reduction result into two branches, one of which is to perform two layers of GSConv through the GSBottleneck subunit, and the other branch is to perform a Concat operation on the results of the two layers of GSConv, and finally output it through 1×1 convolution; Among them, the convolutions in the neck Slim-Neck module are all GSConv.

8. The method for detecting loose bolts of transmission towers based on the improved YOLOv5s model according to claim 7, characterized in that: The calculation method of GSConv is: h_v'=W*concat[(h_v,AGG(h_n1),AGG(h,_n2),...,AGG(h_nk))] Where h_v represents the original feature vector of node v, h_n1, h,_n2, …, h_nk represent the original feature vectors of the neighboring nodes of node v respectively, h_v' represents the updated feature vector of node v, concat represents the concatenation of feature vectors, AGG represents the function of aggregating neighbor features, and W is a learnable weight matrix.

9. The method for detecting loose bolts of transmission towers based on the improved YOLOv5s model according to claim 2, characterized in that: The loss function used by the head module is the GIoU loss function, specifically: In the formula, A represents the predicted bounding box, B represents the real bounding box, and C represents the circumscribed rectangle of the two bounding boxes; The algorithm used by the Head module is the Soft-NMS algorithm, specifically: In the formula, M represents the candidate prior box with a higher score, B i Represents the candidate prior box to be processed, N t is the set threshold, and IoU is the overlap score between the candidate prior box with a high decay score and the candidate prior box of the previous frame.

10. The method for detecting loose bolts of transmission towers based on the improved YOLOv5s model according to claim 1, characterized in that: The training set is obtained after processing based on the transmission tower bolt data set. The improved YOLOv5s model is trained by using a cosine annealing strategy to dynamically adjust the learning rate and a stochastic gradient descent optimization algorithm.

Images