Wind power tower drum surface crack intelligent detection method based on improved YOLOv5s

Abstract

The invention discloses a wind power tower drum surface crack intelligent detection method based on improved YOLOv5s, and the method comprises the steps: S1, obtaining a historical wind power tower drum surface crack image, and carrying out the preprocessing of the historical wind power tower drum surface crack image, and obtaining a preprocessed image; step S2, the cracks in the processed image are marked, and the marked image is obtained; s3, training the improved YOLOv5s algorithm by using the marked image to obtain a wind power tower drum surface crack detection model; and S4, inputting a real-time to-be-processed wind power tower drum crack image or video into the wind power tower drum crack detection model to realize intelligent detection of the surface crack of the wind power tower drum. The problems of low efficiency, false detection and limited detection range of an artificial vision detection method are solved. Compared with a traditional crack detection method based on a YOLOv5s algorithm, the crack detection method based on the YOLOv5s algorithm has the advantages that the accuracy is improved, the phenomena of false detection and missing detection are reduced, and due to the addition of the preprocessing step, the robustness is better.

Description

Technical Field

[0001] The present invention relates to the field of electric power technology, and in particular to an intelligent detection method for surface cracks of a wind power tower based on improved YOLOv5s. Background Art

[0002] Concrete structural defects are a common phenomenon, but they are difficult to quantitatively analyze in engineering and often require a lot of manpower and time.

[0003] For monitoring cracks on the surface of wind turbine towers, currently, high-altitude hanging baskets or spiders are usually used for handheld contact measurement, which has a limited detection range. High-altitude operations are affected by environmental factors such as airflow and temperature, making it difficult to ensure the accuracy of detection and are highly dangerous; at the same time, the increase in labor costs and low detection efficiency lead to high economic costs.

[0004] With the continuous development of computer technology, more and more deep learning algorithms are emerging, but they all have advantages and disadvantages. The accuracy of crack identification based on the traditional YOLOv5s algorithm is average, especially when identifying cracks in tall buildings such as wind turbine towers, it is easy to cause false detection and missed detection. Therefore, the present invention proposes an intelligent detection method for surface cracks of wind turbine towers based on improved YOLOv5s to improve the accuracy of detection and reduce missed detection and false detection. Summary of the invention

[0005] In order to solve the above technical problems, the present invention provides an intelligent detection method for surface cracks of wind turbine tower based on improved YOLOv5s, the method comprising:

[0006] Step S1, obtaining a historical wind turbine tower surface crack image and preprocessing it to obtain a preprocessed image;

[0007] Step S2, marking the cracks in the processed image to obtain a marked image;

[0008] Step S3: using the annotated images to train the improved YOLOv5s algorithm to obtain a wind turbine tower surface crack detection model;

[0009] Step S4: input the real-time wind turbine tower crack image to be processed into the wind turbine tower crack detection model to realize intelligent detection of wind turbine tower surface cracks.

[0010] Optionally, in step S1, the preprocessing process specifically includes:

[0011] Adding salt and pepper noise to the acquired historical wind turbine tower surface crack image: randomly selecting a preset number of pixel points in the historical wind turbine tower surface crack image as noise points, and setting grayscale values ​​for the selected noise points;

[0012] According to the center of the image after adding salt and pepper noise as the central axis, the image is flipped upside down and left to right to obtain a flipped image;

[0013] The pixels of the flipped image are increased or decreased to obtain a scaled image, thereby completing image preprocessing.

[0014] Optionally, in step S3, the improved YOLOv5s algorithm specifically includes: a BackBone module, a Slim-Neck module and a Head module;

[0015] The BackBone module is used to extract feature maps of different sizes, and the BackBone module includes a CBS module, a C3 module and an SPPF module;

[0016] The Slim-Neck module is used to input the obtained feature map into the Slim-Neck module to obtain the aggregated feature;

[0017] The Head module is used to select detection results based on aggregated features.

[0018] Optionally, the ordinary convolution of the CBS module and the C3 module in the BackBone module is replaced by the coordinate convolution:

[0019] O(x,y)=∑∑I(i,j)*H(xi,yi)

[0020] Among them, O(x,y) is the pixel value of the output image O, I(i,j) is the pixel value of the input image I, and H(xi,yi) is the weight of the filter H at position (xi,yi).

[0021] Optionally, insert the CBAM attention mechanism between the C3 module and the SPFF module:

[0022]

[0023]

[0024] F'=MC(F)F

[0025] F"=MS(F)F'

[0026] Where MC(F) represents the channel attention output weight; σ represents the activation function sigmoid; Represents the feature map in space after average pooling; represents the feature map in space after maximum pooling; W0 represents the weight matrix of the first fully connected layer; W1 represents the weight matrix of the second fully connected layer; MS(F) represents the spatial attention output weight; f 7*7 Represents a convolution kernel of size 7×7; Represents the feature map after average pooling on the channel; represents the feature map after maximum pooling on the channel; F represents the input feature map; F' represents the feature map obtained by channel attention weighting; F" represents the feature map obtained by spatial attention weighting.

[0027] Optionally, the Slim-Neck module replaces all ordinary convolutions in the neck with GSConv, and then introduces the VoV-GSCSP module; the VoV-GSCSP module first reduces the dimension of the input feature map by 1×1 ordinary convolution, and inputs the result into two branches at the same time, one of which is input into the GSbottleneck module for two layers of GSConv, and the other branch is concat operation with the results of the two layers of GSConv, and finally outputs through 1×1 convolution; the GSConv calculation formula is:

[0028] h_v'=W*concat[(h_v,AGG(h_n1),AGG(h,_n2),...,AGG(h_nk))]

[0029] Among them, h_v represents the original feature vector of node v, h_n1, h_n2, …, h_nk represent the original feature vectors of the neighboring nodes of node v, and h_v' represents the updated feature vector of node v; concat represents the concatenation of feature vectors, AGG represents the function of aggregating neighbor features, and W is a learnable weight matrix.

[0030] Optionally, in the Head module, the GIOU loss function and the Soft-NMS algorithm are used:

[0031]

[0032] Where A represents the predicted bounding box, B represents the real bounding box, and C represents the circumscribed rectangle of the two bounding boxes;

[0033]

[0034] Where M represents the candidate prior box with a higher score, B i Represents the candidate prior box to be processed, N t is the set threshold, IoU is the overlap score between the candidate prior frame with a high decay score and the candidate prior frame of the previous frame; S i Represents the score of the remaining i-th candidate prior box.

[0035] Optionally, the training parameters of the improved YOLOv5s are set as follows: Batch-size is 8; Workers size is 8; maximum number of iterations is 200; momentum factor is 0.937; initial learning rate is 0.01, and weight decay coefficient is 0.0005.

[0036] Compared with the prior art, the present invention has the following beneficial effects:

[0037] The present invention obtains a wind turbine tower surface crack detection model based on historical wind turbine tower surface crack images and training based on the improved YOLOv5 algorithm; the wind turbine tower surface crack image or video to be processed is input into the wind turbine tower surface crack detection model to realize intelligent detection of wind turbine tower surface cracks. The present invention solves the problems of low efficiency, false detection, and limited detection range based on artificial visual detection methods. Compared with the traditional crack detection method based on the YOLOv5s algorithm, the accuracy is improved, the false detection and missed detection phenomena are reduced, and the robustness is better. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solution of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0039] Figure 1 This is a flow chart of an intelligent method for detecting surface cracks of a wind turbine tower based on improved YOLOv5s according to an embodiment of the present invention;

[0040] Figure 2 This is an effect diagram of adding salt and pepper noise, horizontal flipping, vertical flipping, and scaling in the embodiment of the present invention;

[0041] Figure 3 A schematic diagram of the network structure of the YOLOv5s algorithm according to an embodiment of the present invention;

[0042] Figure 4 Schematic diagram of the structure of coordinate convolution according to an embodiment of the present invention;

[0043] Figure 5 This is a schematic diagram of the structure of a CBAM according to an embodiment of the present invention;

[0044] Figure 6 This is a schematic diagram of the structure of Silm-Neck according to an embodiment of the present invention;

[0045] Figure 7 Schematic diagram of the principle of the GIoU loss function of an embodiment of the present invention;

[0046] Figure 8 This is a rendering of the Soft-NMS algorithm according to an embodiment of the present invention;

[0047] Fig. 9 It is a schematic diagram of the network structure of the improved YOLOv5s algorithm according to an embodiment of the present invention. DETAILED DESCRIPTION

[0048] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0049] Example

[0050] The present invention provides an intelligent detection method for surface cracks of wind turbine tower based on improved YOLOv5s. Figure 1 As shown, specifically including:

[0051] Step S1, obtaining historical wind turbine tower surface crack images and preprocessing them to obtain preprocessed images. As an implementation of this embodiment, this embodiment obtains a total of 2550 historical wind turbine tower surface crack images.

[0052] In the process of preprocessing the image, salt and pepper noise is added, and horizontal flipping, vertical flipping, and scaling are performed. Figure 2 As shown in the figure, there are 10,200 images in total after preprocessing.

[0053] Add salt and pepper noise:

[0054] (1) Selecting noise points: randomly selecting some pixels in the image contained in the acquisition module as noise points (x, y). In this embodiment, the number of selected noise points is less than the number of pixel points.

[0055] (2) Setting the noise value: For each selected noise point (x, y), set its grayscale value to the maximum value 255 or the minimum value 0. That is, let the noise point (x, y) = (255, 255) or (0, 0).

[0056] Flip vertically:

[0057] The image is flipped upside down with the center of the image contained in the acquisition module as the central axis. The coordinate calculation formula of the pixel point after vertical flipping is:

[0058]

[0059] After vertical flipping, the new coordinates of each pixel in the image are (x', y'), where the calculation formula for x' is x'=x, the calculation formula for y' is y'=h-1-y, the original coordinates of the pixel are (x, y), and h is the height of the image.

[0060] Flip Horizontal:

[0061] The center of the image contained in the acquisition module is used as the central axis to flip the image left and right. The coordinate calculation formula of the pixel point after horizontal flipping is:

[0062]

[0063] After horizontal flipping, the new coordinates of each pixel in the image are (x', y'), where the calculation formula for x' is x'=w-1-x, and the calculation formula for y' is y'=y. The original coordinates of the pixel are (x, y), and w is the width of the image.

[0064] Zoom:

[0065] The image size is changed by adding or subtracting pixels from the image contained in the acquisition module, thereby achieving scaling. The coordinate calculation formula of the pixel point after scaling is:

[0066]

[0067] The new coordinates of each pixel in the scaled image are (x', y'), x' = αx, y' = βy. The original coordinates of the pixel are (x, y), α and β are the scaling factors, if α and β are greater than 1, the image is enlarged, if α and β are less than 1, the image is reduced, and if α and β are equal to 1, it is geometric scaling.

[0068] Step S2: label the cracks in the processed image to obtain a labeled image. In this embodiment, a Labeling image labeling tool is used to select the image with a mouse to complete the labeling. The labeling frame is a rectangle, and the labeling frame just surrounds the corresponding crack.

[0069] Step S3: Use the annotated images to train the improved YOLOv5s algorithm to obtain a wind turbine tower surface crack detection model.

[0070] As an implementation method of the embodiment of the present invention, in step 4, the YOLOv5s algorithm includes a BackBone module, a Neck module and a Head module, such as Figure 3 As shown in the figure, the BackBone module includes: CBS module, C3 module and SPPF module. The CBS module consists of Conv, BatchNorm and SiLU, which is used to extract local spatial information in the image and give it nonlinear transformation; the C3 module consists of three standard convolutional layers and a bottleneck layer, which is used to improve the computational efficiency of the algorithm, wherein each C3 module is connected by a Concat operation; the SPPF module is used to extract feature maps of different sizes from the input image.

[0071] Neck module: It consists of FPN and PAN, and is used to skillfully integrate the extracted semantic features with the position features. At the same time, it organically integrates the backbone network layer with the detection layer to enhance the features extracted by different network layers.

[0072] Head module: mainly composed of Anchors, Classification, and Regression. Anchors are used to define target boxes of different sizes and aspect ratios. They are usually obtained by clustering the target boxes of the training set using K-means clustering. They can be calculated before model training and stored in the model to generate detection boxes for prediction. Classification is used to classify each detection box to determine whether it is a target object. Features are usually classified in the form of a fully connected layer plus a Softmax function. Regression is used to regress each detection box to obtain its position and size. Features are usually regressed in the form of a fully connected layer.

[0073] As an implementation method of the present invention, in step 4, the improved YOLOv5s algorithm is to optimize the YOLOv5s algorithm, including: (1) replacing all ordinary convolutions except SPPF in the BackBone module of the YOLOv5s model with coordinate convolutions. The coordinate convolution structure is as follows: Figure 4 As shown, the coordinate convolution can be expressed as the following formula:

[0074] O(x,y)=∑∑I(i,j)*H(xi,yi)

[0075] Among them, O(x,y) is the pixel value of the output image O, I(i,j) is the pixel value of the input image I, and H(xi,yi) is the weight of the filter H at position (xi,yi).

[0076] (2) A CBAM attention mechanism is added between the C3 module and the SPPF module in the backbone network. Its structure is as follows: Figure 5 As shown. The CBAM attention mechanism is a module that combines the channel attention mechanism (CAM) and the spatial attention mechanism (SAM), and its expression is:

[0077]

[0078]

[0079] F'=MC(F)F

[0080] F"=MS(F)F'

[0081] Where MC(F) represents the channel attention output weight; σ represents the activation function sigmoid; Represents the feature map in space after average pooling; represents the feature map in space after maximum pooling; W0 represents the weight matrix of the first fully connected layer; W1 represents the weight matrix of the second fully connected layer. MS(F) represents the spatial attention output weight; f 7*7 Represents a convolution kernel of size 7×7; Represents the feature map after average pooling on the channel; Represents the feature map after maximum pooling on the channel. F represents the input feature map; F' represents the feature map obtained by channel attention weighting; F" represents the feature map obtained by spatial attention weighting.

[0082] (3) The structure of the Neck module of YOLOv5s is modified to the Slim-Neck structure, whose structure is as follows Figure 6 As shown. The Slim-Neck structure replaces all ordinary convolutions in the neck with GSConv, and then introduces the VoV-GSCSP module. The VoV-GSCSP module first reduces the dimension of the input feature map through 1×1 ordinary convolution, and then inputs the result into two branches at the same time. One branch is input to the GSBottleneck module for two layers of GSConv, and the other branch is concat with the results of the two layers of GSConv, and finally outputs through 1×1 convolution. The GSConv calculation formula is:

[0083] h_v'=W*concat[(h_v,AGG(h_n1),AGG(h,_n2),...,AGG(h_nk))]

[0084] Among them, h_v represents the original feature vector of node v, h_n1, h_n2, …, h_nk represent the original feature vectors of the neighbor nodes of node v, and h_v' represents the updated feature vector of node v. concat represents the concatenation of feature vectors, AGG represents the function of aggregating neighbor features, and W is a learnable weight matrix.

[0085] (4) Replace the IoU loss function in the Head module with the GIoU loss function, and replace the NMS algorithm with the Soft-NMS algorithm. The schematic diagram of the GIoU loss function and the Soft-NMS algorithm is as follows: Figure 7 and Figure 8 As shown. Among them, Figure 8 (a) is a schematic diagram of the target detection results using the NMS algorithm. Figure 8 (b) is a schematic diagram of the target detection results using the Soft-NMS algorithm. The expressions of GIoU and the Soft-NMS algorithm are as follows:

[0086] Where A represents the predicted bounding box, B represents the true bounding box, and C represents the circumscribed rectangle of the two bounding boxes.

[0087]

[0088] Where M represents the candidate prior box with a higher score, B i Represents the candidate prior box to be processed, N t is the threshold set, and IoU is the overlap score between the candidate prior frame with the high decay score and the candidate prior frame of the previous frame. The higher the overlap of the target bolt, the greater the decay of the score, S i Represents the score of the remaining i-th candidate prior box.

[0089] As an implementation method of the present invention, in step 4, the following Fig. 9 The improved YOLOv5s algorithm shown in the figure splits the annotated 10,200 images into training sets and validation sets in a ratio of 8:2 for training, and obtains a wind turbine tower surface crack detection model. The training parameters are set as follows: Batch-size is 8; Workers size is 8; maximum number of iterations is 200; momentum factor is 0.937; initial learning rate is 0.01, weight decay coefficient is 0.0005, cosine annealing strategy is used to dynamically adjust the learning rate, and stochastic gradient descent optimization algorithm is used. The wind turbine tower surface crack detection model is evaluated based on Precision, Recall and mAP@0.5 as indicators, and the evaluation results are shown in Table 1.

[0090] Table 1

[0091] Precision Recall mAP@0.5 YOLOv5s 0.79 0.81 0.89 Improving YOLOv5s 0.84 0.83 0.91

[0092] Step S4: input the real-time wind turbine tower crack image or video to be processed into the wind turbine tower crack detection model to realize intelligent detection of wind turbine tower surface cracks.

[0093] The embodiments described above are only descriptions of the preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the protection scope determined by the claims of the present invention.

Claims

1. An intelligent detection method for wind turbine tower surface cracks based on improved YOLOv5s, characterized in that: Methods include: Step S1, obtaining a historical wind turbine tower surface crack image and preprocessing it to obtain a preprocessed image; Step S2, marking the cracks in the preprocessed image to obtain a marked image; Step S3: using the annotated images to train the improved YOLOv5s algorithm to obtain a wind turbine tower surface crack detection model; Step S4: input the real-time wind turbine tower crack image to be processed into the wind turbine tower crack detection model to realize intelligent detection of wind turbine tower surface cracks.

2. The method for intelligent detection of surface cracks of wind turbine towers based on improved YOLOv5s according to claim 1, characterized in that: In step S1, the preprocessing process specifically includes: Adding salt and pepper noise to the acquired historical wind turbine tower surface crack image: randomly selecting a preset number of pixel points in the historical wind turbine tower surface crack image as noise points, and setting grayscale values ​​for the selected noise points; According to the center of the image after adding salt and pepper noise as the central axis, the image is flipped upside down and left to right to obtain a flipped image; The pixels of the flipped image are increased or decreased to obtain a scaled image, thereby completing image preprocessing.

3. The method for intelligent detection of surface cracks of wind turbine towers based on improved YOLOv5s according to claim 1, characterized in that: In step S3, the improved YOLOv5s algorithm specifically includes: a BackBone module, a Slim-Neck module and a Head module; The BackBone module is used to extract feature maps of different sizes, and the BackBone module includes a CBS module, a C3 module and an SPPF module; The Slim-Neck module is used to input the obtained feature map into the Slim-Neck module to obtain the aggregated feature; The Head module is used to select detection results based on aggregated features.

4. The method for intelligent detection of surface cracks of wind turbine towers based on improved YOLOv5s according to claim 3, characterized in that: Replace the ordinary convolution of the CBS module and the C3 module in the BackBone module with coordinate convolution: O(x,y)=ΣΣI(i,j)*H(xi,yi) Among them, O(x,y) is the pixel value of the output image O, I(i,j) is the pixel value of the input image I, and H(xi,yi) is the weight of the filter H at position (xi,yi).

5. The method for intelligent detection of surface cracks of wind turbine towers based on improved YOLOv5s according to claim 4, characterized in that: The CBAM attention mechanism is introduced between the C3 module and the SPFF module: F'=MC(F)F F"=MS(F)F' Where MC(F) represents the channel attention output weight; σ represents the activation function sigmoid; Represents the feature map in space after average pooling; represents the feature map in space after maximum pooling; W0 represents the weight matrix of the first fully connected layer; W1 represents the weight matrix of the second fully connected layer; MS(F) represents the spatial attention output weight; f 7*7 Represents a convolution kernel of size 7×7; Represents the feature map after average pooling on the channel; represents the feature map after maximum pooling on the channel; F represents the input feature map; F' represents the feature map obtained by channel attention weighting; F" represents the feature map obtained by spatial attention weighting.

6. The method for intelligent detection of surface cracks of wind turbine towers based on improved YOLOv5s according to claim 3, characterized in that: The Slim-Neck module replaces all ordinary convolutions in the neck with GSConv, and then introduces the VoV-GSCSP module; the VoV-GSCSP module first reduces the dimension of the input feature map through 1×1 ordinary convolution, and inputs the result into two branches at the same time, one of which is input into the GSbottleneck module for two layers of GSConv, and the other branch is concat with the results of the two layers of GSConv, and finally outputs through 1×1 convolution; the GSConv calculation formula is: h_v'=W*concat[(h_v,AGG(h_n1),AGG(h,_n2),...,AGG(h_nk))] Among them, h_v represents the original feature vector of node v, h_n1, h_n2, …, h_nk represent the original feature vectors of the neighbor nodes of node v, and h_v' represents the updated feature vector of node v; concat represents connecting the feature vectors in series, AGG represents the function of aggregating neighbor features, and W is a learnable weight matrix.

7. The method for intelligent detection of surface cracks of wind turbine towers based on improved YOLOv5s according to claim 3, characterized in that: In the Head module, the GIOU loss function and Soft-NMS algorithm are used: Where A represents the predicted bounding box, B represents the real bounding box, and C represents the circumscribed rectangle of the two bounding boxes; Where M represents the candidate prior box with a higher score, B i Represents the candidate prior box to be processed, N t is the set threshold, IoU is the overlap score between the candidate prior frame with a high decay score and the candidate prior frame of the previous frame; S i Represents the score of the remaining i-th candidate prior box.

8. The method for intelligent detection of surface cracks of wind turbine towers based on improved YOLOv5s according to claim 3, characterized in that: The training parameters of the improved YOLOv5s are set as follows: Batch-size is 8; Workers size is 8; maximum number of iterations is 200; momentum factor is 0.937; initial learning rate is 0.01, and weight decay coefficient is 0.0005.

Images