Fan blade surface damage detection method based on lightweight YOLOv8

Abstract

The invention discloses a fan blade surface damage detection method based on lightweight YOLOv8, and belongs to the field of wind power, and the method comprises the following steps: constructing a data set; constructing a fan blade damage image detection model for the resource-constrained unmanned aerial vehicle detection platform; training the blade damage image detection model to obtain an optimal blade damage image detection model; outputting a fan blade surface damage detection result; and carrying out validity verification on the surface damage detection result of the fan blade through the performance evaluation index. According to the method, the calculation complexity and the parameter quantity of the model are reduced while the average detection accuracy and the detection speed are improved, and it is ensured that high-average-accuracy and high-speed wind driven generator blade damage detection is achieved on a resource-limited unmanned aerial vehicle detection platform.

Description

Technical Field

[0001] The present invention relates to the field of wind power, and in particular to a method for detecting surface damage of wind turbine blades based on lightweight YOLOv8. Background Art

[0002] Since wind turbines are exposed to harsh natural environments for a long time, such as strong winds, rain, snow, sand and dust, the blades are very susceptible to various damages. As one of the core components of wind turbines, the integrity of wind turbine blades is directly related to the stability and safety of the entire wind power generation system. If blade damage is not discovered and repaired in time, it may lead to performance degradation, shutdown and even serious accidents of wind turbines.

[0003] With the continuous development of computer vision, deep learning and other technologies, wind turbine blade surface damage detection technology has made significant progress. Detection methods based on image recognition, deep learning and other technologies can quickly and accurately detect blade damage and improve detection efficiency and accuracy.

[0004] There are also existing technologies that use drones to perform surface damage detection. Drones are a typical resource-constrained platform with limited power, computing, resources, etc. However, wind turbines are now developing in large sizes, with longer and longer blades, which are more susceptible to damage and breakage, and increase the time for detection, which puts higher demands on drone-based detection solutions. In addition, drone photography may sometimes cause low image quality due to factors such as the angle of the photo, light, and complex background, thereby reducing the average accuracy of the detection. Therefore, while ensuring the high detection speed, lightweight, and low computational complexity of the model, it can also improve the average accuracy of the detection, which meets the actual detection technology requirements.

[0005] At present, the YOLO algorithm has made great progress. However, the damage detection of wind turbine blades photographed by drones still has the defects of low average detection accuracy and slow speed, and the real-time detection effect is not good. Summary of the invention

[0006] In order to solve the above technical problems, the present invention provides a wind turbine blade surface damage detection method based on lightweight YOLOv8 with a simple algorithm and higher average detection accuracy.

[0007] The technical solution of the present invention to solve the above technical problem is: a method for detecting surface damage of wind turbine blades based on lightweight YOLOv8, comprising the following steps: S1, constructing a data set: taking images of wind turbine blades by drones, annotating the images of wind turbine blades taken by drones, and dividing them into training set, validation set and test set; S2, model construction: introduce CBAM attention mechanism module, GSConv module, VoV-GSCSP module and Dysample module to improve YOLOv8, and build a wind turbine blade damage image detection model for resource-constrained UAV inspection platform; S3: The fan blade damage image detection model is trained using the training set and tested using the test set to obtain the optimal fan blade damage image detection model; S4: input the test set into the optimal fan blade damage image detection model, and output the fan blade surface damage detection result; S5: Verify the effectiveness of wind turbine blade surface damage detection results through performance evaluation indicators.

[0008] In the above-mentioned wind turbine blade surface damage detection method based on lightweight YOLOv8, the specific process of step S2 is as follows: S21: Introduce the GSConv module into the YOLOv8 backbone, use different convolution types to extract features, and promote information interaction and fusion between features; S22: Embed the Dysample module at the head end of the YOLOv8 neck network for dynamic upsampling; S23: Add a one-time aggregated cross-stage partial network module, namely the VoV-GSCSP module, to the tail end of the YOLOv8 neck network. The VoV-GSCSP module is based on CSPNet and combines lightweight convolution GSConv and one-time aggregated VoV to achieve feature extraction and fusion; S24: Add a CBAM attention mechanism module at the end of the VoV-GSCSP module to adaptively learn channels and spatial attention weights to improve the feature expression ability of the convolutional neural network; S25: Obtain a wind turbine blade surface damage detection model based on lightweight YOLOv8.

[0009] In the above-mentioned wind turbine blade surface damage detection method based on lightweight YOLOv8, in step S21, the GSConv module constructs two convolution layers, namely, standard convolution Conv and depthwise separable convolution DWConv, the number of input channels of standard convolution is C1, the number of output channels is C3, the convolution kernel size is 1, the stride is 1, and the number of groups is 1; the number of input channels of depthwise separable convolution is C4, the number of output channels of depthwise separable convolution is C5, the convolution kernel size is 5, the stride is 1, and the number of groups is C6, C4, C5 and C6 are equal, which means that each output channel is convolved with only one input channel.

[0010] In the above-mentioned wind turbine blade surface damage detection method based on lightweight YOLOv8, in the GSConv module, the standard convolution Conv receives the feature map from the previous layer of the neural network as input, and divides the input feature map into multiple groups, and then performs a standard spatial convolution operation on each group, extracts the spatial information in the feature map through the convolution operation, and outputs a new feature map; the standard convolution result is further subjected to a depthwise separable convolution, and then the output results of the standard convolution and the depthwise separable convolution are spliced ​​in the channel dimension. During the splicing process, the channel shuffle layer shuffle is used to group and shuffle the features in the standard convolution so that its features can fully penetrate into every part of the features generated by the depthwise separable convolution.

[0011] In the above-mentioned wind turbine blade surface damage detection method based on lightweight YOLOv8, in step S22, the input of the Dysample module is the feature map X, and the output is the sampling set S. Given the upsampling scale factor s, the feature map X with a size of C×H×W, and the number of groups g, let the offset and the original sampling grid be represented by O and G respectively, then: ; ; The sigmoid function is used to map the output of the linear layer to [0,1], and then multiply it by 0.5 and the output of the linear layer to get the final offset O. The sampling set S is the sum of the offset O and the original sampling grid G. The original sampling grid G ​​is a fixed grid that represents the position of each pixel in the input feature map. During the sampling process, the bilinear interpolation method is used to generate the upsampled feature map based on the offset in the sampling set S and the pixel value in the original feature map.

[0012] In the above-mentioned wind turbine blade surface damage detection method based on lightweight YOLOv8, in step S23, in the VoV-GSCSP module, the input is first convolved, and the convolution operation uses two convolution layers. The convolution kernel size of the two convolution layers is 1, the stride is 1, and C1 is used as the input channel number to output the channel number rounded to C2 / 2. After the convolution operation, two branch feature maps are obtained, and the first branch feature map is spliced ​​with the second branch feature map after the GSBottleneck operation; GSBottleneck contains two GSConvs and one Conv. The convolution kernel size of the first GSConv is 1, and the convolution kernel size of the second GSConv is 3. The first branch feature map outputs a preliminary feature map through the first GSConv, and the preliminary feature map outputs the first feature map through the second GSConv. In addition, the first branch feature map outputs the second feature map through the Conv with a convolution kernel of 1. Finally, the first feature map and the second feature map are added to obtain the output result of the first branch feature map, which is finally spliced ​​with the second branch feature map.

[0013] In the above-mentioned wind turbine blade surface damage detection method based on lightweight YOLOv8, in step S24, the CBAM attention mechanism module includes a channel attention module and a spatial attention module; In the channel attention module, global maximum pooling and global average pooling are first performed on the input feature map to generate the maximum and average values ​​of each channel; then the maximum and average values ​​of each channel are passed through the shared fully connected layer MLP to learn the attention weight of each channel; then the Sigmoid activation function is used to normalize the attention weight to the [0,1] interval; finally, the learned attention weight is multiplied with each channel of the original input feature map of the channel attention module to achieve feature recalibration in the channel dimension; In the spatial attention module, the feature map output by the channel attention module is first average pooled and max pooled in the channel dimension to generate features of different context scales. Then, the features of two different context scales are concatenated along the channel dimension and processed through a convolutional layer to generate spatial attention weights. The sigmoid activation function is then used to normalize the weights to the [0,1] interval. Finally, the learned spatial attention weights are multiplied by each position of the feature map output by the channel attention module to achieve feature recalibration in the spatial dimension.

[0014] In the above-mentioned wind turbine blade surface damage detection method based on lightweight YOLOv8, in step S24, in the CBAM attention mechanism module, F is used as input, where F represents the input feature map, and is combined with the channel attention weight Multiply channel by channel and add spatial attention weight Multiply position by position to get the feature map after channel attention enhancement And the feature map after spatial attention enhancement , the operation process is expressed as: ; in, Indicates multiplication of corresponding positions; and The specific operation process is as follows: ; ; in Indicates that an average pooling operation is performed on F; Indicates the maximum pooling operation on F; is a shared fully connected layer; , are weights, , They represent the average pooling and maximum pooling operations of F in the channel attention mechanism, respectively. , Respectively Average pooling and maximum pooling operations in the spatial attention mechanism, Represents the sigmoid activation function, W0, W1∈R C / r×C , R is a real number set, r represents the reduction ratio, Represents a 7×7 convolution operation.

[0015] In the above-mentioned wind turbine blade surface damage detection method based on lightweight YOLOv8, in step S25, the wind turbine blade damage image detection model obtained includes a backbone network, a neck network and a head network; The backbone network includes a first convolution module, a second convolution module, a first C2f module, a third convolution module, a second C2f module, a GSConv module, a third C2f module, a fourth convolution module, a fourth C2f module, and an SPPF module connected in sequence; The neck network includes a Dysample module, a first aggregation module, a fifth C2f module, an upsampling module, a second aggregation module, a sixth C2f module, a fifth convolution module, a third aggregation module, a seventh C2f module, a sixth convolution module, a fourth aggregation module, a VoV-GSCSP module, and a CBAM attention mechanism module, which are connected in sequence, wherein the fifth C2f module is connected to the third aggregation module, the second C2f module is connected to the second aggregation module, the third C2f module is connected to the first aggregation module, and the SPPF module is connected to the Dysample module and the fourth aggregation module at the same time; The head network includes a seventh convolution module, an eighth convolution module, a first Conv2d module, a Bbox.loss module connected in sequence, and a ninth convolution module, a tenth convolution module, a second Conv2d module, and a Cls.loss module connected in sequence. The input end of the seventh convolution module is connected to the input end of the ninth convolution module and is respectively connected to the sixth C2f module, the seventh C2f module and the CBAM attention mechanism module.

[0016] In the above-mentioned wind turbine blade surface damage detection method based on lightweight YOLOv8, in step S5, the performance evaluation indicators include precision Precision, recall rate Recall, average accuracy mAP, and frame rate per second FPS. The calculation formula of the performance evaluation indicators is as follows: ; ; ; ; ; Among them, TP represents the number of instances that are actually positive and classified as positive by the classifier, FP represents the number of instances that are actually negative but classified as positive by the classifier, FN represents the number of instances that are actually positive but classified as negative by the classifier, AP represents the average precision, k represents the kth classification, and n represents the total number of classifications. represents the preprocessing time, represents the inference time, Indicates the post-processing time.

[0017] The beneficial effects of the present invention are as follows: in view of the problem that the detection performance of resource-constrained UAV detection platforms is poor due to resource constraints in terms of energy, storage, and computing, the present invention introduces the CBAM attention mechanism module, GSConv module, VoV-GSCSP module, and Dysample module to improve the YOLOv8 model architecture, and constructs a wind turbine blade damage image detection model that can effectively detect low-quality damage images. In the wind turbine blade damage image detection model, a GSConv is used once to replace the traditional convolution module, so that the output is close to the conventional convolution with less calculation; the Dysample module is introduced to achieve better results with fewer parameters and delays; the VoV-GSCSP module is embedded to improve the average accuracy while reducing the network complexity; the CBAM attention mechanism module is added to make inferences in two separate dimensions to extract important features, thereby obtaining better feature extraction capabilities. The present invention reduces the computational complexity and parameter amount of the model while improving the average detection accuracy and detection speed, ensuring that high average accuracy and high speed wind turbine blade damage detection is achieved on a resource-constrained UAV detection platform. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is the overall flow chart of the present invention.

[0019] Figure 2 It is a structural diagram of the GSConv module in the present invention.

[0020] Figure 3It is a structural diagram of the Dysample module in the present invention.

[0021] Figure 4 It is a structural diagram of the VoV-GSCSP module in the present invention.

[0022] Figure 5 This is a structural diagram of the CBAM attention mechanism module in the present invention.

[0023] Figure 6 It is a structural diagram of the wind turbine blade surface damage detection model in the present invention.

[0024] Figure 7 This is a comparison chart of the detection results of the present invention and other methods.

[0025] Figure 8 This is a result diagram of the ablation experiment of the present invention. DETAILED DESCRIPTION

[0026] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0027] like Figure 1 As shown, a method for detecting surface damage of a wind turbine blade based on lightweight YOLOv8 includes the following steps: S1, build a data set: use a drone to take pictures of wind turbine blades, annotate the pictures taken by the drone, and divide them into training set, validation set and test set.

[0028] The image data in this example comes from drone photography, with a total of 1900 pictures, 1685 pictures in the training set and validation set, and 215 pictures in the test set. The distribution of the divided data sets is shown in Table 1.

[0029]

[0030] S2, model construction: introduce CBAM attention mechanism module, GSConv module, VoV-GSCSP module and Dysample module to improve YOLOv8 and build a wind blade damage image detection model for resource-constrained UAV inspection platform.

[0031] The specific process of step S2 is: S21: The GSConv module is introduced into the YOLOv8 backbone. The GSConv module can make full use of different convolution types to extract features and promote information interaction and fusion between features, thereby improving the model's ability to solve problems such as high energy consumption, large model calculation, and long time consumption in outdoor wind turbine blade damage detection.

[0032] like Figure 2As shown in the figure, the GSConv module constructs two convolution layers, namely standard convolution Conv and depth-wise separable convolution DWConv. The number of input channels of standard convolution is C1, the number of output channels is C3, the convolution kernel size is 1, the stride is 1, and the number of groups is 1; the number of input channels of depth-wise separable convolution is C4, the number of output channels of depth-wise separable convolution is C5, the convolution kernel size is 5, the stride is 1, and the number of groups is C6. C4, C5 and C6 are equal, which means that each output channel is convolved with only one input channel.

[0033] In the GSConv module, the standard convolution Conv receives the feature map from the previous layer of the neural network as input, divides the input feature map into multiple groups, and then performs a standard spatial convolution operation on each group. The spatial information in the feature map is extracted through the convolution operation, and a new feature map is output; the standard convolution result is further subjected to a depthwise separable convolution, and then the output results of the standard convolution and the depthwise separable convolution are spliced ​​in the channel dimension. During the splicing process, the channel shuffle layer shuffle is used to group and shuffle the features in the standard convolution so that its features can fully penetrate into every part of the features generated by the depthwise separable convolution.

[0034] S22: The Dysample module is embedded at the beginning of the YOLOv8 neck network for dynamic upsampling. This strategy not only makes the model lighter, but also improves the average detection accuracy.

[0035] like Figure 3 As shown, the Dysample module simplifies the upsampling process through the perspective of point sampling to achieve the advantages of low latency, few parameters and less computation. The feature map is input into the Dysample module, and the Dysample module creates a sampling set through a sampling point generator. The sampling point generator can use the dynamic range factor method to generate an offset and add it to the original grid position to obtain a sampling set. The input features are resampled using the grid_sample function, and the input features are upsampled according to the sampling set. After the upsampling operation, the Dysample module outputs the upsampled feature map; Figure 3 Pixel / shuffle stands for the “shuffle” operation, which shuffles and reorganizes the features.

[0036] The input of the Dysample module is the feature map X, and the output is the sample set S. Given the upsampling scale factor s, the feature map X of size C×H×W, and the number of groups g, let the offset and the original sampling grid be represented as O and G respectively, then: ; ; The sigmoid function is used to map the output of the linear layer to [0,1], and then multiply it by 0.5 and the output of the linear layer to get the final offset O. The sampling set S is the sum of the offset O and the original sampling grid G. The original sampling grid G ​​is a fixed grid that represents the position of each pixel in the input feature map. During the sampling process, the bilinear interpolation method is used to generate the upsampled feature map based on the offset in the sampling set S and the pixel value in the original feature map.

[0037] S23: Add a one-time aggregated cross-stage partial network module (VoVNet-based Grouped Spatial Cross-Stage Partial Networks), namely the VoV-GSCSP module, to the tail end of the YOLOv8 neck network. The VoV-GSCSP module is based on CSPNet and combines lightweight convolution GSConv and one-time aggregation VoV to achieve feature extraction and fusion.

[0038] like Figure 4 As shown in the figure, in the VoV-GSCSP module, the input is first convolved. The convolution operation uses two convolution layers. The convolution kernel size of the two convolution layers is 1, the stride is 1, and C1 is used as the input channel number and the output channel number is the integer of C2 / 2. After the convolution operation, two branch feature maps are obtained. The first branch feature map is spliced ​​with the second branch feature map after the GSBottleneck operation; GSBottleneck contains two GSConvs and one Conv. The convolution kernel size of the first GSConv is 1, and the convolution kernel size of the second GSConv is 3. The first branch feature map outputs a preliminary feature map through the first GSConv, and the preliminary feature map outputs the first feature map through the second GSConv. In addition, the first branch feature map outputs the second feature map through the Conv with a convolution kernel of 1. Finally, the first feature map and the second feature map are added to obtain the output result of the first branch feature map, which is finally spliced ​​with the second branch feature map.

[0039] S24: Add a CBAM attention mechanism module at the end of the VoV-GSCSP module to adaptively learn channels and spatial attention weights to improve the feature expression ability of the convolutional neural network.

[0040] like Figure 5 As shown, the CBAM attention mechanism module includes a channel attention module and a spatial attention module; Figure 5In it, Input Feature is the input feature, Shared MLP is the shared fully connected layer, Sigmoid is the activation function, Channel-refined feature is the channel output feature, Conv layer is the convolution layer, and Refined Feature is the output feature; In the channel attention module, global maximum pooling and global average pooling are first performed on the input feature map to generate the maximum and average values ​​of each channel; then the maximum and average values ​​of each channel are passed through the shared fully connected layer MLP to learn the attention weight of each channel; then the Sigmoid activation function is used to normalize the attention weight to the [0,1] interval; finally, the learned attention weight is multiplied with each channel of the original input feature map of the channel attention module to achieve feature recalibration in the channel dimension; In the spatial attention module, the feature map output by the channel attention module is first average pooled and max pooled in the channel dimension to generate features of different context scales. Then, the features of two different context scales are concatenated along the channel dimension and processed through a convolutional layer to generate spatial attention weights. The sigmoid activation function is then used to normalize the weights to the [0,1] interval. Finally, the learned spatial attention weights are multiplied by each position of the feature map output by the channel attention module to achieve feature recalibration in the spatial dimension.

[0041] In the CBAM attention mechanism module, F is used as input, where F represents the input feature map and is combined with the channel attention weight Multiply channel by channel and add spatial attention weight Multiply position by position to get the feature map after channel attention enhancement And the feature map after spatial attention enhancement , the operation process is expressed as: ; in, Indicates multiplication of corresponding positions; and The specific operation process is as follows: ; ; in Indicates that an average pooling operation is performed on F; Indicates the maximum pooling operation on F; is a shared fully connected layer; , are weights, , They represent the average pooling and maximum pooling operations of F in the channel attention mechanism, respectively. , Respectively Average pooling and maximum pooling operations in the spatial attention mechanism, Represents the sigmoid activation function, W0, W1∈R C / r×C , R is a real number set, r represents the reduction ratio, Represents a 7×7 convolution operation.

[0042] S25: Obtain a wind turbine blade damage image detection model for resource-constrained UAV inspection platform.

[0043] like Figure 6 As shown, the wind turbine blade damage image detection model includes a backbone network, a neck network, and a head network; The backbone network Backbone includes a first convolution module, a second convolution module, a first C2f module, a third convolution module, a second C2f module, a GSConv module, a third C2f module, a fourth convolution module, a fourth C2f module, and an SPPF module connected in sequence; The neck network Neck includes a Dysample module, a first aggregation module, a fifth C2f module, an upsampling module, a second aggregation module, a sixth C2f module, a fifth convolution module, a third aggregation module, a seventh C2f module, a sixth convolution module, a fourth aggregation module, a VoV-GSCSP module, and a CBAM attention mechanism module, which are connected in sequence, wherein the fifth C2f module is connected to the third aggregation module, the second C2f module is connected to the second aggregation module, the third C2f module is connected to the first aggregation module, and the SPPF module is connected to the Dysample module and the fourth aggregation module at the same time; The head network includes a seventh convolution module, an eighth convolution module, a first Conv2d module, a Bbox.loss module connected in sequence, and a ninth convolution module, a tenth convolution module, a second Conv2d module, and a Cls.loss module connected in sequence. The input end of the seventh convolution module is connected to the input end of the ninth convolution module and is respectively connected to the sixth C2f module, the seventh C2f module and the CBAM attention mechanism module.

[0044] S3: The wind blade surface damage detection model is trained using the training set and tested using the test set to obtain the optimal wind blade surface damage detection model.

[0045] S4: Input the test set into the optimal wind blade surface damage detection model and output the wind blade surface damage detection result.

[0046] S5: Verify the effectiveness of wind turbine blade surface damage detection results through performance evaluation indicators.

[0047] The performance evaluation indicators include precision (P), recall (R), average accuracy mAP, and frame rate per second FPS. The calculation formula of performance evaluation indicators is as follows: ; ; ; ; ; Among them, TP represents the number of instances that are actually positive and classified as positive by the classifier, FP represents the number of instances that are actually negative but classified as positive by the classifier, FN represents the number of instances that are actually positive but classified as negative by the classifier, AP represents the average precision, k represents the kth classification, and n represents the total number of classifications. represents the preprocessing time, represents the inference time, Indicates the post-processing time.

[0048] This example uses mAP@50 and FPS as performance reference indicators. mAP@50 represents the mAP when the IoU threshold is 0.5. The larger the mAP, the higher the overall average accuracy of the model. FPS is an indicator that measures the frame rate of a system or algorithm processing images or videos. It indicates the number of frames that can be processed or displayed per second. It directly reflects the ability of the system or algorithm to process images or videos in a unit of time.

[0049] This embodiment adopts the method of ablation experiment and comparative experiment to verify the effectiveness of the improved algorithm. The ablation experiment is shown in Table 3, and the comparative experiment is shown in Table 2. In Table 2, Param represents the parameter quantity, GFLOPs is one billion floating point operations per second, Crack is the average accuracy of crack damage, Layers is the average accuracy of molting damage, and Trachorma is the average accuracy of trachoma damage.

[0050]

[0051]

[0052] The experiment used YOLOv8n as the baseline algorithm on the wind turbine blade surface damage dataset. After the experiment, it was found that YOLOv8n has a great advantage in computing speed, but there is still room for improvement. Subsequent improvements to the model framework need to further improve the speed while improving the average accuracy.

[0053] As can be seen from Table 2, the latest mainstream series of YOLO are compared respectively. Among all the compared models, the present invention has the highest average accuracy, the fastest detection speed, the least number of model parameters and the lowest computational complexity. Among them, the number of parameters and computational complexity of the YOLOv8n model are similar to those of the model of the present invention, but its average accuracy is 9.5% lower. The present invention achieves the highest average accuracy in the detection of crack damage and molting damage. For trachoma damage, the average detection accuracy of the present invention is only 4.1% lower than that of the YOLOv12m model. However, the number of parameters is only about 14.43% of that of the YOLOv12m model, and its average accuracy of 95.136% is achieved, highlighting the extremely high cost performance. Based on the experimental results and analysis, the model of the present invention shows strong comprehensive performance in terms of average detection accuracy, detection speed, model size and computational complexity.

[0054] As shown in Table 3, after introducing the GSConv module and the VoV-GSCSP module respectively, in Experiment A, we can see that the detection speed has been improved and the average accuracy has also been slightly improved, but the model parameters are slightly higher than the original model. By comparing with the original model in terms of mAP (average accuracy), FPS (frames per second) and Param (parameter quantity), the improvement in accuracy is because GSConv can reduce the computational burden while retaining important feature information. These positive experimental results directly support the effectiveness of GSConv and VoV-GSCSP in target detection tasks, and prove that they can significantly improve the overall performance of the model by optimizing the model structure and improving the efficiency of feature extraction.

[0055] We tried to introduce the Dysample module and the CBAM module. After introducing the Dysample module alone, the results of Experiment B showed that Dysample ran more efficiently than the original model in terms of lightweight, calculation speed, and accuracy. After adding the CBAM module alone, the average accuracy in Experiment C was 3.2% higher than the original model. Due to the increase in calculation steps in the model, the model has lower advantages in the number of parameters and calculation speed than the original model. When the performance improvement is not obvious, the detection efficiency decreases.

[0056] In Experiment D, by combining Dysample and CBAM, the improvement is significant. The main reason is that the lightweight of Dysample makes up for the decline in detection efficiency, and the attention mechanism can further enhance the model's attention to key features. The combination of channel attention and spatial attention helps the model focus on the most important feature channels and spatial positions, improving the accuracy of feature processing. Experiment D proves that the combination of Dysample and CBAM attention mechanism is effective in improving detection performance.

[0057] Based on experiment D, the accuracy and number of model parameters of experiment E using GSConv alone and experiment F using VoV-GSCSP alone decreased compared with experiment D, which shows that the optimization strategies of these two modules have certain advantages in lightweight. The reason for the decrease in accuracy is that the feature map of feature fusion is affected by dynamic upsampling and attention mechanism, resulting in the omission of important features in feature extraction.

[0058] Throughout the experiment, the detection results of GSconv combined with VoV-GSCSP and Dysample dynamic upsampling combined with CBAM attention mechanism modules were verified respectively. The experimental results show that the optimization strategies of these two combinations can effectively improve the overall performance of the model. When only GSconv combined with VoV-GSCSP was introduced, the model parameters increased, the calculation speed increased by 17.17fps, but the average accuracy was only increased by 1.9%. At the same time, the Dysample and CBAM optimization strategies were independently verified, and the average accuracy was increased by 3.9% and 3.2% respectively. The calculation speed of the Dysample dynamic sampling strategy increased by 15.8fps, while the calculation speed of the CBAM attention mechanism optimization strategy decreased by 2.95fps. Then, the two optimization strategies were combined, as shown in Experiment D, to achieve a comprehensive improvement in lightweight, average accuracy and speed. Finally, on the basis of Experiment D, GSconv combined with VoV-GSCSP was introduced at the same time, as shown in Experiment G. The overall results show that the model of the present invention has the best comprehensive performance. The minimum parameter of the model is only 2.9M, the frame rate is 171.28fps, and the mAP is 80.8%. This further shows that in Experiment D, GSConv and VoV-GSCSP can give full play to their advantages in the feature fusion process, which not only achieves the lightweight of the model, but also improves the accuracy and performance of the model.

Claims

1. A wind turbine blade surface damage detection method based on lightweight YOLOv8, characterized in that: The following steps are involved: S1, build a dataset: use drones to take images of wind turbine blades, annotate the images, and divide them into training set, validation set, and test set; S2, model construction: Introducing the CBAM attention mechanism module, GSConv module, VoV-GSCSP module and Dysample module to improve YOLOv8 and build a wind turbine blade damage image detection model for resource-constrained UAV inspection platform; S3: Use the training set to train the wind blade damage image detection model and use the test set to test it to obtain the optimal wind blade damage image detection model; S4: Input the test set into the optimal wind turbine blade damage image detection model and output the wind turbine blade surface damage detection results; S5: Verify the effectiveness of wind turbine blade surface damage detection results through performance evaluation indicators.

2. The wind turbine blade surface damage detection method based on lightweight YOLOv8 according to claim 1 is characterized in that: The specific process of step S2 is: S21: Introducing the GSConv module into the YOLOv8 backbone, using different convolution types to extract features and promote information interaction and fusion between features; S22: Embed the Dysample module at the beginning of the YOLOv8 neck network for dynamic upsampling. This strategy not only makes the model more lightweight, but also improves the average detection accuracy. S23: Add a one-time aggregation cross-stage partial network module, namely the VoV-GSCSP module, to the tail end of the YOLOv8 neck network. The VoV-GSCSP module is based on CSPNet and combines lightweight convolution GSConv and one-time aggregation VoV to achieve feature extraction and fusion; S24: Add a CBAM attention mechanism module at the end of the VoV-GSCSP module to adaptively learn channels and spatial attention weights to improve the feature expression capability of the convolutional neural network; S25: Obtain a wind turbine blade surface damage detection model based on lightweight YOLOv8.

3. The wind turbine blade surface damage detection method based on lightweight YOLOv8 according to claim 2 is characterized in that: In step S21, the GSConv module constructs two convolution layers, namely, standard convolution Conv and depth-wise separable convolution DWConv. The number of input channels of standard convolution is C1, the number of output channels is C3, the convolution kernel size is 1, the stride is 1, and the number of groups is 1; the number of input channels of depth-wise separable convolution is C4, the number of output channels of depth-wise separable convolution is C5, the convolution kernel size is 5, the stride is 1, and the number of groups is C6. C4, C5 and C6 are equal, which means that each output channel is convolved with only one input channel.

4. The wind turbine blade surface damage detection method based on lightweight YOLOv8 according to claim 3 is characterized in that: In the GSConv module, the standard convolution Conv receives the feature map from the previous layer of the neural network as input, divides the input feature map into multiple groups, and then performs a standard spatial convolution operation on each group to extract the spatial information in the feature map through the convolution operation and output a new feature map; the standard convolution result is further subjected to depthwise separable convolution, and then the output results of the standard convolution and depthwise separable convolution are spliced ​​in the channel dimension. During the splicing process, the channel shuffle layer shuffle is used to group and shuffle the features in the standard convolution so that its features can fully penetrate into every part of the features generated by the depthwise separable convolution.

5. The wind turbine blade surface damage detection method based on lightweight YOLOv8 according to claim 2 is characterized in that: In step S22, the input of the Dysample module is the feature map X, and the output is the sample set S. Given the upsampling scale factor s, the feature map X of size C×H×W, and the number of groups g, let the offset and the original sampling grid be represented by O and G respectively, then: ； ； The sigmoid function is used to map the output of the linear layer to the range [0, 1], and then multiply it by 0.5 and the output of the linear layer to obtain the final offset O. The sampling set S is the sum of the offset O and the original sampling grid G. The original sampling grid G ​​is a fixed grid that represents the position of each pixel in the input feature map. During the sampling process, the bilinear interpolation method is used to generate the upsampled feature map based on the offset in the sampling set S and the pixel values ​​in the original feature map.

6. The wind turbine blade surface damage detection method based on lightweight YOLOv8 according to claim 2 is characterized in that: In step S23, in the VoV-GSCSP module, the input is first convolved. The convolution operation uses two convolution layers. The convolution kernel size of the two convolution layers is 1, the stride is 1, and the number of channels output is C2 / 2 rounded up with C1 as the input channel number. After the convolution operation, two branch feature maps are obtained. The first branch feature map is spliced ​​with the second branch feature map after the GSBottleneck operation; GSBottleneck contains two GSConv and one Conv. The convolution kernel size of the first GSConv is 1, and the convolution kernel size of the second GSConv is 3. The first branch feature map outputs a preliminary feature map through the first GSConv, and the preliminary feature map then outputs the first feature map through the second GSConv; in addition, the first branch feature map outputs the second feature map through the Conv with a convolution kernel of 1; finally, the first feature map and the second feature map are added together to obtain the output result of the first branch feature map, and finally spliced ​​with the second branch feature map.

7. The wind turbine blade surface damage detection method based on lightweight YOLOv8 according to claim 5 is characterized in that: In step S24, the CBAM attention mechanism module includes a channel attention module and a spatial attention module; In the channel attention module, global maximum pooling and global average pooling are first performed on the input feature map to generate the maximum and average values ​​of each channel. The maximum and average values ​​of each channel are then passed through a shared fully connected layer (MLP) to learn the attention weight of each channel. The sigmoid activation function is then used to normalize the attention weight to the range [0, 1]. Finally, the learned attention weight is multiplied by each channel of the original input feature map of the channel attention module to achieve feature recalibration in the channel dimension. In the spatial attention module, the feature map output by the channel attention module is first average pooled and max pooled in the channel dimension to generate features of different context scales; The features of two different context scales are then concatenated along the channel dimension and processed through a convolutional layer to generate spatial attention weights. The sigmoid activation function is then used to normalize the weights to the [0, 1] interval. Finally, the learned spatial attention weights are multiplied by each position of the feature map output by the channel attention module to achieve feature recalibration in the spatial dimension.

8. The wind turbine blade surface damage detection method based on lightweight YOLOv8 according to claim 7 is characterized in that: In step S24, the CBAM attention mechanism module takes F as input, where F represents the input feature map, and is combined with the channel attention weight Multiply channel by channel and add spatial attention weight Multiply position by position to produce the feature map after channel attention enhancement and the feature map after spatial attention enhancement , the operation process is expressed as: ； in, Indicates multiplication of corresponding positions; and The specific operation process is as follows: ； ； in Indicates the average pooling operation on F; Indicates the maximum pooling operation on F; is a shared fully connected layer; 、 are weights, 、 They represent the average pooling and maximum pooling operations of F in the channel attention mechanism, 、 Respectively Average pooling and maximum pooling operations in the spatial attention mechanism, Represents the sigmoid activation function, W0, W1∈R C / r×C , R is a set of real numbers, r represents the reduction ratio, Represents a 7×7 convolution operation.

9. The wind turbine blade surface damage detection method based on lightweight YOLOv8 according to claim 2 is characterized in that: In step S25, the obtained wind turbine blade damage image detection model includes a backbone network, a neck network and a head network; The backbone network includes a first convolution module, a second convolution module, a first C2f module, a third convolution module, a second C2f module, a GSConv module, a third C2f module, a fourth convolution module, a fourth C2f module, and an SPPF module connected in sequence; The neck network includes a Dysample module, a first aggregation module, a fifth C2f module, an upsampling module, a second aggregation module, a sixth C2f module, a fifth convolution module, a third aggregation module, a seventh C2f module, a sixth convolution module, a fourth aggregation module, a VoV-GSCSP module, and a CBAM attention mechanism module, which are connected in sequence. The fifth C2f module is connected to the third aggregation module, the second C2f module is connected to the second aggregation module, and the third C2f module is connected to the first aggregation module. The SPPF module is connected to the Dysample module and the fourth aggregation module at the same time. The head network includes a seventh convolution module, an eighth convolution module, a first Conv2d module, a Bbox.loss module connected in sequence, and a ninth convolution module, a tenth convolution module, a second Conv2d module, and a Cls.loss module connected in sequence. The input end of the seventh convolution module is connected to the input end of the ninth convolution module and is respectively connected to the sixth C2f module, the seventh C2f module and the CBAM attention mechanism module.

10. The wind turbine blade surface damage detection method based on lightweight YOLOv8 according to claim 1 is characterized in that: In step S5, the performance evaluation indicators include precision, recall, average accuracy (mAP), and frame rate (FPS). The calculation formula of the performance evaluation indicators is as follows: ； ； ； ； ； Among them, TP represents the number of instances that are actually positive and classified as positive by the classifier, FP represents the number of instances that are actually negative but classified as positive by the classifier, FN represents the number of instances that are actually positive but classified as negative by the classifier, AP represents the average precision, k represents the kth category, and n represents the total number of categories. represents the preprocessing time, represents the inference time, Indicates post-processing time.

Images