Wind turbine blade surface damage detection method based on lightweight YOLOv8

Abstract

This invention discloses a wind turbine blade surface damage detection method based on lightweight YOLOv8, belonging to the field of wind power. The method comprises the following steps: constructing a dataset; building a wind turbine blade damage image detection model for a resource-constrained unmanned aerial vehicle (UAV) detection platform; training the blade damage image detection model to obtain an optimal blade damage image detection model; outputting wind turbine blade surface damage detection results; and verifying the effectiveness of the wind turbine blade surface damage detection results using performance evaluation indicators. While improving average detection accuracy and speed, the invention reduces the model's computational complexity and parameter count, ensuring high-average-accuracy and high-speed wind turbine blade damage detection on a resource-constrained UAV detection platform.

Description

Technical Field

[0001] The present invention relates to the field of wind power, and in particular to a method for detecting surface damage of wind turbine blades based on lightweight YOLOv8. Background Art

[0002] Because wind turbines are constantly exposed to harsh natural environments, such as strong winds, rain, snow, and dust, their blades are extremely susceptible to various types of damage. As a core component of wind turbines, the integrity of wind turbine blades is directly related to the stability and safety of the entire wind power generation system. If blade damage is not discovered and repaired promptly, it can lead to reduced wind turbine performance, downtime, or even serious accidents.

[0003] With the continuous development of technologies such as computer vision and deep learning, wind turbine blade surface damage detection technology has made significant progress. Detection methods based on image recognition and deep learning can quickly and accurately detect blade damage, improving detection efficiency and accuracy.

[0004] Existing technologies also use drones for surface damage detection. However, drones are typically resource-constrained platforms, with limited power, computing power, and resources. Wind turbines are currently trending toward larger sizes, with increasingly longer blades that are more susceptible to damage and breakage, increasing detection time and placing higher demands on drone-based detection solutions. Furthermore, drone photography can sometimes result in poor image quality due to factors such as the angle of the image, lighting, and complex backgrounds, thereby reducing average detection accuracy. Therefore, ensuring a high detection speed, lightweight model, and low computational complexity while also improving average detection accuracy is essential to meet the actual requirements of detection technology.

[0005] Although the YOLO algorithm has made great progress, damage detection of wind turbine blades photographed by drones still suffers from low average accuracy and slow speed, resulting in poor real-time detection results. Summary of the Invention

[0006] In order to solve the above technical problems, the present invention provides a wind turbine blade surface damage detection method based on lightweight YOLOv8 with a simple algorithm and higher average detection accuracy.

[0007] The technical solution of the present invention to solve the above technical problems is: a method for detecting surface damage of wind turbine blades based on lightweight YOLOv8, comprising the following steps:

[0008] S1, build a dataset: use drones to take images of wind turbine blades, annotate the images, and divide them into training set, validation set, and test set;

[0009] S2, model construction: Introducing the CBAM attention mechanism module, GSConv module, VoV-GSCSP module and Dysample module to improve YOLOv8 and build a wind turbine blade damage image detection model for resource-constrained UAV inspection platform;

[0010] S3: Use the training set to train the wind blade damage image detection model and use the test set to test it to obtain the optimal wind blade damage image detection model;

[0011] S4: Input the test set into the optimal wind turbine blade damage image detection model and output the wind turbine blade surface damage detection results;

[0012] S5: Verify the effectiveness of wind turbine blade surface damage detection results through performance evaluation indicators.

[0013] In the above-mentioned wind turbine blade surface damage detection method based on lightweight YOLOv8, the specific process of step S2 is as follows:

[0014] S21: Introducing the GSConv module into the YOLOv8 backbone, using different convolution types to extract features and promote information interaction and fusion between features;

[0015] S22: Embed the Dysample module at the beginning of the YOLOv8 neck network for dynamic upsampling;

[0016] S23: Add a one-time aggregation cross-stage partial network module, namely the VoV-GSCSP module, to the tail end of the YOLOv8 neck network. The VoV-GSCSP module is based on CSPNet and combines lightweight convolution GSConv and one-time aggregation VoV to achieve feature extraction and fusion;

[0017] S24: Add a CBAM attention mechanism module at the end of the VoV-GSCSP module to adaptively learn channels and spatial attention weights to improve the feature expression capability of the convolutional neural network;

[0018] S25: Obtain a wind turbine blade surface damage detection model based on lightweight YOLOv8.

[0019] In the above-mentioned wind turbine blade surface damage detection method based on lightweight YOLOv8, in step S21, the GSConv module constructs two convolution layers, namely, standard convolution Conv and depthwise separable convolution DWConv. The number of input channels of standard convolution is C1, the number of output channels is C3, the convolution kernel size is 1, the stride is 1, and the number of groups is 1; the number of input channels of depthwise separable convolution is C4, the number of output channels of depthwise separable convolution is C5, the convolution kernel size is 5, the stride is 1, and the number of groups is C6. C4, C5 and C6 are equal, which means that each output channel is convolved with only one input channel.

[0020] In the above-mentioned wind turbine blade surface damage detection method based on lightweight YOLOv8, in the GSConv module, the standard convolution Conv receives the feature map from the previous layer of the neural network as input, and divides the input feature map into multiple groups. Then, a standard spatial convolution operation is performed on each group to extract the spatial information in the feature map through the convolution operation and output a new feature map. The standard convolution result is further subjected to depthwise separable convolution, and the output results of the standard convolution and depthwise separable convolution are then spliced in the channel dimension. During the splicing process, the channel shuffle layer shuffle is used to group and shuffle the features in the standard convolution so that its features are completely penetrated into every part of the features generated by the depthwise separable convolution.

[0021] In the above-mentioned wind turbine blade surface damage detection method based on lightweight YOLOv8, in step S22, the input of the Dysample module is the feature map X, and the output is the sample set S. Given the upsampling scale factor s, the feature map X with a size of C×H×W, and the number of groups g, let the offset and the original sampling grid be represented by O and G respectively, then:

[0022] ;

[0023] ;

[0024] The sigmoid function is used to map the output of the linear layer to the range [0, 1], and then multiply it by 0.5 and the output of the linear layer to obtain the final offset O. The sampling set S is the sum of the offset O and the original sampling grid G. The original sampling grid G is a fixed grid that represents the position of each pixel in the input feature map.

[0025] During the sampling process, the bilinear interpolation method is used to generate the upsampled feature map based on the offset in the sampling set S and the pixel values in the original feature map.

[0026] In the above-mentioned lightweight YOLOv8-based wind turbine blade surface damage detection method, in step S23, in the VoV-GSCSP module, the input is first convolved. The convolution operation uses two convolution layers. The convolution kernel size of both convolution layers is 1, the stride is 1, and C1 is used as the input channel number and the output channel number is C2 / 2 rounded. After the convolution operation, two branch feature maps are obtained. The first branch feature map is spliced with the second branch feature map after the GSBottleneck operation;

[0027] GSBottleneck contains two GSConv and one Conv. The convolution kernel size of the first GSConv is 1, and the convolution kernel size of the second GSConv is 3. The first branch feature map outputs a preliminary feature map through the first GSConv, and the preliminary feature map then outputs the first feature map through the second GSConv; in addition, the first branch feature map outputs the second feature map through the Conv with a convolution kernel of 1; finally, the first feature map and the second feature map are added together to obtain the output result of the first branch feature map, and finally spliced with the second branch feature map.

[0028] In the above-mentioned wind turbine blade surface damage detection method based on lightweight YOLOv8, in step S24, the CBAM attention mechanism module includes a channel attention module and a spatial attention module;

[0029] In the channel attention module, global maximum pooling and global average pooling are first performed on the input feature map to generate the maximum and average values of each channel. The maximum and average values of each channel are then passed through a shared fully connected layer (MLP) to learn the attention weight of each channel. The sigmoid activation function is then used to normalize the attention weight to the range [0, 1]. Finally, the learned attention weight is multiplied by each channel of the original input feature map of the channel attention module to achieve feature recalibration in the channel dimension.

[0030] In the spatial attention module, the feature map output by the channel attention module is first average pooled and max pooled in the channel dimension to generate features of different context scales; then the features of two different context scales are spliced along the channel dimension and processed through a convolutional layer to generate spatial attention weights; then the Sigmoid activation function is used to normalize the weights to the [0,1] interval; finally, the learned spatial attention weights are multiplied by each position of the feature map output by the channel attention module to achieve feature recalibration in the spatial dimension.

[0031] In the above-mentioned wind turbine blade surface damage detection method based on lightweight YOLOv8, in step S24, in the CBAM attention mechanism module, F is used as input, F represents the input feature map, and is combined with the channel attention weight Multiply channel by channel and add spatial attention weight Multiply position by position to produce the feature map after channel attention enhancement and feature maps enhanced by spatial attention , the operation process is expressed as:

[0032] ;

[0033] in, Indicates multiplication of corresponding positions;

[0034] and The specific operation process is as follows:

[0035] ;

[0036] ;

[0037] in Indicates the average pooling operation on F; Indicates the maximum pooling operation on F; is a shared fully connected layer; 、 are weights, 、 They represent the average pooling and maximum pooling operations of F in the channel attention mechanism, 、 Respectively Average pooling and maximum pooling operations in the spatial attention mechanism, Represents the sigmoid activation function, W0, W1∈R C / r×C , R is a set of real numbers, r represents the reduction ratio, Represents a 7×7 convolution operation.

[0038] In the above-mentioned wind turbine blade surface damage detection method based on lightweight YOLOv8, in step S25, the wind turbine blade damage image detection model obtained includes a backbone network, a neck network, and a head network;

[0039] The backbone network includes a first convolution module, a second convolution module, a first C2f module, a third convolution module, a second C2f module, a GSConv module, a third C2f module, a fourth convolution module, a fourth C2f module, and an SPPF module connected in sequence;

[0040] The neck network includes a Dysample module, a first aggregation module, a fifth C2f module, an upsampling module, a second aggregation module, a sixth C2f module, a fifth convolution module, a third aggregation module, a seventh C2f module, a sixth convolution module, a fourth aggregation module, a VoV-GSCSP module, and a CBAM attention mechanism module, which are connected in sequence. The fifth C2f module is connected to the third aggregation module, the second C2f module is connected to the second aggregation module, and the third C2f module is connected to the first aggregation module. The SPPF module is connected to the Dysample module and the fourth aggregation module at the same time.

[0041] The head network includes a seventh convolution module, an eighth convolution module, a first Conv2d module, a Bbox.loss module connected in sequence, and a ninth convolution module, a tenth convolution module, a second Conv2d module, and a Cls.loss module connected in sequence. The input end of the seventh convolution module is connected to the input end of the ninth convolution module and is respectively connected to the sixth C2f module, the seventh C2f module and the CBAM attention mechanism module.

[0042] In the above-mentioned wind turbine blade surface damage detection method based on lightweight YOLOv8, in step S5, the performance evaluation indicators include precision, recall, mean average accuracy (mAP), and frame rate (FPS). The calculation formula of the performance evaluation indicators is as follows:

[0043] ;

[0044] ;

[0045] ;

[0046] ;

[0047] ;

[0048] Among them, TP represents the number of instances that are actually positive and classified as positive by the classifier, FP represents the number of instances that are actually negative but classified as positive by the classifier, FN represents the number of instances that are actually positive but classified as negative by the classifier, AP represents the average precision, k represents the kth category, and n represents the total number of categories. represents the preprocessing time, represents the inference time, Indicates post-processing time.

[0049] The beneficial effects of the present invention are as follows: To address the problem of poor detection performance in resource-constrained drone inspection platforms due to resource constraints in energy, storage, and computation, the present invention introduces a CBAM attention mechanism module, a GSConv module, a VoV-GSCSP module, and a Dysample module to improve the YOLOv8 model architecture, thereby constructing a wind turbine blade damage image detection model that can effectively detect low-quality damage images. In this wind turbine blade damage image detection model, a single GSConv is used instead of a traditional convolution module, achieving output close to that of a conventional convolution with less computational effort; the Dysample module is introduced to achieve better results with fewer parameters and latency; the VoV-GSCSP module is embedded to reduce network complexity while improving average accuracy; and the CBAM attention mechanism module is added to infer important features using two separate dimensions, thereby achieving better feature extraction capabilities. While improving average detection accuracy and detection speed, the present invention reduces the model's computational complexity and parameter count, ensuring high-average-accuracy and high-speed wind turbine blade damage detection on resource-constrained drone inspection platforms. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 It is the overall flow chart of the present invention.

[0051] Figure 2 This is the structural diagram of the GSConv module in the present invention.

[0052] Figure 3 This is a structural diagram of the Dysample module in the present invention.

[0053] Figure 4 This is a structural diagram of the VoV-GSCSP module in the present invention.

[0054] Figure 5 This is the structural diagram of the CBAM attention mechanism module in the present invention.

[0055] Figure 6 This is a structural diagram of the wind turbine blade surface damage detection model in the present invention.

[0056] Figure 7 This is a comparison chart of the detection results of the present invention and other methods.

[0057] Figure 8 This is a result diagram of the ablation experiment of the present invention. DETAILED DESCRIPTION

[0058] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0059] like Figure 1As shown, a wind turbine blade surface damage detection method based on lightweight YOLOv8 includes the following steps:

[0060] S1, build a data set: use a drone to take pictures of wind turbine blades, annotate the pictures taken by the drone, and divide them into training set, validation set and test set.

[0061] The image data in this example comes from drone photography, with a total of 1900 images. The training set and validation set have a total of 1685 images, and the test set is used as a test set, with a total of 215 images. The distribution of the divided data sets is shown in Table 1.

[0062]

[0063] S2, model construction: Introduce the CBAM attention mechanism module, GSConv module, VoV-GSCSP module and Dysample module to improve YOLOv8 and build a wind blade damage image detection model for resource-constrained UAV inspection platform.

[0064] The specific process of step S2 is:

[0065] S21: The GSConv module is introduced into the YOLOv8 backbone. The GSConv module can fully utilize different convolution types to extract features and promote information interaction and fusion between features, thereby improving the model's ability to solve problems such as high energy consumption, large model computational complexity, and long time consumption in outdoor wind turbine blade damage detection.

[0066] like Figure 2 As shown in the figure, the GSConv module constructs two convolutional layers, namely standard convolution Conv and depth-wise separable convolution DWConv. The number of input channels of standard convolution is C1, the number of output channels is C3, the convolution kernel size is 1, the stride is 1, and the number of groups is 1; the number of input channels of depth-wise separable convolution is C4, the number of output channels of depth-wise separable convolution is C5, the convolution kernel size is 5, the stride is 1, and the number of groups is C6. C4, C5 and C6 are equal, which means that each output channel is convolved with only one input channel.

[0067] In the GSConv module, the standard convolution Conv receives the feature map from the previous layer of the neural network as input, divides the input feature map into multiple groups, and then performs a standard spatial convolution operation on each group to extract the spatial information in the feature map through the convolution operation and output a new feature map; the standard convolution result is further subjected to depthwise separable convolution, and then the output results of the standard convolution and depthwise separable convolution are spliced in the channel dimension. During the splicing process, the channel shuffle layer shuffle is used to group and shuffle the features in the standard convolution so that its features can fully penetrate into every part of the features generated by the depthwise separable convolution.

[0068] S22: Embed the Dysample module at the beginning of the YOLOv8 neck network for dynamic upsampling. This strategy not only makes the model more lightweight, but also improves the average detection accuracy.

[0069] like Figure 3 As shown, the Dysample module simplifies the upsampling process from the perspective of point sampling to achieve the advantages of low latency, few parameters and low computation. The feature map is input into the Dysample module, and the Dysample module creates a sampling set through a sampling point generator. The sampling point generator can use the dynamic range factor method to generate an offset and add it to the original grid position to obtain a sampling set. The input features are resampled using the grid_sample function, and the input features are upsampled according to the sampling set. After the upsampling operation, the Dysample module outputs the upsampled feature map; Figure 3 Pixel / shuffle represents the "shuffle" operation, which shuffles and reorganizes the features.

[0070] The input of the Dysample module is the feature map X, and the output is the sample set S. Given the upsampling scale factor s, the feature map X of size C×H×W, and the number of groups g, let the offset and the original sampling grid be represented by O and G respectively, then:

[0071] ;

[0072] ;

[0073] The sigmoid function is used to map the output of the linear layer to the range [0, 1], and then multiply it by 0.5 and the output of the linear layer to obtain the final offset O. The sampling set S is the sum of the offset O and the original sampling grid G. The original sampling grid G is a fixed grid that represents the position of each pixel in the input feature map.

[0074] During the sampling process, the bilinear interpolation method is used to generate the upsampled feature map based on the offset in the sampling set S and the pixel values in the original feature map.

[0075] S23: Add a one-time aggregated cross-stage partial network module (VoVNet-based Grouped Spatial Cross-Stage Partial Networks), namely the VoV-GSCSP module, to the tail end of the YOLOv8 neck network. The VoV-GSCSP module is based on CSPNet and combines lightweight convolution GSConv and one-time aggregate VoV to achieve feature extraction and fusion.

[0076] like Figure 4 As shown in the figure, in the VoV-GSCSP module, the input is first convolved. The convolution operation uses two convolution layers. The convolution kernel size of the two convolution layers is 1, the stride is 1, and the number of channels output is C2 / 2 rounded to the integer of the input channel number C1. After the convolution operation, two branch feature maps are obtained. The first branch feature map is spliced with the second branch feature map after the GSBottleneck operation;

[0077] GSBottleneck contains two GSConv and one Conv. The convolution kernel size of the first GSConv is 1, and the convolution kernel size of the second GSConv is 3. The first branch feature map outputs a preliminary feature map through the first GSConv, and the preliminary feature map then outputs the first feature map through the second GSConv; in addition, the first branch feature map outputs the second feature map through the Conv with a convolution kernel of 1; finally, the first feature map and the second feature map are added together to obtain the output result of the first branch feature map, and finally spliced with the second branch feature map.

[0078] S24: Add a CBAM attention mechanism module at the end of the VoV-GSCSP module to adaptively learn channels and spatial attention weights to improve the feature expression ability of the convolutional neural network.

[0079] like Figure 5 As shown, the CBAM attention mechanism module includes a channel attention module and a spatial attention module; Figure 5 In the example, Input Feature is the input feature, Shared MLP is the shared fully connected layer, Sigmoid is the activation function, Channel-refined feature is the channel output feature, Conv layer is the convolution layer, and Refined Feature is the output feature.

[0080] In the channel attention module, global maximum pooling and global average pooling are first performed on the input feature map to generate the maximum and average values of each channel. The maximum and average values of each channel are then passed through a shared fully connected layer (MLP) to learn the attention weight of each channel. The sigmoid activation function is then used to normalize the attention weight to the range [0, 1]. Finally, the learned attention weight is multiplied by each channel of the original input feature map of the channel attention module to achieve feature recalibration in the channel dimension.

[0081] In the spatial attention module, the feature map output by the channel attention module is first average pooled and max pooled in the channel dimension to generate features of different context scales; then the features of two different context scales are spliced along the channel dimension and processed through a convolutional layer to generate spatial attention weights; then the Sigmoid activation function is used to normalize the weights to the [0,1] interval; finally, the learned spatial attention weights are multiplied by each position of the feature map output by the channel attention module to achieve feature recalibration in the spatial dimension.

[0082] In the CBAM attention mechanism module, F is used as input, F represents the input feature map, and is combined with the channel attention weight Multiply channel by channel and add spatial attention weight Multiply position by position to produce the feature map after channel attention enhancement and feature maps enhanced by spatial attention , the operation process is expressed as:

[0083] ;

[0084] in, Indicates multiplication of corresponding positions;

[0085] and The specific operation process is as follows:

[0086] ;

[0087] ;

[0088] in Indicates the average pooling operation on F; Indicates the maximum pooling operation on F; is a shared fully connected layer; 、 are weights, 、 They represent the average pooling and maximum pooling operations of F in the channel attention mechanism, 、 Respectively Average pooling and maximum pooling operations in the spatial attention mechanism, Represents the sigmoid activation function, W0, W1∈R C / r×C , R is a set of real numbers, r represents the reduction ratio, Represents a 7×7 convolution operation.

[0089] S25: Obtain a wind turbine blade damage image detection model for a resource-constrained UAV inspection platform.

[0090] like Figure 6 As shown in Figure 2, the wind turbine blade damage image detection model includes a backbone network, a neck network, and a head network;

[0091] The backbone network Backbone includes a first convolution module, a second convolution module, a first C2f module, a third convolution module, a second C2f module, a GSConv module, a third C2f module, a fourth convolution module, a fourth C2f module, and an SPPF module connected in sequence;

[0092] The neck network Neck includes a Dysample module, a first aggregation module, a fifth C2f module, an upsampling module, a second aggregation module, a sixth C2f module, a fifth convolution module, a third aggregation module, a seventh C2f module, a sixth convolution module, a fourth aggregation module, a VoV-GSCSP module, and a CBAM attention mechanism module, which are connected in sequence, wherein the fifth C2f module is connected to the third aggregation module, the second C2f module is connected to the second aggregation module, the third C2f module is connected to the first aggregation module, and the SPPF module is connected to the Dysample module and the fourth aggregation module at the same time;

[0093] The head network includes a seventh convolution module, an eighth convolution module, a first Conv2d module, a Bbox.loss module connected in sequence, and a ninth convolution module, a tenth convolution module, a second Conv2d module, and a Cls.loss module connected in sequence. The input end of the seventh convolution module is connected to the input end of the ninth convolution module and is respectively connected to the sixth C2f module, the seventh C2f module and the CBAM attention mechanism module.

[0094] S3: The wind turbine blade surface damage detection model is trained using the training set and tested using the test set to obtain the optimal wind turbine blade surface damage detection model.

[0095] S4: Input the test set into the optimal wind blade surface damage detection model and output the wind blade surface damage detection results.

[0096] S5: Verify the effectiveness of wind turbine blade surface damage detection results through performance evaluation indicators.

[0097] Performance evaluation indicators include precision (P), recall (R), average accuracy (mAP), and frame rate (FPS). The calculation formula for performance evaluation indicators is as follows:

[0098] ;

[0099] ;

[0100] ;

[0101] ;

[0102] ;

[0103] Among them, TP represents the number of instances that are actually positive and classified as positive by the classifier, FP represents the number of instances that are actually negative but classified as positive by the classifier, FN represents the number of instances that are actually positive but classified as negative by the classifier, AP represents the average precision, k represents the kth category, and n represents the total number of categories. represents the preprocessing time, represents the inference time, Indicates post-processing time.

[0104] This example uses mAP@50 and FPS as performance reference indicators. mAP@50 represents the mean average accuracy (mAP) when the Intersection over Union (IoU) threshold is 0.5. A higher mAP value indicates a higher overall average accuracy of the model. FPS is a metric that measures the frame rate at which a system or algorithm processes images or videos. It represents the number of frames that can be processed or displayed per second and directly reflects the system or algorithm's ability to process images or videos per unit time.

[0105] This embodiment uses the methods of ablation experiments and comparative experiments to verify the effectiveness of the improved algorithm. The ablation experiment is shown in Table 3, and the comparative experiment is shown in Table 2. In Table 2, Param represents the parameter quantity, GFLOPs is one billion floating-point operations per second, Crack is the average accuracy of crack damage, Layers is the average accuracy of molting damage, and Trachorma is the average accuracy of trachoma damage.

[0106]

[0107]

[0108] The experiment used YOLOv8n as the baseline algorithm on a wind turbine blade surface damage dataset. After testing, it was found that YOLOv8n has a great advantage in computing speed, but there is still room for improvement. Subsequent improvements to the model framework need to further improve the speed while improving the average accuracy.

[0109] As can be seen from Table 2, the latest mainstream series of YOLO are compared respectively. Among all the compared models, the present invention has the highest average accuracy, the fastest detection speed, the least number of model parameters and the lowest computational complexity. Among them, the number of parameters and computational complexity of the YOLOv8n model are similar to those of the model of the present invention, but its average accuracy is 9.5% lower. The present invention achieves the highest average accuracy in the detection of crack damage and molting damage. For trachoma damage, the average detection accuracy of the present invention is only 4.1% lower than that of the YOLOv12m model. However, the number of parameters is only about 14.43% of that of the YOLOv12m model, and its average accuracy of 95.136% is achieved, highlighting its extremely high cost-effectiveness. Based on the experimental results and analysis, the model of the present invention shows strong comprehensive performance in terms of average detection accuracy, detection speed, model size and computational complexity.

[0110] Table 3 shows that, in Experiment A, the introduction of the GSConv and VoV-GSCSP modules, respectively, resulted in improved detection speed and a slight increase in average accuracy, though the model parameters were slightly higher than the original model. Comparisons with the original model in terms of mAP (mean average accuracy), FPS (frames per second), and params (parameter count) indicate that the improved accuracy is due to GSConv's ability to preserve important feature information while reducing the computational burden. These positive experimental results directly support the effectiveness of GSConv and VoV-GSCSP in object detection tasks and demonstrate that they can significantly improve overall model performance by optimizing model structure and improving feature extraction efficiency.

[0111] Experiments were conducted to introduce both the Dysample module and the CBAM module. When the Dysample module was introduced alone, the results of Experiment B showed that Dysample was more efficient than the original model in terms of weight reduction, computational speed, and accuracy. When the CBAM module was added alone, the average accuracy in Experiment C improved by 3.2% compared to the original model. Due to the additional computational steps, the model's advantages in terms of parameter count and computational speed were lower than those of the original model. When performance improvements were not significant, detection efficiency decreased.

[0112] In Experiment D, the combination of Dysample and CBAM achieved significant improvements. This is primarily due to the fact that Dysample's lightweighting compensates for the decreased detection efficiency, while the attention mechanism further enhances the model's focus on key features. The combination of channel attention and spatial attention helps the model focus on the most important feature channels and spatial locations, improving feature processing accuracy. Experiment D demonstrates the effectiveness of combining Dysample with the CBAM attention mechanism in improving detection performance.

[0113] Compared to Experiment D, Experiment E, which used GSConv alone, and Experiment F, which used VoV-GSCSP alone, showed a decrease in accuracy and number of model parameters compared to Experiment D, indicating that the optimization strategies of these two modules have certain advantages in terms of lightweightness. The reason for the decrease in accuracy is that the feature maps of feature fusion are affected by dynamic upsampling and the attention mechanism, resulting in the omission of important features in feature extraction.

[0114] Throughout the experiment, we verified the detection results of the GSconv combined with VoV-GSCSP and the Dysample dynamic upsampling combined with the CBAM attention mechanism module. The experimental results show that these two combined optimization strategies can effectively improve the overall performance of the model. When only GSconv combined with VoV-GSCSP is introduced, the model parameters increase and the calculation speed increases by 17.17fps, but the average accuracy only increases by 1.9%. At the same time, the Dysample and CBAM optimization strategies are independently verified, and the average accuracy increases by 3.9% and 3.2%, respectively. The calculation speed of the Dysample dynamic upsampling strategy increases by 15.8fps, while the calculation speed of the CBAM attention mechanism optimization strategy decreases by 2.95fps. Then, combining the two optimization strategies, as shown in Experiment D, achieves comprehensive improvements in lightweighting, average accuracy, and speed. Finally, based on Experiment D, GSconv combined with VoV-GSCSP is also introduced, as shown in Experiment G, to form a comprehensive model. The overall results show that the model of the present invention has the best overall performance. The model has a minimum parameter of only 2.9M, a frame rate of 171.28fps, and a mAP of 80.8%. This further demonstrates that in Experiment D, GSConv and VoV-GSCSP can fully leverage the advantages of both in the feature fusion process, achieving not only a lightweight model but also improved accuracy and performance.

Claims

1. A wind turbine blade surface damage detection method based on lightweight YOLOv8, characterized in that: The following steps are involved: S1, build a dataset: use drones to take images of wind turbine blades, annotate the images, and divide them into training set, validation set, and test set; S2, model construction: Introducing the CBAM attention mechanism module, GSConv module, VoV-GSCSP module and Dysample module to improve YOLOv8 and build a wind turbine blade damage image detection model for resource-constrained UAV inspection platform; The specific process of step S2 is: S21: Introducing the GSConv module into the YOLOv8 backbone, using different convolution types to extract features and promote information interaction and fusion between features; S22: Embed the Dysample module at the beginning of the YOLOv8 neck network for dynamic upsampling. This strategy not only makes the model more lightweight, but also improves the average detection accuracy. S23: Add a one-time aggregation cross-stage partial network module, namely the VoV-GSCSP module, to the tail end of the YOLOv8 neck network. The VoV-GSCSP module is based on CSPNet and combines lightweight convolution GSConv and one-time aggregation VoV to achieve feature extraction and fusion; S24: Add a CBAM attention mechanism module at the end of the VoV-GSCSP module to adaptively learn channels and spatial attention weights to improve the feature expression capability of the convolutional neural network; S25: Obtain a wind turbine blade surface damage detection model based on lightweight YOLOv8; In step S25, the obtained wind turbine blade damage image detection model includes a backbone network, a neck network and a head network; The backbone network includes a first convolution module, a second convolution module, a first C2f module, a third convolution module, a second C2f module, a GSConv module, a third C2f module, a fourth convolution module, a fourth C2f module, and an SPPF module connected in sequence; The neck network includes a Dysample module, a first aggregation module, a fifth C2f module, an upsampling module, a second aggregation module, a sixth C2f module, a fifth convolution module, a third aggregation module, a seventh C2f module, a sixth convolution module, a fourth aggregation module, a VoV-GSCSP module, and a CBAM attention mechanism module, which are connected in sequence, wherein the fifth C2f module is connected to the third aggregation module, the second C2f module is connected to the second aggregation module, the third C2f module is connected to the first aggregation module, and the SPPF module is connected to the Dysample module and the fourth aggregation module at the same time; The head network includes a seventh convolution module, an eighth convolution module, a first Conv2d module, a Bbox.loss module connected in sequence, and a ninth convolution module, a tenth convolution module, a second Conv2d module, and a Cls.loss module connected in sequence. The input end of the seventh convolution module is connected to the input end of the ninth convolution module and is respectively connected to the sixth C2f module, the seventh C2f module, and the CBAM attention mechanism module; S3: Use the training set to train the wind blade damage image detection model and use the test set to test it to obtain the optimal wind blade damage image detection model; S4: Input the test set into the optimal wind turbine blade damage image detection model and output the wind turbine blade surface damage detection results; S5: Verify the effectiveness of wind turbine blade surface damage detection results through performance evaluation indicators.

2. The wind turbine blade surface damage detection method based on lightweight YOLOv8 according to claim 1 is characterized in that: In step S21, the GSConv module constructs two convolution layers, namely, standard convolution Conv and depth-wise separable convolution DWConv. The number of input channels of standard convolution is C1, the number of output channels is C3, the convolution kernel size is 1, the stride is 1, and the number of groups is 1; the number of input channels of depth-wise separable convolution is C4, the number of output channels of depth-wise separable convolution is C5, the convolution kernel size is 5, the stride is 1, and the number of groups is C6. C4, C5 and C6 are equal, which means that each output channel is convolved with only one input channel.

3. The wind turbine blade surface damage detection method based on lightweight YOLOv8 according to claim 2 is characterized in that: In the GSConv module, the standard convolution Conv receives the feature map from the previous layer of the neural network as input, divides the input feature map into multiple groups, and then performs a standard spatial convolution operation on each group to extract the spatial information in the feature map through the convolution operation and output a new feature map; the standard convolution result is further subjected to depthwise separable convolution, and then the output results of the standard convolution and depthwise separable convolution are spliced in the channel dimension. During the splicing process, the channel shuffle layer shuffle is used to group and shuffle the features in the standard convolution so that its features can fully penetrate into every part of the features generated by the depthwise separable convolution.

4. The wind turbine blade surface damage detection method based on lightweight YOLOv8 according to claim 1 is characterized in that: In step S22, the input of the Dysample module is the feature map X, and the output is the sample set S. Given the upsampling scale factor s, the feature map X of size C×H×W, and the number of groups g, let the offset and the original sampling grid be represented by O and G respectively, then: S=G+O O=0.5sigmoid(linear1(X))·linear2(X) The sigmoid function is used to map the output of the linear layer to the range [0, 1], and then multiply it by 0.5 and the output of the linear layer to obtain the final offset O. The sampling set S is the sum of the offset O and the original sampling grid G. The original sampling grid G is a fixed grid that represents the position of each pixel in the input feature map. During the sampling process, the bilinear interpolation method is used to generate the upsampled feature map based on the offset in the sampling set S and the pixel values in the original feature map.

5. The wind turbine blade surface damage detection method based on lightweight YOLOv8 according to claim 1 is characterized in that: In step S23, in the VoV-GSCSP module, the input is first convolved. The convolution operation uses two convolution layers. The convolution kernel size of the two convolution layers is 1, the stride is 1, and the number of channels output is C2 / 2 rounded up with C1 as the input channel number. After the convolution operation, two branch feature maps are obtained. The first branch feature map is spliced with the second branch feature map after the GSBottleneck operation; GSBottleneck contains two GSConvs and one Conv. The convolution kernel size of the first GSConv is 1, and the convolution kernel size of the second GSConv is 3. The first branch feature map outputs a preliminary feature map through the first GSConv, and the preliminary feature map then outputs the first feature map through the second GSConv; in addition, the first branch feature map outputs the second feature map through the Conv with a convolution kernel of 1; finally, the first feature map and the second feature map are added to obtain the output result of the first branch feature map, and finally spliced with the second branch feature map.

6. The wind turbine blade surface damage detection method based on lightweight YOLOv8 according to claim 4 is characterized in that: In step S24, the CBAM attention mechanism module includes a channel attention module and a spatial attention module; In the channel attention module, global maximum pooling and global average pooling are first performed on the input feature map to generate the maximum and average values of each channel. The maximum and average values of each channel are then passed through a shared fully connected layer (MLP) to learn the attention weight of each channel. The sigmoid activation function is then used to normalize the attention weight to the range [0, 1]. Finally, the learned attention weight is multiplied by each channel of the original input feature map of the channel attention module to achieve feature recalibration in the channel dimension. In the spatial attention module, the feature map output by the channel attention module is first average pooled and max pooled in the channel dimension to generate features of different context scales; The features of two different context scales are then concatenated along the channel dimension and processed through a convolutional layer to generate spatial attention weights. The sigmoid activation function is then used to normalize the weights to the [0, 1] interval. Finally, the learned spatial attention weights are multiplied by each position of the feature map output by the channel attention module to achieve feature recalibration in the spatial dimension.

7. The wind turbine blade surface damage detection method based on lightweight YOLOv8 according to claim 6 is characterized in that: In step S24, the CBAM attention mechanism module takes F as input, where F represents the input feature map, and is combined with the channel attention weight M C (F) Channel-by-channel multiplication and spatial attention weight M S (F′) is multiplied position by position to derive the feature map F′ after channel attention enhancement and the feature map F″ after spatial attention enhancement. The operation process is expressed as: in, Indicates multiplication of corresponding positions; M C (F) and M S The specific operation process of (F′) is as follows: Where AvgPool(F) represents the average pooling operation on F; MaxPool(F) represents the maximum pooling operation on F; MLP is a shared fully connected layer; W0 and W1 are weights. They represent the average pooling and maximum pooling operations of F in the channel attention mechanism, Respectively represent the average pooling and maximum pooling operations of F′ in the spatial attention mechanism, σ represents the sigmoid activation function, W0, W1∈R C / r×C , R is a set of real numbers, r represents the reduction ratio, f 7×7 Represents a 7×7 convolution operation.

8. The wind turbine blade surface damage detection method based on lightweight YOLOv8 according to claim 1 is characterized in that: In step S5, the performance evaluation indicators include precision, recall, average accuracy (mAP), and frame rate (FPS). The calculation formula of the performance evaluation indicators is as follows: Among them, TP represents the number of instances that are actually positive and classified as positive by the classifier, FP represents the number of instances that are actually negative but classified as positive by the classifier, FN represents the number of instances that are actually positive but classified as negative by the classifier, AP represents the average precision, k represents the kth category, n represents the total number of categories, T_pre represents the preprocessing time, T_inf represents the inference time, and T_post represents the post-processing time.

Images