Offshore wind turbine blade surface defect detection method based on improved YOLOV8

Abstract

The application belongs to the technical field of computer vision, and specifically provides a kind of offshore wind turbine blade surface defect detection method based on improved YOLOV8, comprising: offshore wind turbine blade defect dataset is constructed, and the defects on the image are labeled, and training set, verification set and test set are constructed based on the dataset;YOLOv8 baseline network model is improved, C2f-RepGhost lightweight module is introduced to replace C2f module in main feature extraction network, BiFPN structure is used to improve it to realize the purpose of feature fusion, and attention module CBAM is introduced to construct the final defect detection model;The network model of YOLOv8 is trained to optimize;Based on the trained weight file, input program, test the detection speed and accuracy of YOLOv8.The improved YOLOv8 algorithm can effectively improve the accuracy and speed of offshore wind turbine blade surface defect recognition, while reducing the parameter amount of the model and reducing the calculation cost, and has better performance compared with other mainstream algorithms, which can be applied to many places.

Description

Technical Field

[0001] This invention belongs to the field of computer vision technology, specifically, it relates to a method for detecting surface defects on offshore wind turbine blades based on an improved YOLOV8. Background Technology

[0002] Wind power generation is the primary form of wind energy utilization and one of the most commercially promising power generation methods. Currently, wind power construction in China is gradually expanding from inland areas to nearshore areas. However, due to the operating environment of offshore wind farms, offshore wind power equipment is susceptible to environmental factors such as humid and hot air, ultraviolet radiation, and corrosion from high-salt spray sea breezes. Wind turbine blades are a crucial component of wind turbines and are more prone to defects during operation, leading to reduced power generation or even equipment damage. To reduce maintenance time and costs, timely detection and early warning of surface defects in offshore wind turbine blades are essential.

[0003] Traditional manual inspection methods are inefficient. Currently, using drones to acquire images and employing computer vision methods to detect blade defects has become the mainstream approach. Deep convolutional neural networks have demonstrated powerful capabilities in learning high-level image features. YOLOv8 is currently a widely used object detection model that is lightweight and performs well. However, for wind turbine blade faults, the blade defects are relatively small, posing challenges for YOLOv8 in feature extraction and processing, and the accuracy of defect detection and classification needs improvement. How to improve the network's ability to detect small targets and achieve a balance between detection speed and accuracy is an urgent problem to be solved. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a surface defect detection method for offshore wind turbine blades based on the improved YOLOV8. In view of the problems of low accuracy and high model complexity in the current surface defect detection of wind turbine blades, the present invention provides an efficient and accurate surface defect detection solution.

[0005] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is: a method for detecting surface defects of offshore wind turbine blades based on an improved YOLOV8, comprising the following steps: Step 1: Construct a dataset of defects in offshore wind turbine blades; Step 2: Constructing a defect detection model: Improve the YOLOv8 baseline network model by introducing a lightweight C2f-RepGhost module to replace the C2f module in the backbone feature extraction network, using a BiFPN structure for feature fusion, introducing an attention module CBAM, and replacing the CSP module with a lightweight VoV-GSCSP module to construct the final defect detection model based on the improved YOLOv8. Step 3: Train to optimize the YOLOv8 network model; Step 4: Based on the trained weight file, input it into the program to test the detection speed and accuracy of YOLOv8.

[0006] In the preferred embodiment, in step 1, images of defects in offshore wind turbine blades are collected using a drone, and the defects in the dataset are labeled using annotation software to generate an annotation file containing defect categories and predicted bounding box coordinate information. The image dataset is then divided into a training set, a validation set, and a test set.

[0007] In a preferred embodiment, step 2 involves a defect detection model based on the improved YOLOv8, comprising a backbone network module, a neck module, and a prediction module. The output of the backbone network module is connected to the input of the neck module, and the output of the neck module is connected to the input of the prediction module. The backbone network module includes a CBS module, an improved C2f_RepGhost module, and an SPPF module; The neck module includes an improved BiFPN layer; The prediction module includes convolutional layers.

[0008] In a preferred embodiment, step 2, in which the backbone network module extracts features from the image dataset of surface defects on offshore wind turbine blades, includes the following steps: 1) The CBS module based on the backbone network module performs slice convolution calculation on the image dataset of surface defects on offshore wind turbine blades to obtain the feature information of the sliced ​​surface defects on offshore wind turbine blades. 2) An improved C2f_RepGhost module based on the backbone network module is used to reduce the dimensionality of the sliced ​​offshore wind turbine blade surface defect image dataset to obtain the dimensionality-reduced offshore wind turbine blade surface defect feature information. 3) Based on the backbone network module, the SPPF module performs spatial pyramid pooling and channel compression operations on the image dataset of surface defects of offshore wind turbine blades with global features to obtain the surface defect features of offshore wind turbine blades.

[0009] In a preferred embodiment, the improved C2f_RepGhost module replaces the duplicate Bottleneck module in the C2f module with RepGhostBottleneck based on GhostConv.

[0010] In a preferred embodiment, the neck module includes: an improved BiFPN feature fusion network based on a feature fusion module, a CBAM attention mechanism, a CBS module, a C2f module, and a splicing operation.

[0011] In a preferred embodiment, the improved BiFPN feature fusion network in the neck module is an improved BiFPN network that adds an edge linking the upper layer output and the lower layer input. The feature fusion network includes a first layer input, a second layer input, a third layer input, a first layer fusion output, a second layer intermediate output, a second layer fusion output, and a third layer fusion output; In the feature fusion module, the first-layer fusion output part receives the first-layer input and the second-layer intermediate output and fuses the two parts of the features and outputs them; the second-layer intermediate output part receives the second-layer input and the third-layer input and fuses the two parts of the features and outputs them; the second-layer fusion output part receives the second-layer input and the second-layer intermediate output and fuses the two parts of the features and outputs them; the third-layer fusion output part receives the third-layer input and the second-layer fusion output and fuses the two parts of the features and outputs them; finally, an edge connecting the second-layer input and the third-layer fusion output is added.

[0012] In the preferred scheme, the introduction of CBAM into the feature fusion network involves the following steps: The CBAM attention module consists of two parts: a channel attention module and a spatial attention module. In the channel attention module, the input feature map is subjected to global max pooling and global average pooling operations for each channel, generating two vectors containing the number of channels. After MLP operation, the channel attention weight vector is obtained. The weights are normalized by the Sigmoid function and finally weighted and added to the original input feature map. In the spatial attention module, the input feature map is first subjected to max pooling and average pooling operations in the channel dimension, the two output feature maps are concatenated in the channel dimension, and finally a convolutional neural network is used to further generate a spatial attention map.

[0013] In the preferred embodiment, in step 3, the method for training the network model is as follows: during the training phase, the number of training iterations is set to 500, the batch size is 4, and the initial learning rate is 0.01. The target detection network is trained iteratively a preset number of times until the prediction accuracy of the target detection network for the images in the validation set reaches a preset accuracy threshold.

[0014] In the preferred embodiment, in step 4, the recall, accuracy, and precision of the preliminary model are measured based on the test set. If the recall, accuracy, and precision all meet the preset standards, the preliminary model is determined to be the recognition model. The image of the wind turbine blade to be detected is input into the trained target detection network, and the defect detection results output by the trained target detection network are obtained.

[0015] The present invention provides a method for detecting surface defects in offshore wind turbine blades based on an improved YOLOV8, which has the following advantages: 1. In view of the problems of low accuracy and high model complexity in the current detection of surface defects of wind turbine blades, this invention provides an efficient and accurate surface defect detection solution.

[0016] 2. In step 1, an image dataset was created using manual annotation and divided into training, validation, and test sets according to a set ratio to reduce the risk of model overfitting.

[0017] 3. In step 2, a new C2f-RepGhost module is designed in the YOLOv8n backbone network to replace the C2f module, which improves the detection speed of the model and reduces the number of model parameters. In the neck module, the CBAM attention module is introduced to make the model pay more attention to small target information, and an improved BiFPN structure is used for feature fusion, which can effectively improve the recognition and localization accuracy of small target defects and has better performance compared with other mainstream algorithms.

[0018] 4. In steps 3 and 4, when the model training is completed and the deployment phase begins, the model can not only effectively improve the accuracy of defect identification, but also reduce the number of model parameters and reduce computational costs, which is of great significance to edge terminal devices. Attached Figure Description

[0019] The present invention will be further described below with reference to the accompanying drawings and embodiments: Figure 1 This is a flowchart of the present invention; Figure 2 The structure diagram for training and inference of the original GhostBottleneck and RepGhostBottleneck; Figure 3 This is a schematic diagram of C2f_RepGhost. Figure 4 A diagram of the improved BiFPN structure; Figure 5 This is a schematic diagram of the CBAM attention module. Figure 6 This is a diagram of the overall network structure. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the invention.

[0021] A method for detecting surface defects in offshore wind turbine blades based on an improved YOLOv8, such as... Figure 1 As shown, it includes the following steps: Step 1: Construct a dataset of defects in offshore wind turbine blades: Images of defects in offshore wind turbine blades were collected using drones. Annotation software was used to label the defects in the dataset, generating an annotation file containing defect categories and predicted bounding box coordinates. The image dataset was divided into training, validation, and test sets in an 8:1:1 ratio.

[0022] Step 2: Improve the YOLOv8 baseline network model by introducing the C2f-RepGhost lightweight module to replace the C2f module in the backbone feature extraction network, using the BiFPN structure for feature fusion, introducing the attention module CBAM, and replacing the CSP module with the VoV-GSCSP lightweight module to construct the final defect detection model based on the improved YOLOv8.

[0023] The defect detection model based on the improved YOLOv8 includes a backbone network module, a neck module, and a prediction module. The output of the backbone network module is connected to the input of the neck module, and the output of the neck module is connected to the input of the prediction module.

[0024] in: The backbone network module includes a CBS module, an improved C2f module, and an SPPF module.

[0025] The neck module includes an improved BiFPN layer.

[0026] The prediction module includes convolutional layers.

[0027] The steps for feature extraction from a dataset of images showing surface defects on offshore wind turbine blades specifically include: The CBS module based on the backbone network module performs slice convolution calculation on the image dataset of surface defects on offshore wind turbine blades to obtain the feature information of the sliced ​​offshore wind turbine blade surface defect images. An improved C2f_RepGhost module based on the backbone network module is used to reduce the dimensionality of the sliced ​​offshore wind turbine blade surface defect image dataset to obtain the dimensionality-reduced offshore wind turbine blade surface defect feature information. The SPPF module based on the backbone network module performs spatial pyramid pooling and channel compression operations on the image dataset of surface defects on offshore wind turbine blades with global features to obtain the surface defect features of offshore wind turbine blades.

[0028] The design concept of the improved C2f_RepGhost module based on the backbone network module is as follows: replace the duplicate Bottleneck module in C2f with RepGhostBottleneck based on GhostConv, such as... Figure 2 and Figure 3 As shown.

[0029] The neck module includes: an improved BiFPN feature fusion network based on the feature fusion module, a CBAM attention mechanism, a CBS module, a C2f module, and a splicing operation.

[0030] The improved BiFPN feature fusion network adds an edge linking the upper layer output and the lower layer input to the basic BiFPN network; The feature fusion module includes a first-layer input, a second-layer input, a third-layer input, a first-layer fusion output, a second-layer intermediate output, a second-layer fusion output, and a third-layer fusion output.

[0031] In the feature fusion module, the first-layer fusion output section receives the first-layer input and the second-layer intermediate output, and fuses the two feature parts into an output; the second-layer intermediate output section receives the second-layer input and the third-layer input, and fuses the two feature parts into an output; the second-layer fusion output section receives the second-layer input and the second-layer intermediate output, and fuses the two feature parts into an output; the third-layer fusion output section receives the third-layer input and the second-layer fusion output, and fuses the two feature parts into an output; finally, an edge connecting the second-layer input and the third-layer fusion output is added, such as... Figure 4 As shown, taking the second layer as an example, the principle is as follows: (1); (2); In the formula, , (i=1, 2, 3…) are the learnable weights of each node participating in the computation. , , These represent the inputs for the first to third layers, respectively. This indicates the intermediate output of the second layer. , These represent the fused outputs of the first and second layers, respectively. Resize() is a scaling function that allows features at different scales to be fused, Conv is a convolution operation, and ε = 0.0001 is used to ensure numerical stability.

[0032] The introduction of CBAM into the feature fusion network of the neck module follows these steps: The input feature map F undergoes global max pooling and global average pooling operations on each feature map in the channel attention mechanism, generating two vectors containing the number of channels. These vectors are then processed through a fully connected MLP to obtain the channel attention weight vector. The weights are normalized using the sigmoid function and finally weighted and added back to the original input feature map. The principle is as follows: (3); in, This represents the weight vector after the channel attention mechanism. For activation function, W 1. W 0 indicates the weight parameters in the fully connected process; AvgPool(F) and This represents average pooling of F. MaxPool(F) and This represents max pooling of F; In the spatial attention module, the input feature map First, max pooling and average pooling operations are performed along the channel dimension. The two output feature maps are then concatenated along the channel dimension. Finally, a convolutional neural network is used to further generate a spatial attention map. The principle is as follows: (4); in, M s (F) This represents the weight vector after the channel attention mechanism. This represents a 7×7 convolution.

[0033] The overall structure of CBAM is as follows Figure 5 As shown.

[0034] The overall structure of the improved YOLOv8 model is as follows: Figure 6 As shown.

[0035] Step 3: Train the network model, the steps are as follows: During the training phase, the number of training iterations was set to 500, the batch size to 4, and the initial learning rate to 0.01. The target detection network was trained iteratively a preset number of times until the prediction accuracy of the target detection network for the images in the validation set reached a preset accuracy threshold.

[0036] Step 4: Save the trained weight file to the improved YOLOv8 network model. Then, based on the trained weight file, test the detection performance of the improved YOLOv8 and output the test results. Measure the recall, accuracy, and precision of the preliminary model based on the test set. If the recall, accuracy, and precision all meet the preset standards, the preliminary model is determined to be the recognition model. Input the image of the wind turbine blade to be detected into the trained target detection network and obtain the defect detection results output by the trained target detection network. To evaluate the ability of different module combinations to optimize the model, ablation experiments were designed, and the data are shown in Table 1. As the improved modules were gradually introduced, the overall performance of the network model continuously improved. Comparing Experiments 1 and 2, it can be seen that after introducing the CBAM module, the model's precision and mAP values ​​increased by 2.4% and 1.5%, respectively, but the increased number of parameters led to a slight decrease in detection speed. Comparing Experiments 1 and 3, it can be seen that after introducing the improved BiFPN, the detection accuracy improved, although the number of parameters did not change significantly, the detection speed decreased by 1.8 FPS. Comparing Experiments 1 and 4, it can be found that adding the lightweight RepGhost module significantly improved the detection speed by 3.4 FPS, but the detection accuracy decreased slightly. When both CBAM and BiFPN were added, the model could not only retain small target information in the feature extraction stage, but also enhance the ability to fuse low semantic information in the feature fusion stage, retaining more small target information and further improving the model's detection accuracy. However, due to the increased computational cost, the detection speed decreased. Finally, after introducing the improved modules, the model achieved the best overall combination of detection speed and accuracy.

[0037]

[0038] To further verify the algorithm's performance, under the same configuration and dataset, the performance of the improved YOLOv8 was compared with other mainstream object detection algorithms such as Faster R-CNN and SSD, as shown in Table 2. Although the detection speed was not optimal, it achieved the best results in detection accuracy, obtaining the highest precision and mAP value, with lower cost, higher detection efficiency, and simpler deployment.

[0039]

[0040] In summary, this invention improves upon the problems of low efficiency and poor accuracy in detecting surface defects on offshore wind turbine blades. On a self-made dataset, the model of this invention achieved significant improvements in all performance metrics, validating the effectiveness and practicality of the method. This work provides an efficient and accurate solution for surface defect detection in offshore wind turbine blade inspection, possessing promising application prospects and practical significance.

[0041] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for detecting surface defects in offshore wind turbine blades based on an improved YOLOv8, characterized in that, Includes the following steps: Step 1: Construct a dataset of defects in offshore wind turbine blades; Step 2: Constructing a defect detection model: Improve the YOLOv8 baseline network model by introducing a lightweight C2f-RepGhost module to replace the C2f module in the backbone feature extraction network, using a BiFPN structure for feature fusion, introducing an attention module CBAM, and replacing the CSP module with a lightweight VoV-GSCSP module to construct the final defect detection model based on the improved YOLOv8. The defect detection model based on the improved YOLOv8 includes a backbone network module, a neck module, and a prediction module. The output of the backbone network module is connected to the input of the neck module, and the output of the neck module is connected to the input of the prediction module. The backbone network module includes a CBS module, an improved C2f_RepGhost module, and an SPPF module; The improved C2f_RepGhost module replaces the duplicate Bottleneck module in the C2f module with RepGhostBottleneck based on GhostConv. The neck module includes an improved BiFPN layer; Specifically, the neck module includes: an improved BiFPN feature fusion network based on the feature fusion module, a CBAM attention mechanism, a CBS module, a C2f module, and a splicing operation; The improved BiFPN feature fusion network in the neck module adds an edge linking the upper layer output and the lower layer input to the basic BiFPN network; The feature fusion network includes a first layer input, a second layer input, a third layer input, a first layer fusion output, a second layer intermediate output, a second layer fusion output, and a third layer fusion output; In the feature fusion module, the first layer fusion output part receives the first layer input and the second layer intermediate output and fuses the two parts of features and outputs them; the second layer intermediate output part receives the second layer input and the third layer input and fuses the two parts of features and outputs them; the second layer fusion output part receives the second layer input and the second layer intermediate output and fuses the two parts of features and outputs them; the third layer fusion output part receives the third layer input and the second layer fusion output and fuses the two parts of features and outputs them; finally, an edge connecting the second layer input and the third layer fusion output is added. The prediction module includes convolutional layers; The backbone network module extracts features from a dataset of images showing surface defects on offshore wind turbine blades, including the following steps: 1) The CBS module based on the backbone network module performs slice convolution calculation on the image dataset of surface defects on offshore wind turbine blades to obtain the feature information of the sliced ​​surface defects on offshore wind turbine blades. 2) An improved C2f_RepGhost module based on the backbone network module is used to reduce the dimensionality of the sliced ​​offshore wind turbine blade surface defect image dataset to obtain the dimensionality-reduced offshore wind turbine blade surface defect feature information. 3) Based on the backbone network module, the SPPF module performs spatial pyramid pooling and channel compression operations on the image dataset of surface defects of offshore wind turbine blades with global features to obtain the surface defect features of offshore wind turbine blades. Step 3: Train to optimize the YOLOv8 network model; Step 4: Based on the trained weight file, input it into the program to test the detection speed and accuracy of YOLOv8.

2. The method for detecting surface defects of offshore wind turbine blades based on an improved YOLOv8 according to claim 1, characterized in that, In step 1, images of defects in offshore wind turbine blades are collected using a drone, and the defects in the dataset are labeled using annotation software to generate an annotation file containing defect categories and predicted bounding box coordinate information. The image dataset is then divided into a training set, a validation set, and a test set.

3. The method for detecting surface defects of offshore wind turbine blades based on an improved YOLOv8 according to claim 1, characterized in that, The steps for introducing CBAM into the feature fusion network are as follows: The CBAM attention module consists of two parts: a channel attention module and a spatial attention module. In the channel attention module, the input feature map is subjected to global max pooling and global average pooling operations for each channel, generating two vectors containing the number of channels. After MLP operation, the channel attention weight vector is obtained. The weights are normalized by the Sigmoid function and finally weighted and added to the original input feature map. In the spatial attention module, the input feature map is first subjected to max pooling and average pooling operations in the channel dimension, the two output feature maps are concatenated in the channel dimension, and finally a convolutional neural network is used to further generate a spatial attention map.

4. The method for detecting surface defects of offshore wind turbine blades based on the improved YOLOv8 according to claim 1, characterized in that, In step 3, the method for training the network model is as follows: during the training phase, the number of training iterations is set to 500, the batch size is 4, and the initial learning rate is 0.

01. The target detection network is trained iteratively a preset number of times until the prediction accuracy of the target detection network for the images in the validation set reaches a preset accuracy threshold.

5. The method for detecting surface defects of offshore wind turbine blades based on an improved YOLOv8 according to claim 1, characterized in that, In step 4, the recall, accuracy, and precision of the preliminary model are measured based on the test set. If the recall, accuracy, and precision all meet the preset standards, the preliminary model is determined to be the recognition model. The image of the wind turbine blade to be detected is input into the trained target detection network to obtain the defect detection results output by the trained target detection network.

Images