Improved YOLOV8-based offshore wind turbine blade surface defect detection method

Abstract

The invention belongs to the technical field of computer vision, and particularly provides an improved YOLOV8-based offshore wind turbine blade surface defect detection method, which comprises the following steps: constructing an offshore wind turbine blade defect data set, marking defects on an image, and constructing a training set, a verification set and a test set based on the data set; a YOLOv8 baseline network model is improved, a C2f-RepGhost lightweight module is introduced to replace a C2f module in a trunk feature extraction network, a BiFPN structure is used to improve the C2f module to achieve the purpose of feature fusion, an attention module CBAM is introduced, and a final defect detection model is constructed; training is carried out to optimize a network model of the YOLOv8; and inputting a program based on the trained weight file, and testing the detection speed and precision of the YOLOv8. The improved YOLOv8 algorithm can effectively improve the precision and speed of identifying the surface defects of the offshore wind turbine blade, reduces the parameter quantity of the model, reduces the calculation cost, has better performance compared with other mainstream algorithms, and can be applied to many places.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision technology, and in particular, relates to a method for detecting surface defects of offshore wind turbine blades based on improved YOLOV8. Background Art

[0002] Wind power generation is the main form of wind energy utilization and one of the most commercially promising power generation methods. At present, domestic wind power construction is gradually developing from inland to offshore. However, affected by the operating environment of offshore wind farms, offshore wind power equipment is susceptible to environmental influences such as hot and humid air, ultraviolet rays, high salt fog and sea breeze corrosion. Wind turbine blades are an important part of wind turbines and are more prone to defects during operation, resulting in reduced power generation or even equipment damage. In order to reduce maintenance time and cost, surface defects of offshore wind turbine blades should be detected and warned immediately.

[0003] Traditional manual detection methods are inefficient. Currently, the mainstream method is to use drones to collect images and computer vision to detect blade defects. Deep convolutional neural networks have shown strong capabilities in learning high-level features of images. YOLOv8 is currently a widely used target detection model, which is lightweight and has good performance. However, for wind turbine blade failures, the blade defects are relatively small in size, and YOLOv8 has certain difficulties in feature extraction and processing. The accuracy of defect detection and classification needs to be improved. How to improve the network's detection ability for small targets and achieve a balance between detection speed and accuracy is an urgent problem to be solved. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a surface defect detection method for offshore wind turbine blades based on improved YOLOV8. Aiming at the current problems of low accuracy and high model complexity of wind turbine blade surface defect detection, an efficient and accurate surface defect detection solution is provided.

[0005] In order to solve the above technical problems, the technical solution adopted by the present invention is: a method for detecting surface defects of offshore wind turbine blades based on improved YOLOV8, comprising the following steps: Step 1: Construct an offshore wind turbine blade defect dataset; Step 2: Build a defect detection model: Improve the YOLOv8 baseline network model, introduce the C2f-RepGhost lightweight module to replace the C2f module in the backbone feature extraction network, use the BiFPN structure for feature fusion, introduce the attention module CBAM, and use the VoV-GSCSP lightweight module to replace the CSP module to build the final defect detection model based on the improved YOLOv8; Step 3: Train to optimize the YOLOv8 network model; Step 4: Based on the trained weight file, input the program to test the detection speed and accuracy of YOLOv8.

[0006] In the preferred scheme, in step 1, defect images of offshore wind turbine blades are collected using a drone, and the defects in the data set are annotated using annotation software to generate an annotation file containing defect categories and prediction box coordinate information, and the image data set is divided into a training set, a validation set, and a test set.

[0007] In a preferred solution, in step 2, the defect detection model based on the improved YOLOV8 includes a backbone network module, a neck module and a prediction module, the output end of the backbone network module is connected to the input end of the neck module, and the output end of the neck module is connected to the input end of the prediction module, wherein: The backbone network module includes a CBS module, an improved C2f_RepGhost module and an SPPF module; The neck module includes an improved BiFPN layer; The prediction module includes a convolutional layer.

[0008] In a preferred solution, in step 2, the backbone network module performs feature extraction on the offshore wind turbine blade surface defect image dataset, comprising the following steps: 1) Based on the CBS module of the backbone network module, the offshore wind turbine blade surface defect image dataset is sliced ​​and convolved to obtain the feature information of the sliced ​​offshore wind turbine blade surface defect image; 2) Based on the improved C2f_RepGhost module of the backbone network module, the sliced ​​offshore wind turbine blade surface defect image dataset is subjected to dimensionality reduction processing to obtain the surface defect feature information of the offshore wind turbine blade after dimensionality reduction; 3) Based on the SPPF module of the backbone network module, spatial pyramid pooling and channel compression operations are performed on the offshore wind turbine blade surface defect image dataset with global features to obtain the surface defect features of the offshore wind turbine blades.

[0009] In a preferred solution, the improved C2f_RepGhost module replaces the repeated Bottleneck module in the C2f module with RepGhostBottleneck based on GhostConv.

[0010] In a preferred solution, the neck module includes: an improved BiFPN feature fusion network based on a feature fusion module, a CBAM attention mechanism, a CBS module, a C2f module and a splicing operation.

[0011] In a preferred solution, the improved BiFPN feature fusion network in the neck module adds an edge linking the upper layer output and the lower layer input to the basic BiFPN network; The feature fusion network includes a first layer input, a second layer input, a third layer input, a first layer fusion output, a second layer intermediate output, a second layer fusion output, and a third layer fusion output; In the feature fusion module, the first layer fusion output part receives the first layer input and the second layer intermediate output and fuses the two parts of features to output, the second layer intermediate output part receives the second layer input and the third layer input and fuses the two parts of features to output, the second layer fusion output part receives the second layer input and the second layer intermediate output and fuses the two parts of features to output, the third layer fusion output part receives the third layer input and the second layer fusion output and fuses the two parts of features to output, and finally an edge connecting the second layer input and the third layer fusion output is added.

[0012] In the preferred solution, the introduction of CBAM in the feature fusion network is as follows: The CBAM attention module consists of two parts: the channel attention module and the spatial attention module. The input feature map performs global maximum pooling and global average pooling operations on each channel in the channel attention module to generate two vectors containing the number of channels. The weight vector of the channel attention is obtained through the MLP operation, and the weight is normalized by the Sigmoid function, and finally weighted to the original input feature map; In the spatial attention module, the input feature map first performs maximum pooling and average pooling operations in the channel dimension, the two output feature maps are concatenated in the channel dimension, and finally a convolutional neural network is used to further generate a spatial attention map.

[0013] In the preferred scheme, in step 3, the method for training the network model is: the number of training iterations in the training phase experiment is set to 500, the batch size is 4, the initial learning rate is 0.01, and the target detection network is iterated for a preset number of times until the prediction accuracy of the target detection network's prediction results for the images in the validation set reaches a preset accuracy threshold.

[0014] In the preferred scheme, in step 4, the recall rate, accuracy rate and precision rate of the preliminary model are measured based on the test set, and when the recall rate, accuracy rate and precision rate all meet the preset standards, the preliminary model is determined to be a recognition model; the image of the wind turbine blade to be detected is input into the trained target detection network, and the defect detection result output by the trained target detection network is obtained.

[0015] The present invention provides a method for detecting surface defects of offshore wind turbine blades based on improved YOLOV8, which has the following beneficial effects: 1. In view of the current problems of low accuracy and high model complexity in wind turbine blade surface defect detection, the present invention provides an efficient and accurate surface defect detection solution.

[0016] 2. In step 1, a self-made image dataset is created through manual annotation methods, and is divided into training sets, validation sets, and test sets in proportion to reduce the risk of model overfitting.

[0017] 3. In step 2, a new C2f-RepGhost module is designed in the YOLOv8n backbone network to replace the C2f module, which improves the detection speed of the model and reduces the number of model parameters; in the neck module, the CBAM attention module is introduced to make the model pay more attention to small target information, and the improved BiFPN structure is used for feature fusion, which can effectively improve the recognition and positioning accuracy of small target defects, and has better performance than other mainstream algorithms.

[0018] 4. In steps 3 and 4, when the model training is completed and enters the deployment stage, the model can not only effectively improve the accuracy of defect recognition, but also reduce the number of model parameters and reduce computing costs, which is of great significance to edge terminal devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The present invention will be further described below in conjunction with the accompanying drawings and embodiments: Figure 1 is a flow chart of the present invention; Figure 2 This is the structural diagram of the original GhostBottleneck and RepGhostBottleneck during training and reasoning; Figure 3 This is the schematic diagram of C2f_RepGhost; Figure 4 This is the improved BiFPN structure diagram; Figure 5 This is the schematic diagram of the CBAM attention module; Figure 6 This is the overall network structure diagram. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0021] A surface defect detection method for offshore wind turbine blades based on improved YOLOV8, such as Figure 1 As shown, the following steps are included: Step 1: Build an offshore wind turbine blade defect dataset: Defect images of offshore wind turbine blades were collected using drones, and the defects in the dataset were annotated using annotation software to generate an annotation file containing defect categories and prediction box coordinate information. The image dataset was divided into training set, validation set, and test set in a ratio of 8:1:1.

[0022] Step 2: Improve the YOLOv8 baseline network model, introduce the C2f-RepGhost lightweight module to replace the C2f module in the backbone feature extraction network, use the BiFPN structure for feature fusion, introduce the attention module CBAM, and use the VoV-GSCSP lightweight module to replace the CSP module to build the final defect detection model based on the improved YOLOv8.

[0023] The defect detection model based on the improved YOLOV8 includes a backbone network module, a neck module and a prediction module, wherein the output end of the backbone network module is connected to the input end of the neck module, and the output end of the neck module is connected to the input end of the prediction module.

[0024] in: The backbone network module includes a CBS module, an improved C2f module and an SPPF module.

[0025] The neck module includes a modified BiFPN layer.

[0026] The prediction module includes a convolutional layer.

[0027] The steps of extracting features from the offshore wind turbine blade surface defect image dataset specifically include: Based on the CBS module of the backbone network module, the offshore wind turbine blade surface defect image dataset is sliced ​​and convolved to obtain the feature information of the offshore wind turbine blade surface defect image after slicing. Based on the improved C2f_RepGhost module of the backbone network module, the dimensionality reduction processing is performed on the sliced ​​offshore wind turbine blade surface defect image dataset to obtain the surface defect feature information of the offshore wind turbine blade after dimensionality reduction; Based on the SPPF module of the backbone network module, spatial pyramid pooling and channel compression operations are performed on the offshore wind turbine blade surface defect image dataset with global characteristics to obtain the surface defect features of the offshore wind turbine blades.

[0028] Among them, the design idea of ​​the improved C2f_RepGhost module based on the backbone network module is to replace the repeated Bottleneck module in C2f with RepGhostBottleneck based on GhostConv, such as Figure 2 and Figure 3 shown.

[0029] The neck module includes: an improved BiFPN feature fusion network based on the feature fusion module, a CBAM attention mechanism, a CBS module, a C2f module and a splicing operation.

[0030] The improved BiFPN feature fusion network adds an edge connecting the upper layer output and the lower layer input to the basic BiFPN network; The feature fusion module includes a first layer input, a second layer input, a third layer input, a first layer fusion output, a second layer intermediate output, a second layer fusion output, and a third layer fusion output.

[0031] In the feature fusion module, the first-layer fusion output part receives the first-layer input and the second-layer intermediate output and fuses the two parts of features to output, the second-layer intermediate output part receives the second-layer input and the third-layer input and fuses the two parts of features to output, the second-layer fusion output part receives the second-layer input and the second-layer intermediate output and fuses the two parts of features to output, the third-layer fusion output part receives the third-layer input and the second-layer fusion output and fuses the two parts of features to output, and finally adds an edge connecting the second-layer input and the third-layer fusion output, such as Figure 4 As shown, taking the second layer as an example, the principle is as follows: (1); (2); In the formula, , (i=1, 2, 3…) is the learnable weight of each node participating in the operation, , , Represent the first to third layer inputs respectively, represents the intermediate output of the second layer, , Represent the first and second layer fusion outputs respectively. Resize() is a resizing function that enables the fusion of features of different scales. Conv is a convolution operation, and ε=0.0001 is used to ensure the stability of the value.

[0032] The introduction of CBAM in the feature fusion network of the neck module is as follows: The input feature map F performs global maximum pooling and global average pooling operations on each feature map in the channel attention mechanism to generate two vectors containing the number of channels. The weight vector of the channel attention is obtained through the MLP full connection operation, and the weight is normalized by the Sigmoid function. Finally, it is weighted to the original input feature map. The principle is as follows: (3); in, represents the weight vector after the channel attention mechanism, is the activation function, W 1. W 0 represents the weight parameter in the full connection process; AvgPool(F) and represents the average pooling of F, MaxPool(F) and Represents the maximum pooling of F; In the spatial attention module, the input feature map First, perform maximum pooling and average pooling operations in the channel dimension, concatenate the two output feature maps in the channel dimension, and finally use the convolutional neural network to further generate the spatial attention map. The principle is as follows: (4); in, M s (F) represents the weight vector after the channel attention mechanism, Represents a 7×7 convolution.

[0033] The overall structure of CBAM is as follows Figure 5 shown.

[0034] The overall structure of the improved YOLOv8 model is as follows Figure 6 shown.

[0035] Step 3: Train the network model. The steps are as follows: In the training phase, the number of training iterations of the experiment is set to 500, the batch size is 4, the initial learning rate is 0.01, and the target detection network is iteratively trained for a preset number of times until the prediction accuracy of the target detection network's prediction results for the images in the validation set reaches a preset accuracy threshold.

[0036] Step 4: Save the trained weight file to the improved YOLOv8 network model, and then test the improved YOLOv8 detection performance based on the trained weight file and output the test results. Measure the recall rate, accuracy rate and precision rate of the preliminary model based on the test set. When the recall rate, accuracy rate and precision rate all meet the preset standards, determine that the preliminary model is a recognition model. Input the image of the wind turbine blade to be detected into the trained target detection network, and obtain the defect detection result output by the trained target detection network; In order to evaluate the ability of different combinations of modules to optimize the model, an ablation experiment was designed, and the data is shown in Table 1. In the process of gradually introducing improved modules, the overall effect of the network model is also constantly improving. Comparing Experiment 1 and Experiment 2, it can be seen that after the introduction of the CBAM module, the Precision and mAP values ​​of the model increased by 2.4% and 1.5% respectively, but the increase in the number of parameters caused a slight decrease in the detection speed; Comparing Experiment 1 and Experiment 3, it can be seen that after the introduction of the improved BiFPN, the detection accuracy has been improved. Although the number of parameters has not changed much, the detection speed has been reduced by 1.8FPS; Comparing Experiment 1 and Experiment 4, it can be found that after adding the lightweight module RepGhost, the detection speed has been greatly improved, increasing by 3.4FPS, but the detection accuracy has decreased slightly; When adding CBAM and BiFPN at the same time, the model can not only retain small target information in the feature extraction stage, but also enhance the ability to fuse low semantic information in the feature fusion stage, retain more small target information, and further improve the detection accuracy of the model, but due to the increase in the computational cost of the model, the detection speed has decreased; After the introduction of the improved module at the same time, the detection speed and accuracy of the model reach the best comprehensive.

[0037]

[0038] To further verify the performance of the algorithm, the performance of the improved YOLOv8 and other mainstream target detection algorithms such as Faster R-CNN and SSD are compared under the same configuration and data set, as shown in Table 2. Although the detection speed is not optimal, it has the best effect in detection accuracy, achieving the highest accuracy and mAP value, with lower cost, higher detection efficiency and simpler deployment.

[0039]

[0040] In summary, the present invention improves the problem of low efficiency and poor detection accuracy of surface defects in offshore wind turbine blades. On a self-made data set, the model of the present invention has achieved significant improvements in various indicators, verifying the effectiveness and practicality of the method. This work provides an efficient and accurate surface defect detection solution for offshore wind turbine blade inspections, which has certain application prospects and practical significance.

[0041] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for detecting surface defects of offshore wind turbine blades based on improved YOLOV8, characterized in that: The following steps are involved: Step 1: Construct an offshore wind turbine blade defect dataset; Step 2: Build a defect detection model: Improve the YOLOv8 baseline network model, introduce the C2f-RepGhost lightweight module to replace the C2f module in the backbone feature extraction network, use the BiFPN structure for feature fusion, introduce the attention module CBAM, and use the VoV-GSCSP lightweight module to replace the CSP module to build the final defect detection model based on the improved YOLOv8; Step 3: Train to optimize the YOLOv8 network model; Step 4: Based on the trained weight file, input the program to test the detection speed and accuracy of YOLOv8.

2. According to claim 1, a method for detecting surface defects of offshore wind turbine blades based on improved YOLOV8 is characterized in that: In step 1, defect images of offshore wind turbine blades are collected using a drone, and the defects in the data set are annotated using annotation software to generate an annotation file containing defect categories and prediction box coordinate information, and the image data set is divided into a training set, a validation set, and a test set.

3. The method for detecting surface defects of offshore wind turbine blades based on improved YOLOV8 according to claim 1, characterized in that: In the step 2, the defect detection model based on the improved YOLOV8 includes a backbone network module, a neck module and a prediction module, the output end of the backbone network module is connected to the input end of the neck module, and the output end of the neck module is connected to the input end of the prediction module, wherein: The backbone network module includes a CBS module, an improved C2f_RepGhost module and an SPPF module; The neck module includes an improved BiFPN layer; The prediction module includes a convolutional layer.

4. The method for detecting surface defects of offshore wind turbine blades based on improved YOLOV8 according to claim 3 is characterized in that: In step 2, the backbone network module extracts features from the offshore wind turbine blade surface defect image dataset, including the following steps: 1) Based on the CBS module of the backbone network module, the offshore wind turbine blade surface defect image dataset is sliced ​​and convolved to obtain the feature information of the sliced ​​offshore wind turbine blade surface defect image; 2) Based on the improved C2f_RepGhost module of the backbone network module, the sliced ​​offshore wind turbine blade surface defect image dataset is subjected to dimensionality reduction processing to obtain the surface defect feature information of the offshore wind turbine blade after dimensionality reduction; 3) Based on the SPPF module of the backbone network module, spatial pyramid pooling and channel compression operations are performed on the offshore wind turbine blade surface defect image dataset with global features to obtain the surface defect features of the offshore wind turbine blades.

5. The method for detecting surface defects of offshore wind turbine blades based on improved YOLOV8 according to claim 3 is characterized in that: The improved C2f_RepGhost module replaces the repeated Bottleneck module in the C2f module with the RepGhostBottleneck based on GhostConv.

6. The method for detecting surface defects of offshore wind turbine blades based on improved YOLOV8 according to claim 3 is characterized in that: The neck module includes: an improved BiFPN feature fusion network based on a feature fusion module, a CBAM attention mechanism, a CBS module, a C2f module and a splicing operation.

7. The method for detecting surface defects of offshore wind turbine blades based on improved YOLOV8 according to claim 6, characterized in that: The improved BiFPN feature fusion network in the neck module adds an edge linking the upper layer output and the lower layer input to the basic BiFPN network; The feature fusion network includes a first layer input, a second layer input, a third layer input, a first layer fusion output, a second layer intermediate output, a second layer fusion output, and a third layer fusion output; In the feature fusion module, the first layer fusion output part receives the first layer input and the second layer intermediate output and fuses the two parts of features to output, the second layer intermediate output part receives the second layer input and the third layer input and fuses the two parts of features to output, the second layer fusion output part receives the second layer input and the second layer intermediate output and fuses the two parts of features to output, the third layer fusion output part receives the third layer input and the second layer fusion output and fuses the two parts of features to output, and finally an edge connecting the second layer input and the third layer fusion output is added.

8. The method for detecting surface defects of offshore wind turbine blades based on improved YOLOV8 according to claim 6 is characterized in that: The introduction of CBAM in the feature fusion network is as follows: The CBAM attention module consists of two parts: the channel attention module and the spatial attention module. The input feature map performs global maximum pooling and global average pooling operations on each channel in the channel attention module to generate two vectors containing the number of channels. The weight vector of the channel attention is obtained through the MLP operation, and the weight is normalized by the Sigmoid function, and finally weighted to the original input feature map; In the spatial attention module, the input feature map first performs maximum pooling and average pooling operations in the channel dimension, the two output feature maps are concatenated in the channel dimension, and finally a convolutional neural network is used to further generate a spatial attention map.

9. The method for detecting surface defects of offshore wind turbine blades based on improved YOLOV8 according to claim 1, characterized in that: In step 3, the method for training the network model is: in the training phase, the number of training iterations is set to 500, the batch size is 4, the initial learning rate is 0.01, and the target detection network is iterated for a preset number of times until the prediction accuracy of the target detection network's prediction results for the images in the validation set reaches a preset accuracy threshold.

10. The method for detecting surface defects of offshore wind turbine blades based on improved YOLOV8 according to claim 1, characterized in that: In step 4, the recall rate, accuracy rate and precision rate of the preliminary model are measured based on the test set. When the recall rate, accuracy rate and precision rate all meet the preset standards, the preliminary model is determined to be a recognition model; the image of the wind turbine blade to be detected is input into the trained target detection network, and the defect detection result output by the trained target detection network is obtained.

Images