Wind power blade small-size defect detection method based on attention mechanism feature fusion

Abstract

The invention relates to the technical field of small target detection, in particular to a wind power blade small-size defect detection method based on attention mechanism feature fusion, and the method comprises the steps: collecting a fan blade defect image; a C2F module in a Backbone layer is improved by using a C2F-DAtention module, and interaction between local features and a global context is enhanced; a Neck layer is improved by using a CAFMFusion module, multi-layer features are fused, features of different scales are integrated, and the feature expression ability and discrimination ability are enhanced; and a loss function is used as a measurement standard for improving the YOLOv8 network. The problems that an existing method is insufficient in precision and weak in generalization ability are solved.

Description

Technical Field

[0001] The present invention relates to the technical field of small target detection, and in particular to a method for detecting small-size defects in wind turbine blades based on attention mechanism feature fusion. Background Art

[0002] Energy has always been an important driving force for human life and social operation. However, due to the limited availability of fossil fuels such as oil and natural gas and their continued exploitation and consumption, the problem of energy shortage is becoming increasingly serious. Wind energy, as a green and environmentally friendly renewable energy source, is of great significance to optimizing my country's energy structure. Among them, wind turbine blades are important components for obtaining wind energy and directly affect the overall power generation performance. Due to the complex coupling of multiple factors such as precision control, processing technology, environmental pollution, and instrument and equipment errors in the manufacturing process, there are inherent defects in the blade molding process. These defects will have a negative impact on the power generation efficiency of the entire system. Therefore, it is crucial to detect defects on the blade surface.

[0003] Traditional detection methods, such as ultrasonic testing, infrared thermal imaging, and image processing technology based on manual feature extraction, usually rely on experts to design specific feature extraction algorithms and require a lot of manual operations, which is not only time-consuming and labor-intensive, but also easily affected by human factors, resulting in insufficient detection accuracy and consistency. Compared with traditional methods, deep learning not only improves detection efficiency and significantly reduces manual intervention, but also has a greater improvement in accuracy, and can more effectively identify defects with complex shapes.

[0004] The publication number is CN118014987A, and Bifpn is used to replace the original PANet; the SE attention mechanism is added to the feature extraction network layer. Although the SE attention mechanism can adjust the attention weight of each channel, it performs global average pooling and compresses the model parameters, resulting in a decrease in the accuracy of small target detection.

[0005] The publication number is CN118644680A. The CBS module in the YoloV5 model is replaced by the Haar wavelet attention module. The Haar wavelet is only for crack detection and has weak target generalization ability. Summary of the invention

[0006] Aiming at the deficiencies of the existing methods, the present invention solves the problems of insufficient precision and weak generalization ability of the existing methods.

[0007] The technical solution adopted by the present invention is: a method for detecting small-size defects of wind turbine blades based on attention mechanism feature fusion comprises the following steps:

[0008] Step 1: Collect defect images of fan blades;

[0009] As a preferred embodiment of the present invention, the defect categories include: paint peeling, sand holes, cracks, and skin breakage.

[0010] Step 2: Use the C2F-DAttention module to improve the C2F module in the Backbone layer to enhance the interaction between local features and global context; use the CAFMFusion module to improve the Neck layer, fuse multi-layer features, integrate features of different scales, and enhance feature expression and discrimination;

[0011] As a preferred embodiment of the present invention, the C2F-Dattention module replaces the Bottleneck layer in the C2F module with the Battleneck_Dattention layer, where Battleneck_Dattention = Dattention (Cov 1×1 (Cov 1×1 (input)), Dattention is deformable attention.

[0012] As a preferred embodiment of the present invention, the CAFMFusion module fuses the output features of the fifth C2F module of the Backbone layer with the output features of the lower sixteen C2F modules of the Neck layer.

[0013] As a preferred embodiment of the present invention, the construction of the CAFMFusion module includes:

[0014] First, use 1×1 convolution to adjust the channel of low-dimensional features, merge the adjusted low-dimensional feature layer with the high-dimensional feature layer to get initial;

[0015] In the local structure, 1×1 convolution is first used to adjust the channel size, followed by a channel shuffle operation, and then 3×3×3 convolution is used to extract features to obtain the local feature L1;

[0016] In the global structure, the query matrix Q, key value matrix K and eigenvalue matrix V are first generated through 1×1 convolution and 3×3×3 convolution to obtain three tensors of shape H×W×C; then, matrix multiplication is performed on Q and K and the weight matrix is ​​obtained through the Softmax operation, which is dot-producted with the V matrix and finally added to the Initial feature to obtain the global feature L2;

[0017] The global feature Pattn1 is concatenated with the initial feature Initial in the channel dimension, and the fused feature is extracted using 7×7 convolution, and then the weighted coefficient Pattn2 is obtained through Sigmoid. The initial feature Initial and the weighted feature are output as feature Out through a 3×3 convolution operation.

[0018] Step 3: Use the loss function as a metric to improve the YOLOv8 network.

[0019] As a preferred implementation of the present invention, the loss function adopts NWDLoss.

[0020] As a preferred embodiment of the present invention, the improved YOLOv8 network is evaluated using precision, recall, IOU-0.5 and IOU-0.5-9.95.

[0021] As a preferred embodiment of the present invention, a small-size defect detection system for wind turbine blades based on attention mechanism feature fusion includes: a memory for storing instructions executable by a processor; and a processor for executing instructions to implement a small-size defect detection method for wind turbine blades based on attention mechanism feature fusion.

[0022] As a preferred embodiment of the present invention, a computer-readable medium stores a computer program code, and when the computer program code is executed by a processor, a method for detecting small-size defects of wind turbine blades based on feature fusion of an attention mechanism is implemented.

[0023] Beneficial effects of the present invention:

[0024] 1. Construct C2F-DAttention to strengthen the interaction between local features and global context, accurately capture local features while maintaining the overall vision;

[0025] 2. Construct the CAFMFusion module to fuse low-level and high-level features to enhance the expressiveness and discrimination of features; introduce the attention mechanism to dynamically adjust the weight distribution between features, so that the detector is more sensitive to the key features of small targets, further improving the accuracy and stability of small target detection;

[0026] 3. Use NWDLoss loss function to improve detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 It is the improved YOLOV8 network model in the present invention

[0028] Figure 2 It is the structure diagram of C2F_Dattention and BottleneckDattention modules in the present invention;

[0029] Figure 3 This is the structural diagram of the CAFMFusion module in the present invention;

[0030] Figure 4 This is a defect category diagram of the fan blades in the present invention. DETAILED DESCRIPTION

[0031] The present invention is further described below in conjunction with the accompanying drawings and embodiments. This figure is a simplified schematic diagram, which only illustrates the basic structure of the present invention in a schematic manner, and therefore it only shows the components related to the present invention.

[0032] like Figure 1 As shown in the figure, the small-size defect detection method for wind turbine blades based on attention mechanism feature fusion includes the following steps:

[0033] Step 1: Collect defect images of fan blades;

[0034] DJI drones are used to take on-site photos of wind turbine blades in service to obtain high-definition images. High-definition images are taken at all angles and positions of wind turbine blades to ensure that the resolution of the captured images reaches 4224×2376 or above to capture subtle defects on the blade surface.

[0035] Defect image screening: Screen out images containing defects from the captured high-definition images to ensure that the dataset contains various types of defect information for subsequent processing and analysis. Significantly different from other datasets, this dataset does not significantly crop the images to reduce the difficulty of detection. The image size used is 4224×2376, making it more in line with reality.

[0036] like Figure 4 As shown, the defect categories include: paint peeling, sand holes, cracks, and skin breakage; among them, sand holes, cracks, and skin breakage are all small target defects.

[0037] Defect labeling: labelimg software is used to label the screened defect images; the defect areas in each image are accurately labeled to provide high-quality labeling data for model training;

[0038] Dataset construction: Integrate the annotated image data to construct a high-definition image dataset containing various types of defects; this dataset maintains the high resolution of the original image and retains more detail information, which is particularly suitable for the detection and annotation of small target defects.

[0039] The images obtained are based on real situations. In actual situations, drones cannot get close to wind turbine blades, so the images are not significantly cropped to reduce the difficulty of detection. Most of the defects are small target defects, which are more in line with reality.

[0040] Step 2: Use the C2F-DAttention module to improve the C2F module in the Backbone layer of the YOLOV8 network; use CAFMFusion to improve the Neck layer of the YOLOV8 network;

[0041] like Figure 2It is a C2F-DAttention structure, which combines the deformable attention module (DeformableAttention, DAttention) with C2F (Coarse-to-Fine). By adjusting the C2F module, it aims to help the model capture contextual information across spatial scales and promote the effective fusion of feature maps at different levels, thereby improving the accuracy of target detection.

[0042] The design focus of C2F-D Attention is the Battleneck_Dattention = Dattention (Cov 1×1 (Cov 1×1 (input)) interaction, this feature helps the detector to accurately capture local features while maintaining the overall field of view; that is, a deformable attention mechanism is introduced on the basis of the C2F structure, so as to more effectively aggregate multi-scale contextual information and enhance the flexibility and expressiveness of feature extraction; in addition, the C2F-DAttention structure reduces the model's excessive dependence on training data, thereby enhancing the generalization ability of the model.

[0043] like Figure 3 As shown in the figure, the structure diagram of the CAFMFusion module is inserted before the small target detection head. The core design of the CAFMFusion module lies in the efficient fusion of multi-layer features, which can effectively integrate features from different scales to enhance the expressiveness and discrimination of features. By introducing the attention mechanism, the weight distribution between features can be dynamically adjusted, making the detector more sensitive to the key features of small targets, further improving the accuracy and stability of small target detection.

[0044] The CAFMMFusion module takes advantage of the convolutional neural network (CNN) and Transformer to enhance the focus on key features in global and local features. Traditional CNN is limited by its local receptive field and is insufficient in capturing global information, while Transformer has a strong global vision. The combination of the two can complement each other and improve the effect of feature extraction.

[0045] The CAFMFusion module first uses 1×1 convolution to adjust the channels of low-dimensional features, and then merges the adjusted low-dimensional feature layer with the high-dimensional feature layer to obtain initial; the low-dimensional feature refers to the output feature of the fifth C2F module of the Backbone layer, and the high-dimensional feature layer refers to the output feature of the lower sixteen C2F modules of the Neck layer.

[0046] Next, the features of Initial are extracted in the local and global structures respectively; in the local structure, the channel size is first adjusted using 1×1 convolution, followed by the channel shuffle (CS) operation, and finally the features are extracted using 3×3×3 convolution to obtain the local feature L1; in the global structure, the query matrix Q, key value matrix K and eigenvalue matrix V are first generated by 1×1 convolution and 3×3×3 convolution to obtain three tensors of shape H×W×C; then, matrix multiplication is performed on Q and K and the weight matrix is ​​obtained through the Softmax operation, which is dot-producted with the V matrix and finally added to the Initial feature to obtain the global feature L2. The formula is as follows:

[0047] initial=Cov 1×1 (X)+Y

[0048] L1=Cov 3×3×3 (CS(Cov 1×1 (initial)));

[0049] L2=Cov 3×3 (Softmax(Q@K)@V)+initial;

[0050] In order to achieve feature fusion, the global feature Pattn1 is concatenated with the initial feature Initial in the channel dimension, and then the fused feature is extracted using a 7×7 convolution. The weighted coefficient Pattn2 is obtained through the activation function Sigmoid. Pattn2 is used to guide the CAFMFusion module to focus on the important areas in each channel, thereby enhancing the model's ability to pay attention to details. Finally, the initial feature Initial and the weighted feature are further extracted and optimized through a 3×3 convolution operation to generate a refined feature output Out. This ensures that the model can effectively capture and utilize global and local features, significantly improving attention to details. The formula is as follows:

[0051] pattn1=L1+L2

[0052] pattn2=sigmoid(Cov 7×7 (CT(pattn1,initial)))

[0053] Out=Cov 3×3 (initial+patten2*X+(1-pattn2)*Y)

[0054] Among them, CT() is Concat concatenation, X is the low-dimensional feature layer, Y is the high-dimensional feature layer, L1 is the merged local feature, L2 is the merged global feature, pattn1 is the global feature after L1 and L2 are merged, and pattn2 is the confidence used to guide the module to pay attention to the changes in the two regions.

[0055] Step 3: Use the NWDLoss loss function as a metric to improve the YOLOv8 network. Since most defective targets account for a very small proportion of the image, they belong to the category of small targets, and the bounding box loss function of the YOLOv8 network is extremely sensitive to the position offset of small targets. NWD (Normalized Wasserstein Distance) is a new metric for calculating bounding box similarity. It replaces the traditional IOU method by modeling the bounding box as a Gaussian distribution and using the Wasserstein distance to measure the similarity between the two distributions. In comparison, NWD is not affected by the target scale and is more stable when dealing with small targets. Assume that the target box is T = (cx t ,cy t ,w t ,h t ), the predicted box is P = (cx p ,cy p ,w p ,h p ), define N t and N p is the Gaussian distribution of the two boxes, W2 represents the second-order Wasserstein distance between them, and the formula is:

[0056]

[0057] Among them, cx t Indicates the x coordinate of the center coordinate of the t box, cy t represents the y coordinate of the center of the t box, w t Indicates the width of the t box, h t Indicates the height of the t box.

[0058] It calculates the distance metric and cannot be used for similarity calculation. The normalization method can be used to obtain a new normalized Wasserstein distance. The formula is:

[0059]

[0060] Wherein, NWD() represents the normalized Wasserstein distance, and C is a constant, and in the experiment, C=12.8.

[0061] The final loss function is:

[0062] L NWD =1-NWD(N t ,N P )

[0063] The model training parameters of this embodiment are the hyperparameters of the YOLOV8 model. The k-means clustering method is used to adjust the reg_max parameter in the target DFL from the original 16 to 12, the input image size is adjusted to 1280, the batch size is 24, the number of training rounds is 400, and the other parameters keep the original model default settings.

[0064] Model hyperparameter settings:

[0065] The model input_size is modified to adjust the original 640×640 size to a more suitable 1280×1280 size.

[0066] The specific training process is to divide the data set into a training set and a test set in a ratio of 8:2; the training set is used to train the network model, and the test set is used to test and output the model's accuracy indicators. After a large number of rounds of training and testing, the weight-removed file is saved every 10 epochs of training, and the weights of the last round and the optimal weights are saved; then the optimal weights are tested to detect their prediction accuracy; the specific results are as follows:

[0067] Table 1 shows the different ways to modify the model.

[0068]

[0069]

[0070] Table 2 shows the index results of different models

[0071] Precision P Recall R IOU-0.5 IOU - 0.5-9.95 YOLOV8 0.626 0.535 0.595 0.364 Model2 0.602 0.534 0.6 0.366 Model 3 0.624 0.563 0.612 0.363 Model 4 0.622 0.564 0.621 0.363 Model5 0.665 0.576 0.646 0.379 The present invention 0.682 0.633 0.665 0.386

[0072] From the experimental results, it can be seen that the improved YOLOv8 model has significantly improved all indicators in the wind turbine blade defect detection task, among which the IOU-0.5 accuracy has increased by 7%, the recall rate has increased by 10%, and the precision rate has increased by 5% compared with the original model. These results show that the optimized model has stronger detection capabilities and higher accuracy than the original model in the small target defect detection task of wind turbine blades.

[0073] In summary, after modifying the original model, the proposed method module using the attention mechanism for feature fusion can efficiently detect various types of defects in wind turbine blades. Compared with the original YOLOv8 model, the accuracy, recall rate and various indicators have been significantly improved. It shows better performance in the task of wind turbine blade defect detection, successfully solves the problems of missed detection and false detection in traditional manual detection, reduces damage to the surface structure of the blade, improves the accuracy of detection and production efficiency, thereby reducing labor costs and better realizing the automation and intelligent detection of wind turbine blade defects. This method effectively improves the performance in the field of small target defect detection of wind turbine blades, improves detection accuracy, and shows higher practical application value.

[0074] Based on the above ideal embodiments of the present invention, the relevant staff can make various changes and modifications without departing from the technical concept of the present invention through the above description. The technical scope of the present invention is not limited to the contents of the specification, and its technical scope must be determined according to the scope of the claims.

Claims

1. A wind turbine blade small-size defect detection method based on attention mechanism feature fusion, characterized in that: The following steps are involved: Step 1: Collect defect images of fan blades; Step 2: Use the C2F-DAttention module to improve the C2F module in the Backbone layer to enhance the interaction between local features and global context; The CAFMFusion module is used to improve the Neck layer, fuse multi-layer features, integrate features of different scales, and enhance feature expression and discrimination capabilities; Step 3: Use the loss function as a metric to improve the YOLOv8 network.

2. The method for detecting small-size defects of wind turbine blades based on attention mechanism feature fusion according to claim 1 is characterized in that: The C2F-Dattention module replaces the Bottleneck layer in the C2F module with the Bottleneck_Dattention layer, where Bottleneck_Dattention = Dattention (Cov 1×1 (Cov 1×1 (input)), Dattention is deformable attention.

3. The method for detecting small-size defects of wind turbine blades based on attention mechanism feature fusion according to claim 1 is characterized in that: The CAFMFusion module fuses the output features of the fifth C2F module of the Backbone layer with the output features of the lower sixteen C2F modules of the Neck layer.

4. The method for detecting small-size defects of wind turbine blades based on attention mechanism feature fusion according to claim 3 is characterized in that: The CAFMFusion module build includes: First, use 1×1 convolution to adjust the channel of low-dimensional features, merge the adjusted low-dimensional feature layer with the high-dimensional feature layer to get initial; In the local structure, 1×1 convolution is first used to adjust the channel size, followed by a channel shuffle operation, and then 3×3×3 convolution is used to extract features to obtain the local feature L1; In the global structure, the query matrix Q, key value matrix K and eigenvalue matrix V are first generated through 1×1 convolution and 3×3×3 convolution to obtain three tensors of shape H×W×C; then, matrix multiplication is performed on Q and K and the weight matrix is ​​obtained through the Softmax operation, which is dot-producted with the V matrix and finally added to the Initial feature to obtain the global feature L2; The global feature Pattn1 is concatenated with the initial feature Initial in the channel dimension, and the fused feature is extracted using 7×7 convolution, and then the weighted coefficient Pattn2 is obtained through Sigmoid. The initial feature Initial and the weighted feature are output as feature Out through a 3×3 convolution operation.

5. The method for detecting small-size defects of wind turbine blades based on attention mechanism feature fusion according to claim 1 is characterized in that: The loss function uses NWDLoss.

6. The method for detecting small-size defects of wind turbine blades based on attention mechanism feature fusion according to claim 1 is characterized in that: The improved YOLOv8 network is evaluated using precision, recall, IOU-0.5 and IOU-0.5-9.

95.

7. The method for detecting small-size defects of wind turbine blades based on attention mechanism feature fusion according to claim 1 is characterized in that: Defect categories include: paint peeling, sand holes, cracks, and skin breakage.

8. A wind turbine blade small-size defect detection system based on attention mechanism feature fusion, characterized in that: include: a memory for storing instructions executable by a processor; A processor, used to execute instructions to implement the method for detecting small-size defects in wind turbine blades based on attention mechanism feature fusion as described in any one of claims 1 to 7.

9. A computer readable medium storing a computer program code, characterized in that: When the computer program code is executed by a processor, the method for detecting small-size defects in wind turbine blades based on attention mechanism feature fusion as described in any one of claims 1 to 7 is implemented.

Images