Wind turbine blade defect detection method based on improved SSD algorithm
Through the improved SSD algorithm, combined with feature fusion, attention mechanism and feature enhancement module, the problem of insufficient accuracy of wind turbine blade defect detection in complex environments is solved, and efficient detection of small targets is achieved.
Patent Information
- Application Number
- CN202510417440.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-03
- Publication Date
- 2025-07-11
AI Technical Summary
The existing computer vision methods are difficult to effectively extract the defect characteristics of wind turbine blades in complex environments, and the detection accuracy of small target defects is insufficient.
Using the improved SSD algorithm, the detection ability of blade defect features is enhanced by introducing feature fusion module, improved scSENet attention mechanism module and feature enhancement module.
It improves the detection accuracy of small-size defect targets in complex contexts, enhances the network's feature expression ability, and improves the detection accuracy.
Smart Images
Figure CN120298370A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of wind turbine blade detection, and specifically provides a method for detecting wind turbine blade defects based on an improved SSD algorithm. Background Technique
[0002] Wind power generation is one of the important new energy technologies in the world today. The core component of a wind turbine is the blade, whose working environment is extremely harsh. It is not only affected by the superposition of aerodynamic loads, gravitational loads, and inertial loads, but also operates continuously in a harsh natural environment for a long time. In particular, severe weather such as typhoons, thunderstorms, ice and snow, and sandstorms may damage the wind turbine blades, thus affecting the wind power generation efficiency. Therefore, in order to ensure the safe, stable, and efficient operation of wind turbines, accurate and effective condition monitoring and analysis must be carried out, and early warnings should be issued when necessary to ensure the normal operation of the facilities.
[0003] With the upgrade of UAV technology and the improvement of the quality of image data obtained, researchers have begun to use UAVs to obtain image data of wind turbine blades and detect defects in wind turbine blades through machine vision technology. However, existing computer vision methods often have difficulty in effectively extracting the defect features of wind turbine blades in complex environments, and the network model still has insufficient capabilities for detecting small target defects, and the accuracy of defect detection needs to be improved. Summary of the Invention
[0004] The purpose of the present invention is to provide a method for detecting wind turbine blade defects based on an improved SSD algorithm to solve the problems mentioned in the above background technique.
[0005] To achieve the above purpose, the present invention provides the following technical solution: A method for detecting wind turbine blade defects based on an improved SSD algorithm, including the following steps:
[0006] Step 1, collect image data of wind turbine blade defects;
[0007] Step 2, expand the image data through image processing methods to obtain an image data set to avoid the problem of overfitting in model training. The image processing methods include but are not limited to flipping, mirroring, changing brightness, and adding Gaussian noise;
[0008] Step 3, use the LabelImg tool to label the image data set, and the labeling types are dirt labels, crack labels, damage labels, and erosion labels;
[0009] Step 4, randomly divide the labeled image data set in Step 3, and divide the image data set into a training set, a validation set, and a test set according to the ratio of 7:1:2;
[0010] Step 5: Build a wind turbine blade defect detection model based on the SSD algorithm. Introduce a feature fusion module to fuse the shallow feature map with the deep feature map after upsampling processing, so as to transfer the deep semantic information to the shallow layer, enabling the shallow feature map to utilize both its own location information and the semantic information of the deep feature map. Then, introduce an improved scSENet attention mechanism module after the feature layer output by feature fusion. Through the feature enhancement module, perform multi-branch convolution operations to mine the features of the shallow network and expand the receptive field of the effective feature layer;
[0011] Step 6: Input the training set into the improved wind turbine blade defect detection model for iterative training;
[0012] Step 7: Use the trained wind turbine blade defect detection model to perform performance tests on the test set in the dataset.
[0013] Furthermore, in Step 5, add a feature fusion module to fuse the feature layers output by convolutional layers Conv4_3 and Conv7 and the feature layers output by convolutional layers Conv7 and Conv8 respectively;
[0014] The feature fusion module first uses two 1×1 channel convolutional layers to reduce the number of channels of the input feature map to 256, and then upsamples the deep feature map to the same size as the shallow feature based on deconvolution; Figure 1 Then, sum the shallow feature map and the sampled deep feature map element by element. The output fused feature map uses a 3×3 convolution to eliminate the aliasing effect of the fused features, and then uses a 1×1 channel convolution for channel dimension increase operation to keep the number of channels the same as that of the shallow feature map before fusion. Finally, output the fused feature map;
[0015] Add an improved scSENet attention mechanism module after the feature layer output by feature fusion. Based on the original scSENet attention mechanism module, add a branch of max pooling in its channel dimension;
[0016] Add a feature enhancement module after the feature layer output by the attention mechanism. The feature enhancement module uses parallel convolutions of different sizes;
[0017] The feature enhancement module uses grouped convolution operations to decompose the k×k convolution into k×1 and 1×k convolutions.
[0018] Furthermore, the feature enhancement module contains 3 convolutional branches and a residual branch. The scales in the convolutional branches are 1, 3, and 5 respectively. After the feature map is input, first use a 1×1 convolutional layer to reduce the number of channels, then perform convolutional calculations of different scales on each convolutional branch, and finally perform a concate feature fusion operation on the convolutional results and splice them with the residual branch for output.
[0019] Furthermore, in step six, a stochastic gradient descent optimizer is adopted, with a weight decay coefficient of 0.0005, a parameter momentum of 0.937, an initial learning rate of 0.01, and a final cosine annealing to 0.001. The batch_size is set to 16, the algorithm iterates 200 rounds, and the benchmark anchor box parameters are set to [30, 60, 111, 162, 213, 264, 315].
[0020] Compared with the prior art, the beneficial effects of the present invention are as follows: Firstly, a feature fusion module is introduced to enhance the semantic information of shallow features; then, an attention mechanism module is introduced, and spatial attention and channel attention are used to suppress background noise and improve the attention to blade defect features; then, the feature expression ability of the network is enhanced through a feature enhancement module, improving the detection ability of the algorithm for multi-scale defects. Therefore, the technical solution can solve the problem of low defect detection accuracy in the prior art under complex backgrounds and small-size defect targets. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] Figure 1 is a flowchart of the present invention.
[0022] Figure 2 is a network structure diagram of the improved SSD algorithm of the present invention.
[0023] Figure 3 is a schematic diagram of the feature fusion module of the present invention.
[0024] Figure 4 is a schematic diagram of the improved scSENet attention mechanism module of the present invention.
[0025] Figure 5 is a schematic diagram of the feature enhancement module of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS
[0026] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.
[0027] Embodiment:
[0028] Please refer to Figures 1-5 , the present invention provides a technical solution: a method for detecting wind turbine blade defects based on an improved Single Shot MultiBox Detector (SSD) algorithm;
[0029] As Figure 1As shown in the figure, the defect detection of wind turbine blades includes the following steps:
[0030] Step 1: Collect defect pictures of wind turbine blades taken by drones provided by the Roboflow website. There are a total of 1265 defect photos in the dataset, and the photos contain defects such as dirt, cracks, damage, and erosion.
[0031] Step 2: Augment the defect pictures of wind turbine blades collected in Step 1 to avoid the problem of overfitting in model training. To simulate the situation of drones in different weather conditions and at different shooting angles, methods such as flipping, mirroring, changing brightness, and adding Gaussian noise are used to augment the number of defect pictures of wind turbine blades. After data augmentation, the dataset is expanded to 5062 pictures, forming a picture dataset.
[0032] Step 3: Use the LabelImg tool to annotate the picture dataset. The dirt label is dirt, the crack label is crack, the damage label is damage, and the erosion label is erosion. The number of each type of label in the dataset is as follows: there are 1694 dirt labels, 1228 crack labels, 2936 damage labels, and 2782 erosion labels.
[0033] Step 4: Randomly divide the labeled blade defect picture dataset in Step 3. The picture dataset is divided into a training set, a validation set, and a test set according to the ratio of 7:1:2.
[0034] Step 5: Build a wind turbine blade defect detection model based on the SSD algorithm (abbreviated as the improved SSD model). The wind turbine blade defect detection model in this embodiment is built based on the SSD algorithm. To adapt to the application scenario of this embodiment, the original SSD algorithm is improved. The improved SSD network structure is as Figure 2 shown.
[0035] Step 5.1: Add a feature fusion module. The shallow feature layer of the original SSD model contains less semantic information of the target, making the model unable to fully learn the features of small-sized targets, resulting in a lower detection accuracy for small-sized wind turbine blade defects. To improve the detection accuracy for small-sized targets, as Figure 2 shown, a feature fusion module is added to fuse the feature layers output by convolutional layers Conv4_3 and Conv7 and the feature layers output by convolutional layers Conv7 and Conv8 respectively. The network structure of the feature fusion module is as Figure 3 shown: First, use two 1×1 channel convolutional layers to reduce the number of channels of the input feature map to 256, and then based on transposed convolution (deconvolution), upsample the deep feature map to the same size as the shallow feature Figure 1Consistent dimensions. Then, the shallow feature map and the sampled deep feature map are summed element by element. The output fused feature map uses a 3×3 convolution to eliminate the aliasing effect of the fused features, and then a 1×1 channel convolution is used for channel upsampling operation to keep the same number of channels as the shallow feature map before fusion. Finally, the fused feature map is output. After each convolution, a Relu activation function layer is adopted to improve the non-linear expression ability of the algorithm and ensure that feature maps of different scales have similar distributions and value ranges to avoid the feature divergence phenomenon caused by excessive feature differences.
[0036] Step 5.2: Add an improved scSENet attention mechanism module. To improve the model's attention to defect features in complex backgrounds, as Figure 2 shown, an attention mechanism module is added after the feature fusion module. The structure of the improved scSENet attention mechanism module is as Figure 4 shown: The improved scSENet is a combination of the Channel Squeeze and Spatial Excitation Block (sSE) module and the Spatial Squeeze and Channel Excitation Block (cSE) module.
[0037] Given a feature map U of size CxHxW as input, where H and W represent the height and width of the feature map respectively, and C represents the number of channels of the feature map. Input U into the cSE module and the sSE module. The cSE module contains two branches: average pooling and max pooling. The global average pooling branch mainly consists of 1 global average pooling layer and two fully connected layers. The input feature map first passes through the global pooling layer to obtain global features, and then passes through two fully connected layers to reduce and increase the dimensions of the features respectively, generating a feature map M1∈R Cx1x1 of size Cx1x1, where C represents the number of channels of the feature map, and the height and width of the feature map are both 1. The max pooling branch has a similar structure to the global average pooling branch, mainly consisting of 1 max pooling layer and two fully connected layers. The input feature map first passes through the max pooling layer to obtain significant features, and then passes through two fully connected layers to reduce and increase the dimensions of the features respectively, generating a feature map M2∈R Cx1x1 of size Cx1x1. The feature maps generated by the two branches are summed element by element and then passed through a sigmoid operation to obtain a feature map M C ∈R Cx1x1 Finally, M C is multiplied with the corresponding space of the original feature U to obtain a new feature map of size CxHxW. The calculation process is as follows:
[0038]
[0039] where σ is the Sigmoid operation, and represents the convolution operation, represents the element-wise multiplication.
[0040] The sSE module is a spatial attention module that compresses the feature map in channels through a 1×1 convolution kernel and performs spatial excitation, and then recalibrates the data through the Sigmoid activation function to obtain a feature map M of size 1xHxW s ∈R 1xHxW , where 1 represents the number of channels of the feature map, and H and W represent the height and width of the feature map respectively. Finally, a new feature map is obtained after multiplying with the original feature U The size of is CxHxW, and the calculation formula is as follows:
[0041]
[0042] The improved scSENet integrates and enhances the information in the spatial dimension and channel dimension by applying the cSE module and the sSE module in parallel and combining their output feature maps to achieve the effect of emphasizing key features and suppressing background noise. The expression for the whole process is:
[0043]
[0044] Step 5.3: Add a feature enhancement module. To improve the ability to extract the defect features of wind turbine blades, as Figure 2 shown, a feature enhancement module is added after the attention mechanism module. The feature enhancement module is as Figure 5 shown: Based on the concept of multi-branch convolution operations, the feature enhancement module uses convolutions of different sizes to enable the network to learn richer non-linear relationships, and combines the idea of residual networks, so as to more effectively capture feature information and expand its receptive field. To improve the computational efficiency while keeping the receptive field size unchanged, grouped convolution operations are used to decompose the k×k convolution into k×1 and 1×k convolutions to reduce the time consumption of this module. The feature enhancement module contains 3 convolution branches and a residual branch. The scales in the convolution branches are 1, 3, and 5 respectively. After the feature map is input, it first passes through a 1×1 convolutional layer to reduce the number of channels, and then performs convolutional calculations of different scales on each convolution branch. Finally, the convolution results are subjected to a concate feature fusion operation and spliced with the residual branch for output, constructing a receptive field algorithm that simulates the human visual system.
[0045] Step 6: Input the training set into the improved SSD model for iterative training.
[0046] Step 6.1, Experimental environment configuration. This experiment is carried out under the Windows 10 operating system. The CPU processor used is the 11th Gen Intel(R) Core(TM) i9-11900K@3.50GHz, and the number of CPUs is 16. The GPU is NVIDIA GeForce RTX 3060Ti, and the video memory of the graphics card is 12GB. The deep learning framework used is Pytorch, and the compilation environment of PyCharm is used.
[0047] Step 6.2, Hyperparameter settings for model training. In the experiment of the present invention, the Stochastic Gradient Descent (SGD) optimizer is adopted, the weight decay coefficient is 0.0005, and its parameter momentum is 0.937. The initial learning rate is 0.01, and the final cosine decay is 0.001. The batch_size is set to 16, and the algorithm iterates 200 rounds in total. The benchmark anchor box parameters are set to [30, 60, 111, 162, 213, 264, 315].
[0048] Step 7, Use the trained SSD blade defect detection model to perform performance testing on the test set in the dataset. After the training process is completed, a wind turbine blade defect detection model weight file will be generated, and the weight file is used to test the model performance. The Average Precision (AP), mean Average Precision (mAP), Precision, and Recall are used to measure the quality of the established neural network model. The larger the numerical value of all indicators, the better the detection performance. The calculation formulas for each indicator are as follows:
[0049]
Claims
1. A method for detecting defects in wind turbine blades based on an improved SSD algorithm, characterized in that, It includes the following steps: Step 1: Collect the defect picture data of wind turbine blades; Step 2: Augment the picture data through image processing methods to obtain a picture dataset, so as to avoid the problem of overfitting in model training; Step 3: Use the LabelImg tool to annotate the picture dataset, and the annotation types are dirt label, crack label, damage label, and erosion label; Step 4: Randomly divide the labeled picture dataset in Step 3, and divide the picture dataset into a training set, a validation set, and a test set according to the ratio of 7:1:2; Step 5: Build a wind turbine blade defect detection model based on the SSD algorithm. Introduce a feature fusion module to fuse the shallow feature map with the deep feature map after upsampling processing, so as to transfer the deep semantic information to the shallow layer, making the shallow feature map utilize both its own position information and the semantic information of the deep feature map. Then, introduce an improved scSENet attention mechanism module after the feature layer output by feature fusion. Through the feature enhancement module, perform multi-branch convolution operations to mine the shallow network features and expand the receptive field of the effective feature layer; Step 6: Input the training set into the improved wind turbine blade defect detection model for iterative training; Step 7: Use the trained wind turbine blade defect detection model to perform performance tests on the test set in the dataset.
2. The method for detecting defects of wind turbine blades based on an improved SSD algorithm according to claim 1, characterized in that: In Step 2, the image processing methods include but are not limited to flipping, mirroring, changing brightness, and adding Gaussian noise.
3. The method for detecting defects of wind turbine blades based on the improved SSD algorithm according to claim 1, characterized in that: In Step 5, add a feature fusion module to fuse the feature layers output by convolutional layers Conv4_3 and Conv7 and the feature layers output by convolutional layers Conv7 and Conv8 respectively; The feature fusion module first uses two 1×1 channel convolutional layers to reduce the number of channels of the input feature map to 256, then upsamples the deep feature map to the same size as the shallow feature map based on deconvolution, and then sums the shallow feature map and the sampled deep feature map element by element. The output fused feature map uses a 3×3 convolution to eliminate the aliasing effect of the fused features, and then performs a channel dimension upsampling operation with a 1×1 channel convolution to keep the number of channels the same as that of the shallow feature map before fusion. Finally, the fused feature map is output.
4. The method for detecting defects of wind turbine blades based on an improved SSD algorithm according to claim 1, characterized in that: In Step 5, add an improved scSENet attention mechanism module after the feature layer output by feature fusion. Based on the original scSENet attention mechanism module, add a branch of max pooling in its channel dimension.
5. A method for detecting defects in wind turbine blades based on an improved SSD algorithm according to claim 1, characterized in that: In Step 5, add a feature enhancement module after the feature layer output by the attention mechanism. The feature enhancement module uses parallel convolutions of different sizes.
6. The method for detecting defects of wind turbine blades based on an improved SSD algorithm according to claim 5, wherein: The feature enhancement module uses grouped convolution operations to decompose the k×k convolution into k×1 and 1×k convolutions.
7. The method for detecting defects of wind turbine blades based on an improved SSD algorithm according to claim 6, characterized in that: The feature enhancement module contains 3 convolutional branches and a residual branch. The scales in the convolutional branches are 1, 3, and 5 respectively. After the feature map is input, it first passes through a 1×1 convolutional layer to reduce the number of channels, then performs convolutional calculations of different scales on each convolutional branch, and finally performs a concate feature fusion operation on the convolutional results and splices them with the residual branch for output.
8. A method for detecting defects in wind turbine blades based on an improved SSD algorithm according to claim 1, characterized in that: In step six, the stochastic gradient descent optimizer is adopted, with a weight decay coefficient of 0.0005, a parameter momentum of 0.937, an initial learning rate of 0.01, and a final cosine decay to 0.
001. The batch_size is set to 16, the algorithm iterates 200 rounds, and the benchmark anchor box parameters are set to 30, 60, 111, 162, 213, 264, 315.
9. The method for detecting defects of wind turbine blades based on an improved SSD algorithm according to claim 1, wherein: In step seven, after the training process is completed, a weight file of the wind turbine blade defect detection model is generated. The performance of the model is tested using the weight file, and the average precision, overall average precision, precision, and recall are used to measure the quality of the established neural network model.