A wind turbine blade defect detection method based on improved YOLOv4

Abstract

The present invention discloses a fan blade defect detection method based on an improved Cascade R-CNN: step 1: photographing a blade part of a fan generator as an original data set; step 2: performing the following data enhancement on the original data set to obtain a data set; step 3: designing a backbone network Tiny-GhostNet of a YOLOv4 target detection network model, and using Tiny-GhostNet to replace the CSPDarknet53 backbone network in the Cascade R-CNN network to obtain a YOLOv4 target detection network model; step 4: designing a channel attention module for the YOLOv4 target detection network model constructed in step 3, and determining a pruning threshold; step 5: replacing the original anchor frame in the YOLOv4 network; step 6: training the YOLOv4 network model, wherein the first half adopts frozen training, that is, the backbone network Tiny-GhostNet is not trained first, and only other parts of the YOLOv4 network model are trained; the second half adopts unfreezing training, that is, the entire YOLOv4 network model is trained; step 7: detecting the collected fan blade images to obtain defect detection results.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision and relates to a fan blade defect detection method based on improved YOLOv4. Background Art

[0002] Wind turbine blades are the most critical components of wind turbines, converting wind energy into electricity. To capture more wind energy, wind turbine blades are becoming increasingly larger. This increase in size increases the likelihood of blade damage. Furthermore, the increased size of wind turbine blades increases their cost, making them more difficult to inspect and maintain. Conventional wind turbine blade inspection methods are extremely labor-intensive and resource-intensive.

[0003] This paper provides a method for detecting defects in wind turbine blades based on an improved YOLOv4, addressing the lack of lightweight wind turbine blade detection models. First, the method lightweights the backbone network, designing a more lightweight Tiny-GhostNet network to replace the original CSPDarknet53 backbone network. Secondly, the YOLOv4 object detection algorithm model is pruned using a channel-attention-based model pruning method. The channel-attention module automatically distinguishes between important and unimportant channels and deletes unimportant channels based on a specific threshold. Finally, the predefined anchor box settings are simplified. Summary of the Invention

[0004] The problem to be solved by the present invention is to provide a fan blade defect detection method based on improved YOLOv4, which addresses the problem of lack of lightweight fan blade detection model.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] A fan blade defect detection method based on an improved Cascade R-CNN includes the following steps:

[0007] Step 1: Use a drone to horizontally circle the stationary wind turbine, focusing on the blades of the wind turbine generator as the original dataset;

[0008] Step 2: Perform the following data enhancement on the original dataset to obtain the dataset;

[0009] Step 3: Design the backbone network Tiny-GhostNet of the YOLOv4 target detection network model, and use Tiny-GhostNet to replace the CSPDarknet53 backbone network in the Cascade R-CNN network to obtain the YOLOv4 target detection network model;

[0010] Step 4: Design a channel attention module for the YOLOv4 target detection network model constructed in step 3 and determine the pruning threshold;

[0011] Step 5: Replace the original anchor box in the YOLOv4 network;

[0012] Step 6: Train the YOLOv4 network model. The first half uses frozen training, that is, the backbone network Tiny-GhostNet is not trained first, and only the other parts of the YOLOv4 network model are trained; the second half uses unfreezing training, that is, the entire YOLOv4 network model is trained;

[0013] After each unfreezing training, a channel attention module is added after the convolutional layer of the Tiny-GhostNet backbone network to perform pruning, that is, to delete the channels corresponding to the channel weights below the pruning threshold; after the model pruning is completed, the channel attention module set after the convolutional layer of the Tiny-GhostNet backbone network is deleted, and the next unfreezing training is performed again with the dataset obtained in step 2, and finally a trained YOLOv4 network model is obtained;

[0014] Step 7: Use the trained YOLOv4 network target detection model obtained in step 6 to detect the collected wind turbine blade images and obtain defect detection results.

[0015] Furthermore, in step 2, the data enhancement includes image flipping, image rotation, image contrast enhancement, image brightness enhancement, adding Gaussian noise, adding salt and pepper noise, and Laplace sharpening.

[0016] Furthermore, in step 3, the steps for designing the backbone network Tiny-GhostNet are as follows:

[0017] Step 3.1: Tiny-Ghost module design

[0018] The Tiny-Ghost module includes Ghost module1, Ghost module1 and Ghost module3, where Ghost module1 is used to implement the following functions: convolve the feature map with the number of channels C into a feature map with the number of channels three-quarters C, then generate a feature map with the number of channels one-quarter C through linear operation on the feature map with the number of channels three-quarters C, and finally superimpose the feature map with the number of channels three-quarters C and the feature map with the number of channels one-quarter C together to output a feature map with the number of channels C;

[0019] Ghost module2 is used to implement the following functions: convolve a feature map with C channels into a feature map with half the number of channels C, then generate a feature map with half the number of channels C through linear operations, and finally superimpose the two feature maps together to output a feature map with C channels;

[0020] Ghost module3 is used to implement the following functions: convolve a feature map with C channels into a feature map with one-quarter of C channels, then generate three feature maps with one-quarter of C channels through linear operations, and finally superimpose the two feature maps together to output a feature map with C channels;

[0021] Step 3.2: Tiny-Ghost Bottleneck Design

[0022] Tiny-Ghost Bottleneck includes three Ghost Bottlenecks: Ghost Bottleneck1, Ghost Bottleneck2, and Ghost Bottleneck3. Each Ghost Bottleneck is constructed through a residual structure. Only one corresponding Ghost module is used in each Ghost Bottleneck, and each Ghost Bottleneck includes two different structures: Ghost Bottleneck with a stride of 1 and Ghost Bottleneck with a stride of 2. The Ghost Bottleneck with a stride of 1 includes two identical Ghost modules; the Ghost Bottleneck with a stride of 2 includes two identical Ghost modules, and a deep convolution DWConv layer is added between the two Ghost modules.

[0023] Step 3.3: Tiny-GhostNet Design

[0024] Use Ghost bottleneck1, Ghost Bottleneck2, and Ghost Bottleneck3 to replace the bottleneck in CSPDarknet53 to obtain the Tiny-GhostNet network. Specifically, use Ghost Bottleneck1 to replace the shallow network of CSPDarknet53, use GhostBottleneck2 to replace the middle layer network of CSPDarknet53, and use Ghost Bottleneck3 to replace the deep network of CSPDarknet53.

[0025] Furthermore, the channel attention module designed in step 4 is used to implement the following functions: global average pooling and global maximum pooling are performed on the input feature map respectively, and weight vectors are generated respectively. The two results are then processed separately by the shared MLP, and the processed results are processed by element-wise addition. Finally, the Sigmoid activation function is used to obtain C channel weights Mc with values between (0, 1), and the C channel weight values w in the channel weight vector Mc.

[0026] Furthermore, in step 4, the pruning threshold is set to 0.35.

[0027] Furthermore, the specific operations of step 5 are as follows: on the smallest feature map 13×13, set a 150×150 anchor box; on the medium feature map 26×26, set an 80×80 anchor box; on the largest feature map 52×52, set 20×20, 15×30, and 30×15 anchor boxes, and the anchor box density is one quarter of the original rough box density in the YOLOv4 network.

[0028] Furthermore, in step 6, during the frozen training phase, the number of batches is set to 16, the learning rate is 0.001, and the weight decay is 0.0005. During the unfrozen training phase, the number of batches is set to 8, the learning rate is 0.0001, and the weight decay is 0.0005. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 Schematic diagram of Ghost module 1, Ghost module 2, and Ghost module 3;

[0030] Figure 2 Schematic diagrams of Ghost Bottleneck 1, Ghost Bottleneck 2, and Ghost Bottleneck 3;

[0031] Figure 3 Schematic diagram of channel attention;

[0032] Figure 4 Schematic diagram of model pruning based on channel attention;

[0033] Figure 5 Configure anchor boxes at different scales;

[0034] Figure 6 Graphs showing different density distributions of anchor boxes. DETAILED DESCRIPTION

[0035] The following describes in detail the implementation of the method of the present invention with reference to the accompanying drawings.

[0036] The technical solution adopted by the present invention is: a fan blade defect detection method based on an improved Cascade R-CNN, comprising the following steps:

[0037] Step 1: Use a drone to horizontally circle the stationary wind turbine, focusing on the blades of the wind turbine generator as the original data set.

[0038] Step 2: Perform the following data enhancement on the original dataset to obtain the dataset.

[0039] Data enhancement includes: image flipping, image rotation, image contrast enhancement, image brightness enhancement, adding Gaussian noise, adding salt and pepper noise, and Laplace sharpening; among them:

[0040] Image flipping: This can be done upside down or left-right. This method can change the position of wind blade defects in the image, enhancing generalization. This data augmentation method is relatively easy and has been proven effective on common datasets.

[0041] Image rotation: The image is rotated by a certain angle. In the present invention, the image is rotated by about 20 degrees. Image rotation can also be regarded as another image flipping, which also changes the position of the fan blade defect in the image and the orientation of the object.

[0042] Image contrast enhancement: used to simulate wind turbine blade images taken under different weather conditions, such as those taken under scorching sun, cloudy days, rainy days, and dusk.

[0043] Image Brightness Enhancement: Changes the brightness of an image to simulate images illuminated by light sources of different intensities, such as photographing wind turbine blades facing or facing away from the light source.

[0044] Adding Gaussian noise: Random values generated by Gaussian distribution are added to the original wind blade image, which can destroy the integrity and continuity of the defect target and simulate various external interference conditions generated in the actual shooting of wind blade images.

[0045] Add salt and pepper noise: Randomly modify the pixels in the original wind blade image. This has the same effect as adding high-speed noise and is used to simulate various external interference conditions that occur in actual wind blade image shooting.

[0046] Laplace sharpening: It enhances some detail features in the original wind blade image and is used to simulate the changes of defects in the wind blade over time.

[0047] Step 3: Based on Cascade R-CNN, build the YOLOv4 target detection network model.

[0048] The backbone network of the YOLOv4 object detection network model is lightweighted. The lightweight backbone network Tiny-GhostNet designed by this invention replaces the CSPDarknet53 backbone network in the original Cascade R-CNN network to obtain the YOLOv4 object detection network model. The steps for designing the lightweight backbone network Tiny-GhostNet are as follows:

[0049] Step 3.1: Tiny-Ghost module design

[0050] The first part shows the Ghost module 1 designed by the present invention, as shown in FIG. Figure 1 As shown in Figure 2 (a), the specific approach is: first, convolve a feature map with C channels into a feature map with three-quarters C channels. Then, perform a linear operation on the three-quarters C feature map to generate a feature map with one-quarter C channels. Finally, superimpose the three-quarters C feature map and the one-quarter C feature map together, outputting a feature map with C channels. As can be seen, in Ghost module 1, the ratio of convolutional operations to linear operations is 3:1, indicating a low degree of simplification. Therefore, Ghost module 1 is primarily used in shallow convolutional layers.

[0051] Assume that the input size is h×w×c, the output is h'×w'×n, and the size of the convolution kernel is k, where k×k and d×d are the same, and s is much smaller than c. The compression ratio of the parameters of Ghost module 1 relative to ordinary convolution is calculated as shown in the following formula. The parameter compression ratio of Ghost module 1 is one-third and one-quarter.

[0052] Where n represents The number of input images, c represents the number of channels of the convolution kernel, k represents the size of the convolution kernel, and d×d represents the number of linear units of the linear operation.

[0053] The second part shows the Ghost module 2 designed by the present invention, which is the same as the original Ghost module. Figure 1 As shown in Figure 2 (b), the specific approach is: first, convolve a feature map with C channels into a feature map with half the number of channels, then perform a linear operation to generate another feature map with half the number of channels, and finally superimpose the two feature maps together to output a feature map with C channels. It can be seen that in Ghost module 2, the ratio of convolution operations to linear operations is 1:1, which is relatively simple. Therefore, Ghost module 2 can be used in intermediate convolutional layers.

[0054] Assume that the input size is h×w×c, the output is h'×w'×n, and the size of the convolution kernel is k, where k×k and d×d are the same, and s is much smaller than c. The compression ratio of the parameters of Ghost module 1 relative to ordinary convolution is calculated as shown in the following formula. The parameter compression ratio of Ghost module 2 is 2.

[0055] r=n×c×k×k≈2×c≈2

[0056] n2×c×k×k+(2-1)×n2×d×d2+c-1

[0057] The third part shows the Ghost module 3 designed by the present invention. Figure 1 As shown in Figure (c), the specific approach is: first, a feature map with C channels is convolved into a feature map with one-quarter C channels. Then, a linear operation is performed to generate three feature maps with one-quarter C channels. Finally, the two feature maps are superimposed together, and the output feature map still has C channels. It can be seen that in Ghost module 3, the ratio of convolution operations to linear operations is 1:3, which is a high degree of simplification. Therefore, Ghost module 3 is mainly used in deep convolutional layers.

[0058] Assume the input size is h×w×c, the output is h'×w'×n, the size of the convolution kernel is k, where k×k and d×d are the same, and s is much smaller than c. The compression ratio of the parameters of Ghost module 1 relative to ordinary convolution is calculated as shown in the following formula. The parameter compression ratio of Ghost module 2 is 4.

[0059]

[0060] Step 3.2: Tiny-Ghost Bottleneck Design

[0061] like Figure 2 (a) Figure 2 (b) and Figure 2As shown in (c), the lightweight innovative modules Ghost Bottleneck1, Ghost Bottleneck2, and Ghost Bottleneck3 are respectively designed to be composed of different Ghost modules. In addition, the present invention adds a DWConv layer to the Ghost module to perform channel weight enhancement on the features generated by the previous convolutional layer to eliminate the redundant information generated in the feature generation process. In addition, the integration of modules with different Ghost bottlenecks is to meet the application requirements of sub-networks with different depths. Only one corresponding Ghost module is used in each Ghost Bottleneck. For example, only Ghost module1 is used in Ghost Bottleneck1, and Ghost module2 and Ghostmodule3 are not used. Each Ghost Bottleneck is constructed through a residual structure, and each Ghost Bottleneck includes two different structures: Ghost Bottleneck with a step size of 1 and Ghost Bottleneck with a step size of 2. Among them, Ghost Bottleneck with a step size of 1 includes two identical Ghost modules. For example, in Ghost Bottleneck1, both Ghost modules are Ghost module1; Ghost Bottleneck with a step size of 2 includes two identical Ghostmodules, and a deep convolution DWConv layer is added between the two Ghost modules.

[0062] Step 3.3: Tiny-GhostNet Design

[0063] Tiny-GhostNet is mainly composed of Ghost Bottleneck1, Ghost Bottleneck2, and GhostBottleneck3. In the early stage of feature extraction, Ghost Bottleneck1 with a large proportion of convolution operations can fully extract the effective information of the input; and when the network gradually deepens, the present invention uses Ghost Bottleneck2 and Ghost Bottleneck3 with more linear operations to simplify the network. Placing modules with different proportions of convolutional layers and linear operations at different depths of the network can maximize the effectiveness of feature extraction. Use Ghostbottleneck1, Ghost Bottleneck2, and Ghost Bottleneck3 to replace the bottleneck in CSPDarknet53 to obtain the Tiny-GhostNet network. Specifically: the input feature map size in CSPDarknet53 is 208 2 ×16, 104 2 The ×24 shallow network is replaced by Ghost Bottleneck1, and the input feature map size in CSPDarknet53 is 52 2 ×40, 26 2 ×80, 26 2 The intermediate layer network of ×122 is replaced by Ghost Bottleneck2, and the input feature map size in CSPDarknet53 is 13 2 The ×160 deep network is replaced with Ghost Bottleneck3. The Tiny-GhostNet network architecture is shown in Table 1.

[0064] Table 1 Tiny-GhostNet network architecture

[0065]

[0066] Step 4: Design a channel attention module for the YOLOv4 object detection network model built in Step 3 and determine the pruning threshold. In this step, the channel attention module is designed to automatically distinguish important channels from unimportant channels. Based on the set pruning threshold, it is used for model pruning during the subsequent network model training process to remove unimportant channels. The specific steps include the following:

[0067] Step 4.1: Design channel attention module

[0068] The present invention designs a channel attention module to obtain the importance of the convolutional layer channels in the network model to be optimized. If the importance of a convolutional layer channel calculated by the channel attention module is smaller, it can be regarded as having a smaller contribution to the generated features, and vice versa. Then, according to the importance, the redundant convolutional channels in the network model to be optimized are deleted to complete the model pruning, such as Figure 3 As shown in the figure, for the input feature map of size W×H×C (W represents width, H represents height, and C represents the number of channels), global average pooling and global maximum pooling are performed respectively, and weight vectors of length C are generated respectively. Then, the two results are processed separately by the shared MLP, and the processed results are processed by element-wise addition. Finally, the Sigmoid activation function is used to obtain C channel weights Mc with values between (0, 1).

[0069] The generated channel weight vector Mc contains C channel weight values w. A larger w value indicates a more important channel to the object detection model, while a smaller w value indicates a less important channel. Therefore, a channel weight w of 0 sets all corresponding channel features to 0. The channel attention module is independent of the width W and height H of the convolutional layer. Therefore, the channel attention module requires very few parameters and has minimal impact on the network model.

[0070] Step 4.2: The channel attention module is added after the convolutional layer of the Tiny-GhostNet backbone network.

[0071] Methods for applying channel attention to model pruning include Figure 4 As shown. Normally, the input feature layer will be input to the next layer after the current convolutional layer performs the convolution operation. However, before inputting to the next layer, the present invention adds the channel attention mechanism module (Channel Attention Module) obtained in step 3.1 after the last convolutional layer of the network to be pruned (i.e., the Tiny-GhostNet network obtained in step 3), and finally uses the Sigmoid activation function to calculate the weight of the output channel. In this way, by using channel attention, the network can autonomously learn which channels are important and which channels are not important. In this way, by using the channel weights output by the attention mechanism module and setting a certain threshold, the corresponding channels can be selected for deletion, so that the network model is simplified.

[0072] The present invention conducted the following pruning threshold experiments and obtained a pruning threshold of 0.35. As shown in Table 2, as the threshold gradually increases, the mAP value gradually decreases, and the proportion of pruned channels gradually increases, so it is necessary to find a balance point from the data in the table. Initially, the mAP value decreases less, and the proportion of pruned channels is also small. After that, the mAP value decreases more, and the proportion of pruned channels is also higher. When the threshold is set to 0.35, the mAP value of the YOLOv4 object detection network model decreases by 0.87%, and the proportion of pruned channels is 10.0%. When the pruning threshold is less than 0.35, the mAP value decreases slightly, but the proportion of pruned channels is also relatively small. When the pruning threshold is greater than 0.35, although the proportion of pruned channels is larger, the mAP value decreases significantly, which significantly affects the performance of the network model. Therefore, setting the model pruning threshold to 0.35 can achieve a balance between accuracy and parameter count.

[0073] Table 2 Pruning results with different weight thresholds

[0074]

[0075] Step 5: Replace the original anchor boxes in the YOLOv4 network. Specifically, on the smallest feature map (13×13), set an anchor box of (150×150); on the medium feature map (26×26), set an anchor box of (80×80); on the largest feature map (52×52), set anchor boxes of (20×20, 15×30, and 30×15). The anchor box density is one-fourth of the original raw box density in the YOLOv4 network. The anchor box design experiment is as follows:

[0076] Experiment 1: Anchor box configurations at different scales

[0077] The default anchor box is set to (such as Figure 5 (a)): On the smallest feature map (13×13), three larger-scale anchor boxes (116×90, 156×198, 373×326) are set, which are the three black anchor boxes in the figure; on the middle feature map (26×26), three medium-scale anchor boxes (30×61, 62×45, 59×119) are set, which are the three blue anchor boxes in the figure; on the largest feature map (52×52), three smaller-scale anchor boxes (10×13, 16×30, 33×23) are set, which are the three green anchor boxes in the figure.

[0078] like Figure 5As shown in (b), a preliminary screening is performed on three feature maps, leaving two horizontal and vertical anchor boxes on each feature map, and removing the square anchor boxes. That is, on the smallest feature map (13×13), anchor boxes of (200×320, 320×200) are set; on the medium feature map (26×26), anchor boxes of (120×60, 60×120) are set; on the largest feature map (52×52), anchor boxes of (15×30, 30×15) are set. Figure 5 (c) is the opposite of solution b. It leaves a square anchor box on each feature map and removes the two horizontal and vertical anchor boxes. That is, on the smallest feature map (13×13), a (150×150) anchor box is set; on the medium feature map (26×26), an (80×80) anchor box is set; on the largest feature map (52×52), a (20×20) anchor box is set. Figure 5 (d) shows an approach based on solution c. Considering the large number of small objects in the wind turbine blade dataset, two anchor boxes are added to the largest feature map, while feature maps of other scales remain unchanged. Specifically, on the smallest feature map (13×13), an anchor box of (150×150) is set; on the medium feature map (26×26), an anchor box of (80×80) is set; and on the largest feature map (52×52), anchor boxes of (20×20, 15×30, and 30×15) are set. The final experimental results show that solution d reduces the number and types of anchor boxes without significantly compromising detection accuracy. Therefore, this paper ultimately selected the anchor box setting of solution d.

[0079] Experiment 2: Anchor box distribution with different density configurations

[0080] After reducing the number and types of predefined anchor frames for a single feature point, we can consider starting with the distribution density between predefined anchor frames, abandoning the previous dense distribution and using a sparse distribution of predefined anchor frames, that is, reducing the number of anchor frames as a whole. The previous analysis of the target object in the wind blade dataset concluded that the defects of the wind blades in the wind blade dataset are sparsely distributed. Therefore, in principle, it is feasible to reduce the distribution density between predefined anchor frames. Figure 6 As shown in the figure, solution (a) shows the default predefined anchor box distribution density, solution (b) shows half the density of solution (a), solution (c) shows one-quarter the density of solution (a), and solution (d) shows one-eighth the density of solution (a). Similar to reducing the number of predefined anchor boxes for a single feature point, the accuracy loss should also be considered when selecting predefined anchor boxes with different densities. In the experimental results, the density of solution (c) significantly reduces the number of anchor boxes while causing little loss in accuracy.

[0081] Step 6: Train the YOLOv4 network model. Specifically, the training process is divided into two parts. The first half uses frozen training, meaning the Tiny-GhostNet backbone network is not trained first. Instead, the rest of the YOLOv4 network model is trained to fine-tune other parameters. During the frozen training phase, the batch size is set to 16, the learning rate is 0.001, and the weight decay is 0.0005. The second half uses unfrozen training, meaning the Tiny-GhostNet backbone network is unfrozen and the entire YOLOv4 network model is trained. All parameters in the network are modified. During the unfrozen training phase, the batch size is 8, the learning rate is 0.0001, and the weight decay is 0.0005. After each unfrozen training session, a channel attention module is added after the convolutional layers of the Tiny-GhostNet backbone network to perform pruning. This removes channels whose weights fall below the pruning threshold. After the model pruning is completed, the network structure is adjusted, the channel attention module set after the convolution layer of the Tiny-GhostNet backbone network is deleted, and the dataset obtained in step 2 is used again for the next unfreeze training to obtain the trained YOLOv4 network model.

[0082] In this step, pruning is combined with training to ensure that the accuracy of the target detection network is effectively restored.

[0083] At the beginning of formal training, to speed up the training process, we first used the backbone network to train on the common classification dataset Imagenet. This can enable faster convergence and better network training results. Throughout the training process, a weight file is saved after each epoch. The experimental parameters are shown in Table 3 below:

[0084] Table 3 Experimental parameter settings

[0085]

[0086] Step 7: Use the trained YOLOv4 network target detection model obtained in step 6 to detect the collected wind turbine blade images and obtain defect detection results.

[0087] To verify the effectiveness of each improvement strategy, we conducted further ablation experiments, as shown in the table below. The baseline is set to YOLOv4, backbone network simplification involves replacing the original CSPDarknet53 with Tiny-GhostNet designed in this invention, model pruning is the channel-attention-based model pruning of this invention, and anchor box simplification involves reducing the number and density of anchor boxes in this invention. The evaluation metrics compared on the right are mAP, parameter count, and FPS. Each simplification strategy reduces the number of parameters compared to the original, while also ensuring minimal loss of accuracy.

[0088] Table 4 Ablation experiment

[0089]

[0090] The table below shows the final ablation test results. To compare the performance of Tiny-GhostNet, MobileNetv1, MobileNetv2, MobileNetv3, and GhostNet were added as backbone networks for comparison. Model 1 in the table is the original Yolov4 model; Model 2 incorporates GhostNet; Model 3 incorporates MobileNetv1; Model 4 incorporates MobileNetv2; Model 5 incorporates MobileNetv3; and Model 6 incorporates Tiny-GhostNet. Models 7 and 8 further refine Model 6 by performing channel pruning and anchor box simplification, respectively. The evaluation metrics compared on the right include mAP, parameter count, and FPS. Each simplification strategy reduces the number of parameters compared to the original model while ensuring minimal loss of accuracy.

[0091] Table 5 Comparative experimental results

[0092]

Claims

1. A fan blade defect detection method based on improved Cascade R-CNN, characterized in that: The following steps are involved: Step 1: Use a drone to horizontally circle the stationary wind turbine, focusing on the blades of the wind turbine generator as the original dataset; Step 2: Perform the following data enhancement on the original data set to obtain a data set; wherein the data enhancement includes image flipping, image rotation, image contrast enhancement, image brightness enhancement, adding Gaussian noise, adding salt and pepper noise, and Laplace sharpening; Step 3: Design the backbone network Tiny-GhostNet of the YOLOv4 target detection network model and use Tiny-GhostNet to replace the CSPDarknet53 backbone network in the Cascade R-CNN network to obtain the YOLOv4 target detection network model. The steps for designing the backbone network Tiny-GhostNet are as follows: Step 3.1: Tiny-Ghost module design The Tiny-Ghost module includes Ghost module1, Ghost module1 and Ghost module3, where Ghost module1 is used to implement the following functions: convolve the feature map with the number of channels C into a feature map with the number of channels three-quarters C, then generate a feature map with the number of channels one-quarter C through linear operation on the feature map with the number of channels three-quarters C, and finally superimpose the feature map with the number of channels three-quarters C and the feature map with the number of channels one-quarter C together to output a feature map with the number of channels C; Ghost module2 is used to implement the following functions: convolve a feature map with C channels into a feature map with half the number of channels C, then generate a feature map with half the number of channels C through linear operations, and finally superimpose the two feature maps together to output a feature map with C channels; Ghost module3 is used to implement the following functions: convolve a feature map with C channels into a feature map with one-quarter of C channels, then generate three feature maps with one-quarter of C channels through linear operations, and finally superimpose the two feature maps together to output a feature map with C channels; Step 3.2: Tiny-Ghost Bottleneck Design Tiny-Ghost Bottleneck includes three Ghost Bottlenecks: Ghost Bottleneck1, Ghost Bottleneck2, and Ghost Bottleneck3. Each Ghost Bottleneck is constructed through a residual structure. Only one corresponding Ghost module is used in each Ghost Bottleneck, and each Ghost Bottleneck includes two different structures: Ghost Bottleneck with a stride of 1 and Ghost Bottleneck with a stride of 2. The Ghost Bottleneck with a stride of 1 includes two identical Ghost modules; the Ghost Bottleneck with a stride of 2 includes two identical Ghost modules, and a deep convolution DWConv layer is added between the two Ghost modules. Step 3.3: Tiny-GhostNet Design Use Ghost bottleneck1, Ghost Bottleneck2, and Ghost Bottleneck3 to replace the bottleneck in CSPDarknet53 to obtain the Tiny-GhostNet network. Specifically, use Ghost Bottleneck1 to replace the shallow network of CSPDarknet53, use Ghost Bottleneck2 to replace the middle layer of CSPDarknet53, and use Ghost Bottleneck3 to replace the deep network of CSPDarknet53. Step 4: Design a channel attention module for the YOLOv4 target detection network model constructed in step 3 and determine the pruning threshold; Step 5: Replace the original anchor box in the YOLOv4 network; Step 6: Train the YOLOv4 network model. The first half uses frozen training, that is, the backbone network Tiny-GhostNet is not trained first, and only the other parts of the YOLOv4 network model are trained; the second half uses unfreezing training, that is, the entire YOLOv4 network model is trained; After each unfreezing training, a channel attention module is added after the convolutional layer of the Tiny-GhostNet backbone network to perform pruning, that is, to delete the channels corresponding to the channel weights below the pruning threshold; after the model pruning is completed, the channel attention module set after the convolutional layer of the Tiny-GhostNet backbone network is deleted, and the next unfreezing training is performed again with the dataset obtained in step 2, and finally a trained YOLOv4 network model is obtained; Step 7: Use the trained YOLOv4 network target detection model obtained in step 6 to detect the collected wind turbine blade images and obtain defect detection results.

2. The fan blade defect detection method based on the improved Cascade R-CNN according to claim 1, characterized in that: The channel attention module designed in step 4 is used to achieve the following functions: perform global average pooling and global maximum pooling on the input feature map, generate weight vectors respectively, and then process the two results separately by the shared MLP, and process the processed results by element-wise addition. Finally, use the Sigmoid activation function to obtain C channel weights Mc between (0, 1), and C channel weight values w in the channel weight vector Mc.

3. The fan blade defect detection method based on the improved Cascade R-CNN according to claim 1, characterized in that: In step 4, the pruning threshold is set to 0.

35.

4. The fan blade defect detection method based on the improved Cascade R-CNN according to claim 1, characterized in that: The specific operations of step 5 are as follows: on the smallest feature map 13×13, set an anchor box of 150×150; on the medium feature map 26×26, set an anchor box of 80×80; on the largest feature map 52×52, set anchor boxes of 20×20, 15×30, and 30×15, and the anchor box density is one quarter of the original rough box density in the YOLOv4 network.

5. The fan blade defect detection method based on the improved Cascade R-CNN according to claim 1, characterized in that: In step 6, during the frozen training phase, the number of batches is set to 16, the learning rate is 0.001, and the weight decay is 0.0005; during the unfrozen training phase, the number of batches is set to 8, the learning rate is 0.0001, and the weight decay is 0.0005.

Images