Fan blade surface defect detection method based on improved YOLOv8n

Abstract

The invention relates to the technical field of image processing, in particular to an improved YOLOv8n-based fan blade surface defect detection method, which comprises the following steps of: collecting a wind driven generator blade surface defect data set, and preprocessing the data set; replacing C2f modules of a seventh layer and a ninth layer in a trunk feature extraction module of the YOLOv8n network with C2fESEDC modules; a DWConv attention mechanism and an EffectiveSE attention mechanism are added into the C2fESEDC module, so that the capturing capability of global information is improved; a SlimNeck structure is introduced into a neck network of the YOLOv8n; and a detection head of the YOLOv8n network adopts a SegmentSGNC structure, and a target detection model based on the improved YOLOv8n is constructed. According to the method, the problems of insufficient detection precision and large parameter quantity when an existing YOLOv8n network is used for detecting the surface defects of the fan blade are solved.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method for detecting surface defects of wind turbine blades based on improved YOLOv8n. Background Art

[0002] Defects in wind turbines may have many impacts on the surface of the unit, which not only affect the operating efficiency of the unit, but also the safety and service life of the unit. Among them, the impact on operating efficiency includes: reduced aerodynamic performance and weakened heat dissipation effect; unit safety includes: reduced structural strength and failure of lightning protection system; service life includes: accelerated aging and increased maintenance costs; and visual pollution: affecting the quality of power generation and other issues.

[0003] The patent publication number is CN118735900A. It improves the backbone part of YOLOv8n and integrates StarNet_Blocks into the C2f module, but the detection accuracy is not significantly improved. In addition, the detection head part and the use of the conventional Powerful-IoUv2 loss function have problems such as insufficient detection accuracy and a large number of parameters. Summary of the invention

[0004] In view of the shortcomings of the existing methods, the present invention solves the problems of insufficient detection accuracy and large number of parameters in the existing YOLOv8n network for detecting surface defects of wind turbine blades.

[0005] The technical solution adopted by the present invention is: a method for detecting surface defects of fan blades based on improved YOLOv8n comprises the following steps:

[0006] Step 1: Collect a data set of surface defects of wind turbine blades and preprocess the data set;

[0007] As a preferred embodiment of the present invention, the blade surface defects include: blade edge erosion, lightning receptor abnormality, blade surface damage and vortex generator abnormality.

[0008] As a preferred implementation of the present invention, the preprocessing includes: horizontally flipping, vertically flipping, randomly rotating, and randomly adjusting the brightness of the defective image.

[0009] Step 2: Replace the C2f modules of the seventh and ninth layers in the backbone feature extraction module of the YOLOv8n network with the C2f_ESEDC module; integrate the DWConv and Effective_SE attention mechanisms into the C2f module to improve the ability to capture global information;

[0010] As a preferred implementation of the present invention, the C2f_ESEDC module improves the Bottleneck block of the C2f module, and inserts DWConv and Effective_SE attention in sequence between the two Conv convolutions of the Bottleneck block.

[0011] Step 3: Introduce the SlimNeck structure into the neck network of YOLOv8n;

[0012] Step 4: The detection head of the YOLOv8n network adopts the Segment_SGNC structure to build a target detection model based on the improved YOLOv8n;

[0013] As a preferred embodiment of the present invention, the Segment_SGNC structure includes: the P3, P4 and P5 feature maps in the neck network are successively passed through a 1*1 Conv_GN and two 3*3 Conv_GNs to obtain Bbox_Loss and Cls_Loss; the Segment part directly generates Proto from the P3 feature map, and P3, P4 and P5 are successively passed through a 1*1 Conv_GN and a 3*3 Conv_GN to obtain Mask_Coef; and Mask_Coef and Proto are linearly superimposed.

[0014] As a preferred implementation of the present invention, Conv_GN replaces batch normalization in Conv with group normalization.

[0015] As a preferred implementation of the present invention, two 3*3 Conv_GNs share parameters.

[0016] As a preferred embodiment of the present invention, a wind blade surface defect detection system based on an improved Yolov8 network includes: a memory for storing instructions executable by a processor; and a processor for executing instructions to implement a wind blade surface defect detection method based on an improved YOLOv8n.

[0017] As a preferred embodiment of the present invention, a computer readable medium stores a computer program code, and when the computer program code is executed by a processor, a method for detecting surface defects of wind turbine blades based on improved YOLOv8n is implemented.

[0018] Beneficial effects of the present invention:

[0019] 1. Replace the C2f modules of the seventh and ninth layers in the YOLOv8n backbone feature extraction module with the C2f_ESEDC module to improve detection accuracy and robustness;

[0020] 2. The detection head adopts the Segment_SGNC structure, which effectively reduces the number of parameters in the detection head and enables the detection head to have higher multi-scale perception capabilities;

[0021] 3. The improved network reduces the demand for device memory and processors required for deployment, is more capable of edge deployment, and can better adapt to image data detection tasks in generalized environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 This is a schematic diagram of the improved YOLOv8n network structure of the present invention;

[0023] Figure 2 It is a schematic diagram of the structure of the C2f_ESEDC module of the present invention;

[0024] Figure 3 This is the schematic diagram of the Effective_SE module structure;

[0025] Figure 4 It is a schematic diagram of the Segment_SGNC structure of the present invention;

[0026] Figure 5 A schematic diagram of the training results of the wind turbine blade surface defect detection method based on the improved YOLOv8n;

[0027] Figure 6 Schematic diagram of the detection results of the wind blade surface defect detection method based on the improved YOLOv8n. DETAILED DESCRIPTION

[0028] The present invention is further described below in conjunction with the accompanying drawings and embodiments. This figure is a simplified schematic diagram, which only illustrates the basic structure of the present invention in a schematic manner, and therefore it only shows the components related to the present invention.

[0029] like Figure 1 As shown, the wind turbine blade surface defect detection method based on improved YOLOv8n includes the following steps:

[0030] Step 1: Collect a data set of surface defects of wind turbine blades and preprocess the data set;

[0031] First, the public wind turbine blade surface defect dataset was downloaded from the Roboflow network platform. Then, the initial defect images were enhanced by four methods: horizontal flipping, vertical flipping, random rotation, and random brightness adjustment. After data enhancement, the wind turbine blade surface defect dataset has a total of 2401 images, and the dataset is divided into a training set and a test set in a ratio of 8:2.

[0032] Among them, the defect types include blade edge erosion, lightning receptor anomaly, blade surface damage and vortex generator anomaly.

[0033] Step 2: Replace the C2f modules of the seventh and ninth layers in the backbone feature extraction module of the YOLOv8n network with the C2f_ESEDC module;

[0034] The YOLOv8n network uses YOLOv8n-seg.

[0035] The YOLOv8n network extracts features from the input image, and the C2f modules of the seventh and ninth layers of Backbone are replaced with C2f_ESEDC modules.

[0036] The network used to extract feature information from the images in the training set includes:

[0037] The first layer of Backbone is the Conv module, which has 64 channels and a convolution kernel size of 3*3. The convolution operation is performed on the input image, and the number of channels becomes 64, and the feature map is reduced to half of the original;

[0038] The second layer is the Conv module, with 128 channels and a convolution kernel size of 3*3. The convolution operation is continued on the feature map, the number of channels becomes 128, and the feature map is reduced to half of the original size.

[0039] The third layer is the C2f module with 128 channels, which fuses feature maps of different scales.

[0040] The fourth layer is the Conv module, with 256 channels and a convolution kernel size of 3*3. The feature map is convolved again, with the number of channels becoming 256 and the feature map reduced to half of its original size.

[0041] The fifth layer is the C2f module with 256 channels, which further fuses the feature maps;

[0042] The sixth layer is the Conv module, with 512 channels and a convolution kernel size of 3*3. The convolution operation is continued on the feature map, the number of channels becomes 512, and the feature map is reduced to half of the original size.

[0043] The seventh layer is the C2f_ESEDC module with 512 channels, which further fuses the feature maps;

[0044] The eighth layer is the Conv module, with 1024 channels and a convolution kernel size of 3*3. The feature map is convolved again, with the number of channels becoming 1024 and the feature map reduced to half of its original size.

[0045] The ninth layer is the C2f_ESEDC module with 1024 channels, which further fuses the feature maps;

[0046] The tenth layer is the SPPF module with 1024 channels and a pooling kernel size of 5. It performs spatial pyramid pooling operations to extract features at different scales, thereby capturing detailed information of the target object at different scales.

[0047] Among them, Figure 2 The figure shows the C2f_ESEDC module. The feature map passes through a Conv to generate an intermediate feature map. The generated intermediate feature map is split into two parts through the Split operation. One part is directly passed to the final Concat block for splicing, and the other part is passed to n Bottleneck blocks for further processing. The spliced ​​feature map is input to the final Conv to generate the final output feature map.

[0048] The feature map of the Bottleneck block passes through a Conv, then a DWConv, then an Effective_SE attention mechanism, then another Conv, and finally performs residual fusion with the input feature map of the top layer.

[0049] Traditional convolution Conv means that in the convolution operation, each convolution kernel needs to learn a complete two-dimensional parameter matrix; DWConv is a method that decomposes traditional convolution into two steps: deep convolution and point-by-point convolution; DWConv first performs convolution operation on each channel of the input feature map, and then performs point-by-point convolution on the output channel; assuming that the size of the convolution kernel is D K *D K , the number of input channels is M, the number of output channels is N, and the size of the output feature map is D F *D F ; Then the number of parameters of traditional convolution is D K *D K *M*N, the amount of calculation is D K *D K *M*N*D F *D F ; The number of parameters of DWConv is D K *D K *M+M*N; the amount of calculation is D K *D K *M*D F *D F +M*N*D F *D F ; The parameter ratio of traditional convolution and DWConv is The computational complexity ratio between traditional convolution and DWConv is

[0050]

[0051] In the above formula, N is generally large. Can be ignored; if the size of the convolution kernel D K =3, the number of parameters and computational complexity after using DWConv is reduced to about one-ninth of the original; therefore, compared with traditional convolution, DWConv has fewer parameters and computational complexity, and is easier to implement and deploy on different platforms.

[0052] Effective_SE attention mechanism can effectively extract the dependencies between features, thereby improving the model's ability to capture global information; its module structure is as follows Figure 3 As shown in the figure, the input feature map is first compressed into a scalar after Global Pooling, and then passed through a fully connected layer FC to convert the compressed feature vector into a weight vector for weighting the feature map of each channel, and then passed through a Sigmoid activation function, and finally passed through a scale layer Scale to re-weight and redistribute the weights of each channel before output.

[0053] Step 3: Introduce the SlimNeck structure into the neck network of YOLOv8n;

[0054] The SlimNeck structure is the same as the structure of publication number CN118735900A, and will not be described again here.

[0055] Step 4: The detection head of the YOLOv8n network adopts the Segment_SGNC structure to build a target detection model based on the improved YOLOv8n;

[0056] The original detection head of YOLOv8n has a large number of parameters, accounting for one-fifth of the entire algorithm's computational complexity; all three detection heads need to extract image information through two 3*3 convolutions and one 1*1 convolution respectively; this structure leads to a significant increase in the number of algorithm parameters; secondly, the traditional single-scale prediction structure used by the original algorithm cannot handle multi-scale targets well; it only predicts from one scale of the feature map, ignoring the contribution of other scale features to detection; therefore, the Segment_SGNC (Shared GroupNorm Convolutional) structure is adopted.

[0057] Segment_SGNC structure diagram Figure 4 As shown, first Figure 1 The results of the sixteenth, nineteenth and twenty-second layers of the middle Neck part are given as P3, P4 and P5.

[0058] In the Detect part, P3, P4 and P5 are respectively passed through a 1*1 Conv_GN and then two 3*3 Conv_GNs, and finally Bbox_Loss (bounding box loss) and Cls_Loss (category loss) are obtained; Bbox_Loss is used to calculate the difference between the predicted bounding box and the true bounding box, and Cls_Loss is used to measure the difference between the category distribution predicted by the model and the true label; at the same time, in order to deal with the problem of inconsistent target scales detected by each detection head, the result of Bbox_Loss uses the scale layer Scale to scale the features; in the Segment part, P3 first directly generates Proto (mask prototype map), and then P3, P4 and P5 are respectively passed through a 1*1 Conv_GN and then a 3*3 Conv_GN, and finally Mask_Coef (confidence of the mask prototype map) is obtained; the segmentation result Mask of the target object can be generated by linear superposition of Mask_Coef and Proto.

[0059] The Conv_GN in the Segment_SGNC structure replaces the batch normalization BN in the ordinary convolution Conv with the group normalization GN, and the convolution shared parameters in the red part, that is, the second and third Conv_GN shared parameters in the Detect part; the basic principle of shared convolution is that the weight parameters used by the same convolution kernel are consistent when performing convolution operations at different positions in the image; this weight sharing mechanism means that no matter where the convolution kernel operates in the image, the weight applied is always the same, which significantly reduces the number of parameters that the network needs to learn, thereby improving computational efficiency and reducing the risk of overfitting; in YOLOv8n, the original Detect part P3, P4 and P5 go through different 3*3 Conv, while the Segment_SGNC structure only goes through the same 3*3 Conv, which reduces the 3*3 Conv from the original six to two, significantly reducing the number of network parameters.

[0060] Through the above structure, the number of parameters of the detection head can be effectively reduced, while the detection head can have higher multi-scale perception capabilities so that it can be deployed on resource-constrained devices.

[0061] The dataset is trained on the improved YOLOv8n network, the batch size is set to 16, the total number of iterations is 350 rounds, and the training platform parameters are shown in Table 1.

[0062] Table 1 Experimental hardware and software platform

[0063]

[0064] The detection accuracy and edge application capability of the improved algorithm are evaluated. The parameter size (Params) and the floating-point size (GFLOPs) determine the demand for device memory channels and the pressure of the processor to process image parameters when the network model is deployed. The detection accuracy (mAP@0.5) determines the ability and practicality of this method for edge detection. This dataset is trained on the standard YOLOv8n-seg and compared in the above aspects. The target box average precision Box_mAP@0.5 and the mask average precision Mask_mAP@0.5 when IoU=0.5 are selected as the detection accuracy. The comparison results are shown in Table 2.

[0065] Table 2 Comparative experimental data

[0066]

[0067] Compared with the standard YOLOv8n, the network model of the present invention improves Box_mAP@0.5 by 2.6%, Mask_mAP@0.5 by 2%, reduces the number of parameters by 24%, and reduces GFLOPs by 25%, greatly reducing the pressure requirements on the memory and processor of the edge deployment device; while the model in the patent publication number CN118735900A tested in the data set: YOLOv8n+C2f_Star+SlimNeck+Powerful-IoUv2 structure, although the number of parameters and GFLOPs are reduced, Box_mAP@0.5 and Mask_mAP@0.5 are not improved; therefore, the improved algorithm of the present invention has better detection capabilities for more complex application scenarios.

[0068] Figure 5 The graph in shows the performance of the invented model during the training process. Figure 6 A schematic diagram of the wind turbine blade surface defect detection results is shown.

[0069] Based on the above ideal embodiments of the present invention, the relevant staff can make various changes and modifications without departing from the technical concept of the present invention through the above description. The technical scope of the present invention is not limited to the contents of the specification, and its technical scope must be determined according to the scope of the claims.

Claims

1. A wind turbine blade surface defect detection method based on improved YOLOv8n is characterized in that: The following steps are involved: Step 1: Collect a data set of surface defects of wind turbine blades and preprocess the data set; Step 2: Replace the C2f modules of the seventh and ninth layers in the backbone feature extraction module of the YOLOv8n network with the C2f_ESEDC module. The C2f_ESEDC module integrates the DWConv and Effective_SE attention mechanisms into the C2f module to improve the ability to capture global information. Step 3: Introduce the SlimNeck structure into the neck network of YOLOv8n; Step 4: The detection head of the YOLOv8n network adopts the Segment_SGNC structure to build a target detection model based on the improved YOLOv8n.

2. The method for detecting surface defects of wind turbine blades based on improved YOLOv8n according to claim 1, characterized in that: The C2f_ESEDC module improves the Bottleneck block of the C2f module by inserting DWConv and Effective_SE attention sequentially between the two Conv convolutions of the Bottleneck block.

3. The method for detecting surface defects of wind turbine blades based on improved YOLOv8n according to claim 1, characterized in that: The Segment_SGNC structure includes: passing the P3, P4 and P5 feature maps in the neck network through 1*1 Conv_GN and two 3*3 Conv_GNs in turn to obtain Bbox_Loss and Cls_Loss; the Segment part directly generates Proto from the P3 feature map, and passes the P3, P4 and P5 feature maps through 1*1 Conv_GN and 3*3 Conv_GN in turn to obtain Mask_Coef; and linearly superimposes Mask_Coef and Proto.

4. The method for detecting surface defects of wind turbine blades based on improved YOLOv8n according to claim 3 is characterized in that: Conv_GN replaces batch normalization in Conv with group normalization.

5. The method for detecting surface defects of wind turbine blades based on improved YOLOv8n according to claim 3 is characterized in that: Two 3*3 Conv_GN share parameters.

6. The method for detecting surface defects of wind turbine blades based on improved YOLOv8n according to claim 1, characterized in that: Blade surface defects include: blade edge erosion, lightning receptor anomaly, blade surface damage and vortex generator anomaly.

7. The method for detecting surface defects of wind turbine blades based on improved YOLOv8n according to claim 1, characterized in that: Preprocessing includes: horizontal flipping, vertical flipping, random rotation and random brightness adjustment of the defect image.

8. The wind turbine blade surface defect detection system based on improved YOLOv8n is characterized by: include: a memory for storing instructions executable by a processor; A processor, used to execute instructions to implement the wind turbine blade surface defect detection method based on improved YOLOv8n as described in any one of claims 1-7.

9. A computer readable medium storing a computer program code, characterized in that: When the computer program code is executed by a processor, the computer program code implements the wind blade surface defect detection method based on improved YOLOv8n as described in any one of claims 1 to 7.

Images