YOLOv9-based wind turbine blade small target detection method

Abstract

The invention discloses a wind turbine blade small target defect detection method based on YOLOv9, and the method comprises the steps: obtaining a wind turbine blade defect image data set, and constructing a small target defect detection model; according to the small target defect detection model, on the basis of YOLOv9, a large selective kernel attention mechanism is adopted to dynamically adjust a receiving field, an original down-sampling module in the YOLOv9 is replaced with an SPD-Conv down-sampling module, an original up-sampling module in the YOLOv9 is replaced with a CARAF up-sampling module, and the small target defect detection model is trained. And carrying out small target defect detection on the wind turbine blade by utilizing the trained small target defect detection model. The method can greatly improve the detection precision of the turbine blade small target, and is suitable for low-quality image data.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence technology, and relates to wind power equipment fault detection based on deep learning, and in particular to a small target detection method for wind turbine blades based on YOLOv9. Background Art

[0002] Due to the random fluctuations in wind direction, coupled with long-term exposure to extreme temperatures, rainfall, and more complex and variable loads, wind power equipment fails. Among them, wind turbine blade (WTB) failure is the main cause of wind turbine accidents. These failures involve the performance of blades under various internal and external forces, such as blade damage, scratches, corrosion, etc. Therefore, early and effective WTB surface defect detection is crucial to improve operational efficiency and ensure the safety of wind power generation systems.

[0003] Compared with traditional WTB defect detection methods such as visual inspection, infrared thermal imaging, ultrasonic and other non-destructive testing methods, the automatic detection method based on deep learning has the advantages of high efficiency, good real-time performance, strong real-time performance, high non-contact nature, and cost-effectiveness. In order to solve the problem of small target detection, the YOLO series of algorithms have attracted widespread attention due to their efficiency and real-time performance. As a representative single-stage detection algorithm, the core idea of ​​the YOLO series of target detection is to transform the core idea of ​​target detection into a regression problem, so as to achieve the simultaneous prediction of the target position and category, which has significant advantages in speed and real-time performance. The researchers have made adaptive innovations to the YOLO series and applied it to small target detection in WTB, achieving good results. For example, in a detection algorithm using YOLOv8 (Lizhao Liu, Pinrui Li, Dahan Wang, and Shunzhi Zhu. A wind turbine damage detection algorithm designed based on yolov8. Applied Soft Computing, 154: 111364, 2024), the backbone is replaced with the C2f-FocalNextBlock module, and the ResNet-EMA module and the thin neck structure are introduced in the neck part. The experimental results show that the average accuracy of the algorithm in detecting wind turbine damage is 79.9%, indicating that the algorithm has high accuracy and applicability in an environment with limited computing resources. But it also has the following challenges:

[0004] (1) Existing models have limited selective attention to input features, especially when dealing with complex backgrounds and small object defects in WTB.

[0005] (2) Insufficient ability to handle low-quality images in WTB detection. Summary of the invention

[0006] In order to overcome the shortcomings of the above-mentioned prior art, the object of the present invention is to provide a small target detection method for wind turbine blades based on YOLOv9 to improve the detection accuracy of small targets on turbine blades and is suitable for low-quality image data.

[0007] In order to achieve the above object, the technical solution adopted by the present invention is:

[0008] A small target defect detection method for wind turbine blades based on YOLOv9 comprises the following steps:

[0009] Step 1, obtaining a wind turbine blade defect image dataset;

[0010] Step 2, constructing a small target defect detection model; the small target defect detection model is based on YOLOv9, adopts a large selective kernel attention mechanism to dynamically adjust the receiving field, and replaces the original downsampling module in YOLOv9 with the SPD-Conv downsampling module, and replaces the original upsampling module in YOLOv9 with the CARAF upsampling module;

[0011] Step 3: Use the data set of step 1 to train the small target defect detection model of step 2, and use the trained small target defect detection model to perform small target defect detection on wind turbine blades.

[0012] In one embodiment, in step 1, images in a wind turbine blade defect image dataset are collected by shooting, and the defects in the images include corrosion, cracks, hidden cracks, surface attachments, surface damage, surface oil stains, and lightning strike marks; the collected images are cleaned, resized, enhanced, standardized, and label encoded to obtain a wind turbine blade defect image dataset.

[0013] In one embodiment, the large selective kernel (LSK) attention mechanism is combined with the feature extraction module, denoted as RepNCSPELAN4_LSK, and arranged in the Backbone and Neck parts of the model to enhance the detection capability of small objects by selectively focusing on key features.

[0014] In one embodiment, for an input feature map F, the large selective kernel attention mechanism produces the following output:

[0015]

[0016] A C =W 1×1 *Z C

[0017]

[0018] in, represents the feature map after the convolution operation, C represents the input channel, d represents the expansion rate, represents the convolution kernel of the Cth channel, with a size of (2d-1)×1, F C represents the Cth channel of the input feature map F, Z C represents the feature map after the convolution operation, k represents the size of the convolution kernel, Represents the convolution kernel of the Cth channel, with a size of A C represents the feature map after 1×1 convolution operation, W 1×1 represents a 1×1 convolution kernel, represents the feature map after Hadamard product, H and W represent the height and width of the input feature map F, * and Represent convolution and Hadamard product respectively.

[0019] In one embodiment, the RepNCSPELAN4_LSK performs the following process:

[0020] For the input feature map, feature extraction is first performed through a convolution layer, and the extracted features are divided into two paths. One path is input into the Rep module 1 to extract and process features, and then further extracted and processed by the Rep module 2; the other path is input into the LSK module for feature extraction, and the outputs of the Rep module 1, Rep module 2 and LSK module are spliced ​​together through the Concat operation, and finally passed through a convolution layer to obtain the output feature map; wherein the Rep module 1 and the Rep module 2 both contain multiple convolution layers.

[0021] In one embodiment, the input LSK module performs feature extraction, and the process is as follows:

[0022] For the input feature map, a convolution layer is first used to extract features, and the extracted features are divided into two paths. One path passes through a spatial convolution layer and a convolution layer in sequence for two feature extractions, and the other path passes through a convolution layer for one feature extraction. The features obtained from the two paths are spliced ​​together through the Concat operation, and then processed by average pooling and maximum pooling respectively, and then spliced ​​together again through the Concat operation. After that, they are normalized, dimensionally expanded with the outputs of the two feature extractions and the one feature extraction, and then passed through a convolution layer, and then merged with the input feature map of the LSK module as the output of the LSK module.

[0023] In one embodiment, the SPD-Conv downsampling module considers both local and global feature maps, and its downsampling method is as follows:

[0024] Given an intermediate feature map of dimension S×S×C1, after convolution operation, it is sliced ​​into the following subsequences:

[0025] f 0,0 =F[0:S:scale,0:S:scale],f 1,0 =F[1:S:scale,0:S:scale],...,

[0026] f 0,1 =F[0:S:scale,0:S:scale],f 1,1 =F[1:S:scale,0:S:scale],..., ...

[0028] f scale,scale =F[scale-1:S:scale,scale-1:S:scale],...,

[0029] Among them, scale represents the scale of the convolution operation, f scale,scale It means taking the last possible slice in both height and width directions, that is, starting from scale-1 and ending at S in each direction, with a step length of scale; the first subscript scale-1 means taking a slice every scale units from scale-1 to S (including scale-1) in the height direction; the shape of each subsequence is (S / scale, S / scale, C1), and double downsampling is performed. The obtained subsequences are fused, and then passed through a convolution-free step layer C2 with a step length of 2 to fuse the feature maps of different scales together to obtain an output with a shape of (S / scale, S / scale, C2).

[0030] In one embodiment, scale=2 is taken, and the intermediate feature map is divided into four sub-sequences, namely odd rows, odd columns, even rows and even columns.

[0031] In one embodiment, the CARAFE upsampling module considers both local and global feature maps, and its upsampling method is as follows:

[0032] Kernel prediction: convolution is performed on the input feature map to dynamically generate a convolution kernel that adapts to the input features;

[0033] Upsampling process: According to the upsampling size of the target, the low-resolution features that each high-resolution pixel should correspond to are calculated and position mapping is performed;

[0034] Content-aware reconstruction: Using the predicted convolution kernel, low-resolution feature regions are weighted and reconstructed to generate high-resolution features.

[0035] In one embodiment, given an input feature map of dimension C×H×W and integer upsampling rate σ, the CARAFE upsampling module generates an output feature map of dimension C×σH×σW, where each position l′=(I′,J′) in the output feature map corresponds to a source position l=(I,J) in the input feature map, expressed as:

[0036]

[0037] N(F l ,k) represents a k×k subregion of the input feature map centered at the source location l, i.e., region F l Neighborhood, combined with the l The convolution kernel W of each position l′ is predicted using the kernel prediction module ψ based on the contextual information of the neighborhood. l′ , as shown below:

[0038] W l′ =ψ(N(F l ,k encoder ))

[0039] Using content-aware reorganization module The region F is calculated by the following formula l To reorganize:

[0040]

[0041] The kernel prediction module generates the reconstructed kernel in a content-aware manner. Each position in the input feature map corresponds to a σ2 position in the output feature map. For each target position, a kernel of size k is required. up ×k up The reorganization kernel of up is the kernel size, k encoder is the convolution kernel size used for encoding.

[0042] Compared with the prior art, the detection accuracy of the small target of the turbine blade is improved by the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 The figure is a schematic diagram of the overall process of performing small target defect detection on wind turbine blades based on LSC-YOLO in the present invention.

[0044] Figure 2 It is a schematic diagram of the structure of the RepNCSPELAN4_LSK module of the present invention.

[0045] Figure 3 Schematic diagram of the structure of the SPD-Conv module of the present invention.

[0046] Figure 4 is the confusion matrix of YOLOv9-c.

[0047] Figure 5 It is the confusion matrix of the model of the present invention. DETAILED DESCRIPTION

[0048] The embodiments of the present invention are described in detail below with reference to the accompanying drawings and examples.

[0049] The present invention proposes a small target defect detection method for wind turbine blades based on YOLOv9, obtains a wind turbine blade defect image dataset and constructs a small target defect detection model for small target defect detection of wind turbine blades, trains the small target defect detection model using the dataset, and uses the trained small target defect detection model to perform small target defect detection on wind turbine blades.

[0050] The small target defect detection model of the present invention is based on YOLOv9, and for the convenience of description, it is named LSC-YOLO. Different from the classic YOLOv9, firstly, LSC-YOLO adopts a large selective kernel (LSK) attention mechanism to dynamically adjust the receiving field, and more effectively solves the difference in background information required for small target detection to alleviate the first problem described in the background technology. Then, the SPD-Conv downsampling module and the CARAF upsampling module are used to replace the original downsampling and upsampling modules respectively to alleviate the second problem described in the background technology.

[0051] The overall process of the present invention using the model to detect small target defects in wind turbine blades is as follows: Figure 1 As shown in the figure. The synergistic combination of LSK attention mechanism, SPD-Conv downsampling and CARAFE upsampling techniques exerts mutually reinforcing collective advantages. LSK attention mechanism enhances the detection capability of small objects by selectively focusing on key features. SPD-Conv downsampling improves detail preservation by integrating local and global feature maps. CARAFE upsampling enhances the reconstruction of fine-grained information. By combining these three techniques, the present invention successfully solves the problem of difficult detection of small defects on wind turbine blades, improves detection precision and recall, and shows superior overall performance compared with any single technique.

[0052] The specific aspects of the present invention will be described in detail below.

[0053] In the dataset construction phase of the present invention, images in the wind turbine blade defect image dataset can be obtained by shooting and collecting, or the existing image dataset can be directly used. In any case, the samples in the image should cover various common types of defects, such as corrosion, cracks, hidden cracks, surface attachments, surface damage, surface oil stains, and lightning strike marks. If the data is collected, necessary preprocessing is required, mainly including:

[0054] Data collection and import: Import the required image files in the image dataset into an appropriate environment, such as through a folder structure or database.

[0055] Data Cleansing: Check image integrity: Make sure all image files are not corrupted and that the target images exist.

[0056] Remove irrelevant images: Delete images that are irrelevant to wind blade defects and retain the classes that need to be analyzed.

[0057] Image resizing: Resizing: Unify the image size, for example, resize to 640×640 for input into the deep learning model.

[0058] Crop / Padding: For images of different sizes, you can choose to crop or pad to make them the same size.

[0059] Data augmentation: Use data augmentation techniques to increase the diversity of the data set. Common methods include: image rotation, translation, mirror flipping, random cropping, etc.

[0060] Image normalization: Normalize the image pixel values, usually converting the pixel values ​​from the range of 0-255 to between 0-1, or normalizing the mean and standard deviation to make the model converge faster.

[0061] Label encoding: Encode the labels of defect categories, such as using one-hot encoding, to ensure that the model can identify different defect types.

[0062] Dataset partitioning: Divide the dataset into training set, validation set, and test set, with a typical ratio of 7:2:1.

[0063] Visual inspection: After applying the preprocessing steps, check the quality of the dataset and the effect of preprocessing through visualization (such as showing some images and labels).

[0064] The present invention combines the large selective kernel attention mechanism with the feature extraction module, denoted as RepNCSPELAN4_LSK. By integrating the attention mechanism LSK into the feature extraction module, key information can be extracted more effectively. This integrated wind turbine blade defect representation in complex backgrounds is outstanding, providing a wide receptive field and enhanced detection capability for small target defect detection in complex environments, thus highlighting the advantages of this network in wind turbine blade defect detection.

[0065] Among them, RepN refers to Reparameterized, which is a reparameterization technology used to reduce the number of parameters and computation of the model. CSP refers to Cross Stage Partial Network, which is a technology used to reduce the amount of computation by sharing convolutional layers between different stages of the network. RepNCSP is the original YOLOv9 feature extraction module. ELAN refers to Efficient Layerwise Attention Network, which is an attention mechanism used to improve the performance of deep networks. It enhances feature representation by introducing attention mechanisms within and between layers. 4 indicates that this is the fourth version or variant in the series. RepNCSPELAN4_LSK is arranged in the Backbone and Neck parts of the model to enhance the detection ability of small objects by selectively focusing on key features.

[0066] The large selective kernel (LSK) attention mechanism of the present invention is derived from the basic large kernel attention (LKA) block, which involves a large convolution kernel in a two-dimensional depthwise convolution. For an input feature map of input channel C, height H and width W, its output can be expressed as:

[0067]

[0068] A C =W 1×1 *Z C

[0069]

[0070] where * and Denote convolution and Hadamard product respectively, and k×k represents the size of the convolution kernel. Decomposing the two-dimensional weight kernel of the depth and dilation convolution into two independent one-dimensional weight kernels, an enhanced LKA structure is obtained, namely the LSK module, which produces the following output:

[0071]

[0072] A C =W 1×1 *ZC

[0073]

[0074] in, Represents the feature map after the convolution operation, C represents the input channel, and d represents the dilation rate, which is used in the dilated convolution to control the interval covered by the convolution kernel. represents the convolution kernel (filter) of the Cth channel, with a size of (2d-1)×1, where 1 represents the size of the convolution kernel in the width direction and 2d-1 represents the size in the height direction. C represents the Cth channel of the input feature map F, z C represents the feature map after the convolution operation, k represents the size of the convolution kernel, Represents the convolution kernel of the Cth channel, with a size of A represents the actual convolution kernel size in the height direction, taking into account the expansion rate; C represents the feature map after 1×1 convolution operation, W 1×1 Represents a 1×1 convolution kernel, which is used to integrate information between channels. Represents the feature map after Hadamard product (element product), that is, A C With F C The element-wise product of , H and W represent the height and width of the input feature map F, * and Represent convolution and Hadamard product respectively.

[0075] The RepNCSPELAN4_LSK module process of the present invention is as follows Figure 2 As shown. For the input feature map, a convolution layer is first used to extract features, and the extracted features are divided into two paths. One path is input to the Rep module 1 to extract and process features, and then to the Rep module 2 to further extract and process features; the other path is input to the LSK module for feature extraction, and the outputs of the Rep module 1, the Rep module 2 and the LSK module are spliced ​​together through the Concat operation, and finally the output feature map is obtained after passing through a convolution layer; wherein the Rep module 1 and the Rep module 2 both contain multiple convolution layers.

[0076] The process of feature extraction for a single LSK module is as follows:

[0077] The input feature map is divided into two paths. One path is defined as attn. attn directly copies the original feature map and is completely consistent with the input feature map. The other path first passes through a convolution layer to extract features. The extracted features are also divided into two paths. One path of feature extraction passes through a spatial convolution layer and a convolution layer to extract features twice, which is defined as attn2. Spatial convolution is used to reduce spatial redundancy between features. The other path of feature extraction passes through a convolution layer to extract features once, which is defined as attn1. The features attn1 and attn2 obtained from the two paths are spliced ​​together through the Concat operation, and then processed by average pooling and maximum pooling respectively, and then spliced ​​together again through the Concat operation. After normalization, they are dimensionally expanded with the features attn1 and attn2 obtained from the two paths. Dimensional expansion is equivalent to increasing the area of ​​the features for better recognition. After the expanded features pass through a convolution layer, they are merged with one path of the input feature map of the LSK module, namely attn2, as the output of the LSK module.

[0078] The core idea of ​​the SPD-Conv Downsampling Module is to downsample local and global feature maps, that is, it considers both local and global feature maps at the same time. The implementation method of downsampling is as follows:

[0079] Given an intermediate feature map of dimension S×S×C1, after convolution operation, it is sliced ​​into subsequences as follows:

[0080] f 0,0 =F[0:S:scale,0:S:scale],f 1,0 =F[1:S:scale,0:S:scale],...,

[0081] f 0,1 =F[0:S:scale,0:S:scale],f 1,1 =F[1:S:scale,0:S:scale],..., ...

[0083] f scale-1,scale =F[scale-1:S:scale,scale-1:S:scale],...,

[0084] Among them, scale represents the scale of the convolution operation, f scale-1,scaleIndicates that the last possible slice is taken in both the height and width directions, that is, in each direction, it starts from scale-1 and ends at S, with a step length of scale; the first subscript scale-1 means that in the height direction, from scale-1 to S (including scale-1), a slice is taken every scale units. The second subscript scale-1 means that in the width direction, from scale-1 to S (including scale-1), a slice is taken every scale units.

[0085] The shape of each subsequence is (S / scale, S / scale, C1), and double downsampling is performed, that is, each dimension (length and width) is downsampled, and the resulting subsequences are fused, and then passed through a non-convolutional step layer C2 with a step size of 2 to obtain an output of shape (S / scale, S / scale, C2). The C2 layer is used to fuse feature maps of different scales together instead of changing the size of the feature map through convolution operations.

[0086] In the embodiment of the present invention, the scale is 2, and the intermediate feature map is divided into four subsequences, namely odd rows, odd columns, even rows, and even columns. The shape of each subsequence is (S / 2, S / 2, C1), and double downsampling is performed. The complete SPD-Conv flow chart schematic design is as follows Figure 3 As shown, the spatial to depth conversion of the input feature map is completed, as described below:

[0087] Input feature map: A multi-channel feature map that receives input, usually from the output of the previous layer.

[0088] Spatial downsampling: Reduce the width and height of the feature map by downsampling in the spatial dimension using methods such as maximum pooling or average pooling. The purpose of this step is to reduce computational complexity and retain important features.

[0089] Channel convolution operation: Depthwise Convolution is applied to the downsampled feature map. This process performs convolution independently on each input channel to reduce the number of parameters and computational burden.

[0090] Feature fusion: Combine spatial information and channel information, and further fuse features through pointwise convolution to enhance feature expression capabilities.

[0091] Activation function: Apply activation function to introduce nonlinearity to improve the learning ability of the model.

[0092] The CARAFE Upsampling Module takes into account both local and global feature maps, and its implementation is as follows:

[0093] Given an input feature map of dimension C×H×W and integer upsampling rate σ, the upsampling module generates an output feature map of dimension C×σH×σW, where each position l′=(I′,J′) in the output feature map corresponds to the source position l=(I,J) in the input feature map, expressed as:

[0094]

[0095] In this case, N(F l ,k) represents a k×k subregion of the input feature map centered at the source location l, i.e., region F l Neighborhood of . , combined with l The convolution kernel W of each position l′ is predicted using the kernel prediction module ψ based on the contextual information of the neighborhood. l′ , as shown below:

[0096] W l′ =ψ(N(F l ,k encoder ))

[0097] Using content-aware reorganization module The region F is calculated by the following formula l To reorganize:

[0098]

[0099] Content-aware reorganization module Use the predicted convolution kernel W l′ Adjust F l Nearby layout.

[0100] The kernel prediction module generates the reconstructed kernel in a content-aware manner. Each position in the input feature map corresponds to a σ2 position in the output feature map, i.e., for each position in the output feature map, it is mapped from the corresponding position in the input feature map by an upsampling rate σ. For each target position, a k-th position is required. up ×k up The reorganization kernel of up is the kernel size. Therefore, this module outputs a content-aware reordering kernel that adjusts the layout around the input feature map.

[0101] k encoder Refers to the convolution kernel size used for encoding, which determines the neighborhood size of the input feature map considered by the kernel prediction module when predicting the convolution kernel.

[0102] Corresponding to the downsampling module, the process of the CARAFE upsampling module of the present invention can be described as follows:

[0103] Input feature map: Receives the low-resolution feature map from the convolutional layer.

[0104] Kernel prediction: Convolution is performed on the input feature map to extract the features used to generate the convolution kernel, and the convolution kernel adapted to the input features is dynamically generated;

[0105] Upsampling process: According to the upsampling size of the target, the low-resolution features that each high-resolution pixel should correspond to are calculated and position mapping is performed;

[0106] Content-aware reconstructing: Using the predicted convolution kernel, low-resolution features are reconstructed into high-resolution feature maps. Specifically, the kernel is used to perform a weighted sum of appropriate low-resolution feature regions to generate high-resolution features.

[0107] Output feature map: The final output is a high-resolution feature map after content-aware reconstruction.

[0108] The present invention effectively solves the problem of poor image quality by replacing the original downsampling module with an improved SPDConv and an upsampling module with a CARAFE upsampling module. Specifically, SPD-Conv cleverly converts the spatial information of the image into depth information, solving problems such as small target size and blurred features. At the same time, CARAFE uses the content of the input features to guide the upsampling process, promoting accurate and efficient feature reconstruction, thereby building an excellent network.

[0109] To verify the superior performance of the method, detection experiments were conducted on the WTB dataset. Tables 1 and 2 show the evaluation results of the model of the present invention. The WTB dataset is a proprietary dataset covering wind turbine generators in the Ningxia region, totaling about 12,000 images. Techniques such as random flipping and translation have been used to increase the dataset to about 20,000 images. The dataset contains nine types of blade defects, including damage, scratches, corrosion, etc., which comprehensively covers potential blade problems in wind turbine generators. The target size distribution is concentrated within 5%, which meets the definition of small target size.

[0110] The present invention selected the previous YOLO series target detection algorithm for comparative experiments, but the single-stage detection algorithm is not limited to the YOLO series, and the present invention also compares these models. It can be seen that the present invention has achieved the best results in both prediction accuracy and error, showing higher accuracy than the existing methods.

[0111] Table 1 Evaluation results of all baseline models and the model proposed in this paper

[0112]

[0113] Table 2

[0114]

[0115] As can be seen from Tables 1 and 2, the present invention is not only superior to other YOLO algorithms, but also superior to other single-stage target detection algorithms. Compared with other models, the model of the present invention achieved an accuracy of 94.66%, a recall rate of 93.88%, a mAP_0.5 of 93.40, a multi-threshold average precision (mAP_0.5:0.95) of 74.33%, and an F 1-confidence of 0.94. The accuracy was improved by at least 2.9%, indicating that the detection was more accurate and the number of false positives was reduced. In terms of recall, the model of the present invention was improved by at least 2.8%, showing a more comprehensive detection capability of all actual positive samples and significantly reducing the number of missed detection instances. At the same time, compared with the YOLOv9 baseline, the model of the present invention showed improvements in various evaluation indicators: mAP_0.5 increased by 2.2%, mAP_0.5 increased by 3.8%:0.95, and F 1-confidence increased by 0.3. These enhancements demonstrate significant improvements in multi-threshold accuracy, indicating that the improved model of the present invention exhibits better robustness and generalization capabilities. Figure 4 and Figure 5 The displayed confusion matrix also demonstrates the overall superiority of the present invention.

[0116] The comparison of experimental data verifies that the network of the present invention is more accurate and more effective in identification than the existing network, and optimizes the missed detection problem existing in other networks.

[0117] In summary, the proposed method optimizes the YOLOv9 model by integrating the LSK attention mechanism. Experimental results show that the proposed method outperforms the original YOLOv9 model and other leading object detection models in various indicators. These results show the excellent performance and robustness of the model in detecting defects in wind turbine blades.

Claims

1. A small target defect detection method for wind turbine blades based on YOLOv9, characterized in that: The steps include: Step 1, obtaining a wind turbine blade defect image dataset; Step 2, constructing a small target defect detection model; the small target defect detection model is based on YOLOv9, adopts a large selective kernel attention mechanism to dynamically adjust the receiving field, and replaces the original downsampling module in YOLOv9 with the SPD-Conv downsampling module, and replaces the original upsampling module in YOLOv9 with the CARAF upsampling module; Step 3: Use the data set of step 1 to train the small target defect detection model of step 2, and use the trained small target defect detection model to perform small target defect detection on wind turbine blades.

2. The small target detection method for wind turbine blades based on YOLOv9 according to claim 1 is characterized in that: In the step 1, the images in the wind turbine blade defect image dataset are collected by shooting, and the defects in the images include corrosion, cracks, hidden cracks, surface attachments, surface damage, surface oil stains and lightning strike marks; the collected images are cleaned, resized, enhanced, standardized and labeled to obtain the wind turbine blade defect image dataset.

3. The small target detection method for wind turbine blades based on YOLOv9 according to claim 1, characterized in that: The large selective kernel (LSK) attention mechanism is combined with the feature extraction module, denoted as RepNCSPELAN4_LSK, and arranged in the Backbone and Neck parts of the model to enhance the detection ability of small objects by selectively focusing on key features.

4. The small target detection method for wind turbine blades based on YOLOv9 according to claim 1, characterized in that: For the input feature map F, the large selective kernel attention mechanism produces the following output: AND C =In 1×1 *WITH C in, represents the feature map after the convolution operation, C represents the input channel, d represents the expansion rate, represents the convolution kernel of the Cth channel, with a size of (2d-1)×1, F C represents the Cth channel of the input feature map F, Z C represents the feature map after the convolution operation, k represents the size of the convolution kernel, Represents the convolution kernel of the Cth channel, with a size of A C represents the feature map after 1×1 convolution operation, W 1×1 represents a 1×1 convolution kernel, represents the feature map after Hadamard product, H and W represent the height and width of the input feature map F, * and ° represent convolution and Hadamard product respectively.

5. The small target detection method for wind turbine blades based on YOLOv9 according to claim 3 or 4, characterized in that: The RepNCSPELAN4_LSK performs the following process: For the input feature map, feature extraction is first performed through a convolution layer, and the extracted features are divided into two paths. One path is input into the Rep module 1 to extract and process features, and then further extracted and processed by the Rep module 2; the other path is input into the LSK module for feature extraction, and the outputs of the Rep module 1, Rep module 2 and LSK module are spliced ​​together through the Concat operation, and finally passed through a convolution layer to obtain the output feature map; wherein the Rep module 1 and the Rep module 2 both contain multiple convolution layers.

6. The small target detection method for wind turbine blades based on YOLOv9 according to claim 1, characterized in that: The input LSK module performs feature extraction, and the process is as follows: For the input feature map, a convolution layer is first used to extract features, and the extracted features are divided into two paths. One path passes through a spatial convolution layer and a convolution layer in sequence for two feature extractions, and the other path passes through a convolution layer for one feature extraction. The features obtained from the two paths are spliced ​​together through the Concat operation, and then processed by average pooling and maximum pooling respectively, and then spliced ​​together again through the Concat operation. After that, they are normalized, dimensionally expanded with the outputs of the two feature extractions and the one feature extraction, and then passed through a convolution layer, and then merged with the input feature map of the LSK module as the output of the LSK module.

7. The small target detection method for wind turbine blades based on YOLOv9 according to claim 1, characterized in that: The SPD-Conv downsampling module considers both local and global feature maps, and its downsampling method is as follows: Given an intermediate feature map of dimension S×S×C1, after convolution operation, it is sliced ​​into the following subsequences: f 0,0 =F[0:S:scale,0:S:scale],f 1,0 =F[1:S:scale,0:S:scale],…, f 0,1 =F[0:S:scale,0:S:scale],f 1,1 =F[1:S:scale,0:S:scale],…, … f scale-1,scale-1 =F[scale-1:S:scale,scale-1:S:scale],…, Among them, scale represents the scale of the convolution operation, f scale-1,scale-1 It means taking the last possible slice in both height and width directions, that is, starting from scale-1 and ending at S in each direction, with a step length of scale; the first subscript scale-1 means taking a slice every scale units from scale-1 to S (including scale-1) in the height direction; the shape of each subsequence is (S / scale, S / scale, C1), and double downsampling is performed. The obtained subsequences are fused, and then passed through a convolution-free step layer C2 with a step length of 2 to fuse the feature maps of different scales together to obtain an output with a shape of (S / scale, S / scale, C2).

8. The small target detection method for wind turbine blades based on YOLOv9 according to claim 7, characterized in that: If scale = 2, the intermediate feature map is divided into four subsequences, namely odd rows, odd columns, even rows and even columns.

9. The method for detecting small targets of wind turbine blades based on YOLOv9 according to claim 1, characterized in that: The CARAFE upsampling module considers both local and global feature maps, and its upsampling method is as follows: Kernel prediction: convolution is performed on the input feature map to dynamically generate a convolution kernel that adapts to the input features; Upsampling process: According to the upsampling size of the target, the low-resolution features that each high-resolution pixel should correspond to are calculated and position mapping is performed; Content-aware reconstruction: Using the predicted convolution kernel, low-resolution feature regions are weighted and reconstructed to generate high-resolution features.

10. The small target detection method for wind turbine blades based on YOLOv9 according to claim 9, characterized in that: Given an input feature map of dimension C×H×W and integer upsampling rate σ, the CARAFE upsampling module generates an output feature map of dimension C×σH×σW, where each position l′=(I′,J′) in the output feature map corresponds to the source position l=(I,J) in the input feature map, expressed as: N(F l ,k) represents a k×k size sub-region of the input feature map centered at the source location l, i.e., region F l Neighborhood, combined with the l The convolution kernel W of each position l′ is predicted using the kernel prediction module ψ based on the contextual information of the neighborhood. l′ , as shown below: W l′ =ψ(N(F l ,k encoder )) Using content-aware reorganization module The region F is calculated by the following formula l To reorganize: The kernel prediction module generates the reconstructed kernel in a content-aware manner. Each position in the input feature map corresponds to a σ2 position in the output feature map. For each target position, a kernel of size k is required. up ×k up The reorganization kernel of up is the kernel size, k encoder is the convolution kernel size used for encoding.

Images