A fan blade damage identification method based on a faster R-SU model

Abstract

The application discloses a fan blade damage identification method based on a Faster R-SU model, belongs to the field of intelligent nondestructive testing of wind power equipment, and comprises the following steps: S1, collecting a visible light damage image of a fan blade surface; S2, constructing a Faster R-SU model integrated with detection, segmentation and identification; S3, dividing the fan blade damage data set into a training set and a test set; and S4, inputting a visible light image of a fan blade to be identified into the trained Faster R-SU model. Through the synergistic effect of an efficient channel attention mechanism, a progressive feature pyramid and a normalized Wasserstein distance loss function, the small target detection problem is solved in a targeted manner, wherein the efficient channel attention module adopts a local cross-channel interaction design without dimension reduction, and the feature sensitivity of the network to micro defects is improved under low computational complexity.

Description

Technical Field

[0001] This invention belongs to the field of intelligent non-destructive testing of wind power equipment, specifically involving a method for identifying wind turbine blade damage based on the Faster R-SU model. Background Technology

[0002] As one of the core forms of renewable energy utilization, wind power has seen continuous expansion in installed capacity, playing a crucial role in energy structure transformation and green, low-carbon economic development. Wind turbine blades are the core energy-capturing components of wind turbine generators, exposed to complex outdoor environments for extended periods. They are susceptible to damage from multiple factors, including lightning strikes, wind and sand erosion, hail impacts, and alternating loads, gradually leading to various forms of damage such as cracks, delamination, leading-edge erosion, coating peeling, and even structural fractures. These damages are often small and hidden in their early stages; if not detected and repaired in time, they will continue to expand over time, causing localized stress concentrations, weakening the blade's structural integrity, reducing power generation efficiency, and potentially triggering major safety accidents such as blade breakage, significantly increasing the overall lifecycle operation and maintenance costs of wind farms.

[0003] However, existing wind turbine blade damage identification technologies still have significant shortcomings: most current studies adopt a phased processing paradigm, optimizing target detection and image segmentation as independent tasks, lacking an end-to-end integrated joint optimization mechanism, which easily leads to error accumulation, feature redundancy, and semantic inconsistency, making it difficult to meet the needs of real-world scenarios for highly robust diagnostic systems. Summary of the Invention

[0004] The purpose of this invention is to provide a wind turbine blade damage identification method based on the Faster R-SU model, so as to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a method for identifying wind turbine blade damage based on the Faster R-SU model, comprising the following steps: S1. Collect visible light damage images of the wind turbine blade surface, label the location, category and segmentation mask of the damage area in the image, construct a wind turbine blade damage dataset, and perform standardization preprocessing on the images in the dataset. S2. Construct a Faster R-SU model that integrates detection, segmentation and recognition. The model uses Faster R-CNN as the detection backbone network and embeds a Swin-Unet network as a pixel-level segmentation branch. The model also integrates a small target enhancement module, an edge-guided attention mechanism and a structure perception module in sequence. S3. Divide the wind turbine blade damage dataset into a training set and a test set. Use the training set to iteratively train the Faster R-SU model. Optimize the model network parameters through backpropagation of the joint loss function until the model converges. S4. Input the visible light image of the wind turbine blade to be identified into the trained Faster R-SU model. After detection and localization, fine segmentation and feature enhancement recognition, the model outputs the bounding box of the damaged target, the pixel-level segmentation mask and the damage classification result.

[0006] 2. The wind turbine blade damage identification method based on the Faster R-SU model according to claim 1, characterized in that: the small target enhancement module is set between the backbone network and the region candidate network of Faster R-CNN, and consists of an efficient channel attention module, a progressive feature pyramid structure, and a normalized Wasserstein distance loss function, wherein the calculation process of the normalized Wasserstein distance loss function is as follows: For the center coordinates are , , width is Gao Wei The horizontal bounding box is mapped to a four-dimensional feature vector. This is used to characterize the mean and standard deviation parameters of the corresponding Gaussian distribution; For the Gaussian distributions corresponding to the two bounding boxes respectively and The square of its second-order Wassstein distance is:

[0007] In the formula, , These are the bounding boxes. The center x and y coordinates, , These are the bounding boxes. Width and height, , These are the bounding boxes. The center x and y coordinates, , These are the bounding boxes. Width and height, The 2-norm of a vector is used to normalize the Wasserstein distance, resulting in the normalized Wasserstein distance metric:

[0008] In the formula, Let be the square of the second-order Wasserstein distance between the two bounding boxes corresponding to the Gaussian distribution. The normalization constant is related to the dataset and takes the value of the average size of all damaged targets in the dataset. The normalized Wasserstein distance loss function is:

[0009] In the formula, This represents the normalized Wasserstein distance loss value.

[0010] In a preferred embodiment, the edge-guided attention mechanism is integrated into the skip connection path between the encoder and decoder of the Swin-Unet network, and consists of a hollow spatial pyramid pooling module, a spatial attention mechanism, and a boundary loss function; wherein, the calculation process of the boundary loss function is as follows: Generate an edge mask from the real labels using an edge detection operator. If pixels ( () represents the boundary between the target and the background. ,in A weight coefficient greater than 1, if pixel ( If ) is a non-edge region, then 1; The single-pixel weighted cross-entropy loss is:

[0011] In the formula, For pixels ( The true label for () is a value of 0 for background and 1 for target. The probability that the pixel is the target is predicted by the model, and the value range is [0,1]. The full-image boundary loss is the average of the losses of all pixels:

[0012] In the formula, For feature map height, The width of the feature map. This represents the boundary loss value.

[0013] In a preferred embodiment, the structure-aware module is located at the output end of the segmentation branch and is composed of a convolutional block attention module and a context guidance module connected in series. The convolutional block attention module sequentially performs weighted calibration on the input features through channel attention and spatial attention; the context guidance module aggregates multi-scale contextual information and enhances the feature representation of the damaged area through parallel local feature extraction, surrounding context extraction, edge feature optimization and global feature extraction branches.

[0014] As a preferred implementation, the data flow of the Faster R-SU model is as follows: the input image is processed by the Faster R-CNN backbone network to extract features, which are then optimized by the small target enhancement module. A damage candidate region is generated by the region candidate network, and after the feature size is unified by the RoI pooling layer, the initial detection box and preliminary classification result of the damage target are output. The region of interest corresponding to the initial detection box is cropped and input into the Swin-Unet segmentation branch. The segmentation accuracy is optimized by the edge-guided attention mechanism, and a pixel-level segmentation mask of the damage is output. The segmentation mask and the feature map of the corresponding region are input into the structure perception module, and after feature enhancement, the fine classification and recognition of the damage are completed.

[0015] As a preferred implementation, the high-efficiency channel attention module adopts a dimensionless local cross-channel interaction structure, which generates channel attention weights through one-dimensional convolution. The size of the convolution kernel is adaptively determined according to the number of input channels, and the attention weight of each channel is jointly determined by its k neighboring channels, where k is the convolution kernel size.

[0016] As a preferred implementation, the progressive feature pyramid adopts a progressive multi-stage fusion mechanism, starting from the deepest feature layer and gradually fusing it with adjacent shallow features, and then passing the fusion result to shallower layers; the fusion process adopts an adaptive spatial weighting strategy, dynamically learning the fusion weights of different levels of features based on the content of each spatial location.

[0017] As a preferred implementation, the dilated spatial pyramid pooling module applies multiple dilated convolutions with different dilation rates in parallel on the input feature map, while setting a 1×1 convolution branch and a global average pooling branch. After capturing multi-scale contextual information, the outputs of each branch are spliced ​​and fused in the channel dimension without changing the spatial resolution of the feature map.

[0018] As a preferred implementation, the calculation process of the spatial attention mechanism is as follows: global average pooling and global max pooling are performed on the input feature map along the channel dimension to obtain two single-channel feature maps, which are then concatenated along the channel dimension; a spatial attention weight matrix is ​​generated by 7×7 convolution and sigmoid activation function, and the weight matrix is ​​multiplied element-wise with the input feature map to obtain a spatially weighted feature map.

[0019] In a preferred embodiment, the joint loss function in step S3 is composed of a weighted sum of the detection branch loss and the segmentation branch loss, wherein the detection branch loss includes the classification loss and the normalized Wasserstein distance loss, and the segmentation branch loss includes the cross-entropy loss and the boundary loss.

[0020] Compared with the prior art, the beneficial effects of the present invention are: The small target enhancement module of this invention addresses the challenge of small target detection through the synergistic effect of an efficient channel attention mechanism, a progressive feature pyramid, and a normalized Wasserstein distance loss function. Specifically, the efficient channel attention module employs a dimensionality-reduction-free local cross-channel interaction design, enhancing the network's sensitivity to minute defects with low computational complexity. The progressive feature pyramid efficiently injects deep semantic information into shallow detail features through multi-stage progressive fusion and adaptive spatial weighting strategies, strengthening the feature representation of small targets. The normalized Wasserstein distance loss function uses a Gaussian distribution to measure bounding box similarity, overcoming the shortcomings of traditional IoU loss, which exhibits drastic fluctuations and poor regression robustness in small target scenarios. These designs collectively improve the detection recall and localization accuracy of small-scale damage such as pinholes and microcracks on wind turbine blade surfaces, effectively mitigating the problem of missed detections caused by insufficient feature representation of small targets.

[0021] The edge-guided attention mechanism constructed in this invention specifically optimizes the blurry edges in wind turbine blade damage images. It embeds a hollow spatial pyramid pooling module into the skip connection path of the Swin-Unet encoder and decoder, expanding the receptive field without reducing feature resolution and enhancing the model's ability to perceive multi-scale context. A spatial attention mechanism is embedded at the output of each skip connection level to adaptively strengthen the feature response of boundary regions and refine the feature representation of damage edges. Combined with a boundary loss function, weighted supervision is applied to edge pixels to optimize the fit of segmentation boundaries. This mechanism effectively improves the boundary continuity and accuracy of pixel-level damage segmentation masks, providing more reliable contour information for subsequent damage quantification assessment.

[0022] The structural perception module of this invention enhances the feature representation capability of the damaged area through the cascaded cooperation of the convolutional block attention module and the context guidance module. Specifically, the convolutional block attention module dynamically calibrates the feature response by serially weighting channel attention and spatial attention, thereby improving the discriminativeness of damage features. The context guidance module extracts local features, surrounding context, edge features, and global semantic information in parallel, realizing the effective aggregation of multi-scale context and modeling global semantic dependencies. This module can significantly suppress the interference of complex backgrounds such as blade surface texture, illumination changes, and environmental debris, reduce false detections, and improve the overall accuracy of wind turbine blade damage classification and identification. Attached Figure Description

[0023] Figure 1 This is a diagram of the Faster R-CNN network architecture of this invention; Figure 2 This is a diagram of the Swin-Unet network architecture of the present invention. Figure 3 This is a structural diagram of the context guidance module of the present invention. Figure 4This is a diagram of the Faster R-SU network structure of the present invention; Figure 5 This is a model prediction graph of the present invention; Figure 6 This is another model prediction diagram of the present invention. Detailed Implementation

[0024] The present invention will be further described below with reference to embodiments.

[0025] The following embodiments are used to illustrate the present invention, but should not be used to limit the scope of protection of the present invention. The conditions in the embodiments can be further adjusted according to specific conditions, and simple improvements to the method of the present invention under the premise of the concept of the present invention are all within the scope of protection claimed by the present invention.

[0026] Please see Figures 1-6 This invention provides a wind turbine blade damage identification method based on the Faster R-SU model, which constructs a small target enhancement module.

[0027] First, we propose introducing an efficient channel attention (ECA) mechanism between the Faster R-CNN backbone network and the RPN in the Faster R-SU model to improve the model's sensitivity to minor defects. Second, we propose constructing an Asymptotic Feature Pyramid Network (AFPN) structure to fuse feature maps from different levels of the feature extraction network, strengthening the feature representation corresponding to small targets and thus alleviating the problem of insufficient feature representation for small-scale targets. Finally, we propose introducing a Normalized Wasserstein Distance (NWD) loss function, which measures the difference by calculating the Wasserstein distance between the predicted box and the ground truth box according to the Gaussian distribution, significantly improving the robustness of bounding box regression to small targets.

[0028] To address the issue of blurred edges in wind turbine blade damage images, this invention proposes an edge-guided attention mechanism. First, an Atrous Spatial Pyramid Pooling (ASPP) module is introduced into the skip connection path between the Swin-Unet encoder and decoder in the Faster R-SU model to enhance the model's ability to perceive multi-scale features. Second, a spatial attention mechanism is embedded at the output of each skip connection to further refine the feature representation of boundary regions. Finally, in the design of the model's loss function, a boundary loss function is introduced. This loss function is specifically optimized for segmenting boundary regions, and by weighted supervision of the difference between the predicted results and the true labels in the edge regions, the network's learning ability for boundary pixels is strengthened.

[0029] To address the issue of insufficient damage representation in wind turbine blades, this invention proposes to construct a structure-aware module. First, a Convolutional Block Attention Module (CBAM) is proposed to be introduced into the Faster R-SU model. This module enhances the discriminative power of feature representations through a concatenated channel attention and spatial attention mechanism. Second, a Context Guided Module (CGM) is proposed to be embedded into the Faster R-SU model. By aggregating multi-scale contextual information, this module effectively models global semantic dependencies, enhances the feature representation capability of the wind turbine blade damage area, and thus suppresses the interference of complex backgrounds on the target.

[0030] Specifically, this includes the Faster R-CNN algorithm and the Swin-Unet algorithm. The Faster R-CNN network structure consists of four parts: a feature extraction network, a region candidate network, RoI pooling layers, and a classification and regression network. The Faster R-CNN network structure is as follows: Figure 1 As shown.

[0031] Faster R-CNN's feature extraction network is primarily used to extract high-level semantic features from input images. Commonly used backbone networks include VGG, ResNet, and Xception. The Region Proposal Network (RPN) enhances feature representation through 3×3 convolutional layers and then splits into two parallel branches: a classification branch outputs features through 1×1 convolutions, which are then reshaped and softmaxed to classify anchor boxes as foreground or background; the regression branch outputs features through 1×1 convolutions to predict the coordinate offset of anchor boxes. Subsequently, the RoI pooling layer converts the feature map region corresponding to each candidate box into a fixed-size (e.g., 7×7) feature representation through max pooling, resolving the issue of inconsistent input dimensions in subsequent networks. Finally, the classification and regression networks, based on the fixed-dimensional features output by the RoI pooling, perform accurate target category classification and secondary refinement of the detection box coordinates through two parallel fully connected layer branches, respectively, outputting the final high-precision detection result.

[0032] The Swin-Unet algorithm consists of four parts: an encoder, a bottleneck layer, a decoder, and skip connections. The Swin-Unet network structure is as follows: Figure 2 As shown.

[0033] The encoder in the Swin-Unet network architecture mainly consists of a Linear Embedding module, a Swin Transformer module, and a Patch Merging module. The Linear Embedding module projects the feature dimension to an arbitrary dimension. The Swin Transformer module is used for learning feature representations to capture local and global contextual features.

[0034] The Patch Merging module is used for downsampling and feature dimension expansion to achieve more efficient multi-level feature extraction.

[0035] The bottleneck layer of the Swin-Unet network architecture is located between the encoder and decoder, typically consisting of two consecutive SwinTransformer Blocks. This layer further refines and aggregates feature information at the highest semantic level without altering the feature map's resolution or channel dimension. The decoder in the Swin-Unet network architecture mainly comprises a Swin Transformer module and a Patch Expanding module. The Patch Expanding module upsamples low-resolution, high-channel-count feature maps to high-resolution, low-channel-count feature maps, achieving a 2x improvement in spatial resolution.

[0036] In the final Patch Expanding module, 4* upsampling is performed to restore the feature map resolution to the input image resolution (W*H). The skip connections in the Swin-Unet network structure directly concatenate the multi-scale shallow feature maps output from each stage of the encoder with the upsampled deep feature maps from the corresponding stages in the decoder. This effectively compensates for the spatial and detail information lost during layer-by-layer downsampling, helping the model recover more accurate and detailed segmentation boundaries.

[0037] The small target enhancement module consists of an efficient channel attention (ECA) module, an asymptotic feature pyramid network (AFPN) structure, and a normalized Wasserstein distance (NWD) loss function.

[0038] (1) High-efficiency channel attention module The Squeeze-and-Excitation (SE) module is a lightweight channel attention module. Its core idea is to dynamically recalibrate channel feature responses by explicitly modeling the dependencies between feature channels in a convolutional neural network. This module comprises three consecutive operations: First, global average pooling is performed on each channel of the input feature map, compressing its spatial dimension into a scalar to aggregate global contextual information. Second, a sub-network containing two fully connected layers learns the non-linear dependencies between channels: the first layer uses the ReLU activation function to introduce non-linearity, and the second layer uses the Sigmoid activation function, thus generating a weight between 0 and 1 for each channel. Finally, the resulting weights are multiplied channel-by-channel with the original feature map, adaptively enhancing important features and suppressing redundant features.

[0039] The ECA (Efficient Channel Attention) module proposes an efficient channel attention mechanism based on the SE (Squeeze-and-Excitation) module. Through dimensionality reduction-free design and local cross-channel interaction, it alleviates the information loss and computational redundancy problems of the SE module. Specifically, ECA abandons the structure of SE, which uses two fully connected layers for dimensionality reduction followed by dimensionality increase. Instead, it uses a single unbiased one-dimensional convolutional layer to directly generate channel weights. This convolutional kernel has a size of k and contains only k learnable parameters, ensuring that the attention weight of each channel is determined only by its k neighboring channels, rather than all channels. Here, k is adaptively determined based on the number of input channels, thus significantly reducing model complexity while maintaining performance.

[0040] (2) Progressive feature pyramid The Asymptotic Feature Pyramid Network (AFPN) is an improvement on the Feature Pyramid Network (FPN).

[0041] FPN is a multi-scale feature extraction architecture for object detection. Its core idea is to utilize the multi-scale feature maps generated by the forward propagation of the backbone network. Through a top-down path, it progressively upsamples high-level semantically rich but low-resolution features and fuses them with corresponding shallow, high-resolution features via lateral connections, thereby constructing a multi-scale feature pyramid. Each layer of this pyramid can be used for subsequent predictions, significantly improving the detection performance of objects at different scales with almost no increase in computational overhead.

[0042] Compared to FPN, AFPN's core improvement lies in introducing a progressive multi-stage fusion mechanism: starting with the deepest features, it gradually fuses them with adjacent shallow features, and then propagates the fusion result to even shallower layers. This allows deep semantic information to be more effectively injected into shallow detail features, reducing information loss during long-distance transmission. Simultaneously, AFPN employs an adaptive spatial weighted fusion strategy, replacing the simple element-wise addition in FPN. This enables the network to dynamically learn the fusion weights of different feature levels based on the content of each spatial location, mitigating information conflicts caused by differences in receptive fields. Furthermore, AFPN performs multiple iterative fusions during feature pyramid construction, rather than a single fusion after feature extraction, ensuring that each layer's output features incorporate multi-level contextual information. These design elements significantly enhance the detection capability for small targets while maintaining computational efficiency.

[0043] (3) Normalized Wasserstein distance loss function Currently, the mainstream bounding box regression loss function in the field of object detection is the IoU loss function. Its calculation is based on the intersection-over-union (IoU) ratio of the predicted bounding box A and the ground truth bounding box B, defined as the ratio of the area of ​​their intersection to the area of ​​their union.

[0044] Where the numerator represents the intersection area of ​​the two boxes, and the denominator represents the union area of ​​the two boxes. The range of IoU is [0, 1], with IoU = 1 when the two boxes completely overlap and IoU = 0 when they do not overlap at all. The IoU loss is defined as:

[0045] The loss range is [0,1]. The higher the overlap, the smaller the loss; when there is no overlap, the loss = 1. However, for small object detection tasks, even the original IoU loss faces challenges: a small shift can cause the IoU value to fluctuate drastically or even drop to zero.

[0046] To address the aforementioned issues, this paper employs the Normalized Wasserstein Distance (NWD) loss function. The NWD loss function is a bounding box similarity metric specifically designed to solve the challenge of small object detection. Its core idea is to treat the geometric bounding box as a probability distribution, thereby utilizing optimal transport theory for measurement.

[0047] First, the bounding boxes are modeled as a two-dimensional Gaussian distribution, that is, each horizontal bounding box... , (where is The center coordinates (w, h are the width and height) are transformed into a two-dimensional Gaussian distribution. The specific conversion formula is as follows:

[0048] In the formula, It is the mean vector (corresponding to the center of the box). It is a diagonal covariance matrix, with its diagonal elements being the squares of the half-width and half-height of the bounding box, respectively. Through this parameterization method, each bounding box is uniquely modeled as a two-dimensional Gaussian distribution.

[0049] Then, the two bounding boxes and Modeled as Gaussian distributions respectively and The similarity between them can be measured by the Wasserstein distance between the two distributions. In optimal transport theory, the second-order Wasserstein distance between two two-dimensional Gaussian distributions has a concise closed-form solution. For two bounding boxes... and Its second-order Wasserstein distance is:

[0050] Subsequently, normalization is performed to obtain the NWD metric. The Wasserstein distance is a distance metric (smaller values ​​indicate greater similarity), while IoU is a similarity metric (larger values ​​indicate greater similarity). To facilitate substitution for IoU, it needs to be normalized to the [0,1] interval and converted to a similarity form. The normalized NWD is defined as:

[0051] In the formula, C is a constant related to the dataset, typically set as the average size of all targets in the dataset. Through exponential normalization, the NWD range is (0,1], where NWD=1 when two boxes completely overlap, and approaches 0 as the distance increases. Finally, the NWD loss is calculated.

[0052] The edge-guided attention mechanism consists of an Atrous Spatial Pyramid Pooling (ASPP) module, a Spatial Attention Mechanism (SAM), and a Boundary Loss function.

[0053] (1) Hollow space pyramid pooling module Atrous Spatial Pyramid Pooling (ASPP) is an improvement on traditional multi-scale context modeling methods. This module applies multiple dilated convolutions with different dilation rates in parallel on the input feature map, while introducing 1×1 convolution branches and global average pooling branches. This allows each branch to capture contextual information at different scales, and finally, the outputs of all branches are concatenated and fused along the channel dimension. Unlike traditional methods that rely on pooling operations to expand the receptive field, ASPP effectively expands the receptive field using dilated convolutions without reducing the feature map resolution, thus ensuring that the output feature map maintains the same spatial size as the input, fully preserving spatial location information, which is beneficial for accurate boundary localization.

[0054] (2) Spatial attention mechanism Spatial Attention Mechanism (SAM) is a module that allows neural networks to automatically focus on important regions while suppressing irrelevant backgrounds when processing images.

[0055] First, a channel-wise global max pooling and global average pooling are performed on an input feature map F of size H×W×C, resulting in two feature maps of size H×W×1. Then, the results of the global max pooling and global average pooling are concatenated by channel to obtain a feature map of size H×W×2. Finally, a 7×7 convolution operation is performed on the concatenated result to obtain a feature map of size H×W×1, which is then activated by the Sigmoid activation function to obtain the spatial attention weight matrix. Output of the spatial attention module It can be calculated using the following formula:

[0056] in, This represents a 7x7 convolution operation. [AvgPool(F); MaxPool(F)] means concatenating the results of average pooling and max pooling along the channel axis.

[0057] (3) Boundary loss function Boundary loss is a loss function specifically designed for image segmentation tasks. Its core design idea is to measure the degree of fit between the predicted segmentation region boundary and the ground truth labeled boundary, rather than evaluating the pixel overlap of the entire region.

[0058] First, from the true label (Y, In the context of (0 = background, 1 = target), an "edge mask" M is generated using edge detection operators (such as Canny, Sobel) or morphological operations. If pixel ( If ) is the true edge (the boundary between the target and the background), then =w (w>1, the weighting coefficient is usually taken as 3-5); If pixel (i,j) is not an edge, then =1 (basic weight).

[0059] Then, the weighted cross-entropy loss is calculated. Let the model's prediction probability for pixel (i,j) be... ( ∈[0,1], representing the probability that the pixel is the target, then the single-pixel weighted cross-entropy loss is:

[0060] Structure-aware module: The structure-aware module consists of a Convolutional Block Attention Module (CBAM) and a Context Guided Module (CGM).

[0061] (1) Convolutional block attention module The Convolutional Block Attention Module (CBAM) is a lightweight attention mechanism module that includes a channel attention module (C-channel) and a spatial attention module (S-channel). These two modules can be embedded in different layers of a CNN to enhance feature representation.

[0062] The output of the channel attention module Specifically:

[0063] In the formula, F is the input feature map, AvgPool and MaxPool represent global average pooling and max pooling operations, respectively, and MLP represents a multilayer perceptron. This represents the Sigmoid activation function. The output of the spatial attention module. Specifically:

[0064] In the formula, This represents a 7x7 convolution operation. `[AvgPool(F); MaxPool(F)]` concatenates the results of average pooling and max pooling along the channel axes. The final result is the spatial attention weight matrix. ; (2) Contextual guidance module The Context Guided Module (CGM) consists of a local feature extractor. Surrounding context extractor Edge feature optimizer and global feature extractor Composition, such as Figure 3 As shown, specifically, given an input feature map, the context guidance module uses parallel operations to obtain feature maps with different contextual information from the initial features, where the surrounding context extractor... By analyzing the initial features Figure 3 Obtained by a dilated convolution of ×3; Local feature extractor The edge feature optimizer is obtained by standard 3×3 convolution of the initial feature map. It is obtained through the Edge Feature Generation Module (EFG). Then, it is combined with... and The fusion is performed by adding elements one by one, and the characteristics of the fused product are then compared with... The outputs are concatenated along the channel dimension, then processed sequentially through batch normalization and PReLU activation functions before being fed into the global feature extractor. middle. Global average pooling is applied to the input features, followed by two fully connected layers to extract global contextual representations. Finally, the concatenated feature map is combined with the global feature extractor. The final result is obtained by multiplying the output element by element.

[0065] Faster R-SU model The Faster R-SU model structure is as follows: Figure 4As shown in the figure. This paper innovatively embeds the Swin-Unet model into the Faster R-CNN model to construct the Faster R-SU model, which accurately segments the targets in the detection boxes to improve recognition accuracy. First, an efficient channel attention mechanism and AFPN are introduced into Faster R-SU to improve the recall rate of small-scale damaged targets. Second, the image region (RoI) corresponding to the detected bounding box is used as the input of Swin-Unet in the Faster R-SU model, and ASPP and spatial attention mechanisms are embedded in its backbone network to generate a high-precision pixel-level damage segmentation mask. Finally, based on the segmentation mask results, a convolutional block attention mechanism and a context guidance module are introduced into the Faster R-SU model to enhance the discriminative feature representation of the damaged region, thereby improving the overall recognition performance.

[0066] Comparative Experiment: This comparative experiment compares the Faster R-CNN model, the YOLOv8 model, the YOLOv10 model, the YOLOv11 model, the YOLOv12 model, and the YOLOv26 model.

[0067] The dataset contains 3071 images of wind turbine blade damage. It includes two damage types: scratches and pinholes.

[0068] Model prediction: For the Faster R-CNN, YOLOv8, YOLOv10, YOLOv11, YOLOv12, and YOLOv26 models, three images were selected for prediction, and the results are as follows. Figure 5 and Figure 6 As shown.

[0069] The problems of the six models in wind turbine blade damage identification mainly include two aspects: First, the size of the damaged area of ​​the wind turbine blade is small and easily submerged by the surrounding background noise, resulting in the missed detection of some damage to the wind turbine blade, or although it is detected, the confidence score is very low; Second, because the large area and high response background area tend to dominate the attention allocation operation, it leads to the false detection of some damage to the wind turbine blade.

[0070] Model evaluation: In object detection tasks, commonly used metrics for evaluating model performance include precision, recall, and mAP (mean average precision).

[0071] Precision is the proportion of samples correctly predicted as positive (i.e., detected as targets) by the model. It measures the "accuracy" or "reliability" of the model's predictions, reflecting the degree to which the model produces false positives. The formula is:

[0072] TP (True Positive): The number of targets correctly detected by the model.

[0073] FP (False Positive): The number of false positives that the model incorrectly detects (i.e., false detections, such as detecting background as an object).

[0074] Recall is the proportion of true positive class (i.e., real target) samples that the model successfully predicts correctly. It measures the "comprehensiveness" of the model's target finding, reflecting the extent to which the model misses detections (false negatives). The formula is:

[0075] TP (True Positive): The number of targets correctly detected by the model.

[0076] FN (False Negative): False negatives. The number of real targets that were not detected by the model (i.e., missed detections).

[0077] mAP is the average AP (Average Precision) across all categories, and it is the most crucial comprehensive evaluation metric in object detection.

[0078] mAP@0.5: mAP calculated when the Intersection over Union (IoU) threshold between the predicted bounding box and the ground truth bounding box is 0.5.

[0079] mAP@0.5:0.95: Calculates mAP at multiple IoU thresholds (from 0.5 to 0.95, with a step size of 0.05) and then takes the average value, which places stricter requirements on the model's positioning accuracy.

[0080] The evaluated precision rates of the Faster R-CNN, YOLOv8, YOLOv10, YOLOv11, YOLOv12, and YOLOv26 models were 51.39%, 71.2%, 60.6%, 72.9%, 61%, and 63.7%, respectively. The YOLOv11 model had a higher evaluation precision, while the Faster R-CNN model had a lower evaluation precision. The evaluated recall rates of the Faster R-CNN, YOLOv8, YOLOv10, YOLOv11, YOLOv12, and YOLOv26 models were 66.43%, 55.9%, 45%, 56.2%, 53.1%, and 57.2%, respectively. Again, the Faster R-CNN model had a higher evaluation recall, while the YOLOv10 model had a lower evaluation recall.

[0081] The mAP (mAP) of the Faster R-CNN, YOLOv8, YOLOv10, YOLOv11, YOLOv12, and YOLOv26 models are 45.3%, 61.4%, 48.8%, 60.5%, 52.3%, and 60.9%, respectively. The YOLOv8 model has a higher mAP, while the Faster R-CNN model has a lower mAP. The mAP50-95 of the Faster R-CNN, YOLOv8, YOLOv10, YOLOv11, YOLOv12, and YOLOv26 models are 16.81%, 30.6%, 22.9%, 30.8%, 24.4%, and 31.7%, respectively. The YOLOv26 model has a higher average mAP50-95, while the Faster R-CNN model has a lower average mAP50-95. An overall evaluation of the Faster R-CNN, Yolov8, Yolov10, Yolov11, Yolov12, and Yolov26 models shows that the Yolov8 model performs better, while the Faster R-CNN model performs worse. The corresponding results are shown in the table below.

[0082] ; Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for identifying wind turbine blade damage based on the Faster R-SU model, characterized in that, Includes the following steps: S1. Collect visible light damage images of the wind turbine blade surface, label the location, category and segmentation mask of the damage area in the image, construct a wind turbine blade damage dataset, and perform standardization preprocessing on the images in the dataset. S2. Construct a Faster R-SU model that integrates detection, segmentation and recognition. The model uses Faster R-CNN as the detection backbone network and embeds a Swin-Unet network as a pixel-level segmentation branch. The model also integrates a small target enhancement module, an edge-guided attention mechanism and a structure perception module in sequence. S3. Divide the wind turbine blade damage dataset into a training set and a test set. Use the training set to iteratively train the Faster R-SU model. Optimize the model network parameters through backpropagation of the joint loss function until the model converges. S4. Input the visible light image of the wind turbine blade to be identified into the trained Faster R-SU model. After detection and localization, fine segmentation and feature enhancement recognition, the model outputs the bounding box of the damaged target, the pixel-level segmentation mask and the damage classification result.

2. The wind turbine blade damage identification method based on the Faster R-SU model according to claim 1, characterized in that: The small target enhancement module is located between the backbone network and the region candidate network of Faster R-CNN. It consists of an efficient channel attention module, a progressive feature pyramid structure, and a normalized Wasserstein distance loss function. The calculation process of the normalized Wasserstein distance loss function is as follows: For the center coordinates are , , width is Gao Wei The horizontal bounding box is mapped to a four-dimensional feature vector. This is used to characterize the mean and standard deviation parameters of the corresponding Gaussian distribution; For the Gaussian distributions corresponding to the two bounding boxes respectively and The square of its second-order Wassstein distance is: In the formula, , These are the bounding boxes. The center x and y coordinates, , These are the bounding boxes. Width and height, , These are the bounding boxes. The center x and y coordinates, , These are the bounding boxes. Width and height, The 2-norm of a vector is used to normalize the Wasserstein distance, resulting in the normalized Wasserstein distance metric: In the formula, Let be the square of the second-order Wasserstein distance between the two bounding boxes corresponding to the Gaussian distribution. The normalization constant is related to the dataset and takes the value of the average size of all damaged targets in the dataset. The normalized Wasserstein distance loss function is: In the formula, This represents the normalized Wasserstein distance loss value.

3. The wind turbine blade damage identification method based on the Faster R-SU model according to claim 1, characterized in that: The edge-guided attention mechanism is integrated into the skip connection path between the encoder and decoder of the Swin-Unet network, and consists of a hollow spatial pyramid pooling module, a spatial attention mechanism, and a boundary loss function; wherein, the calculation process of the boundary loss function is as follows: Generate an edge mask from the real labels using an edge detection operator. If pixels ( () represents the boundary between the target and the background. ,in A weight coefficient greater than 1, if pixel ( If ) is a non-edge region, then 1; The single-pixel weighted cross-entropy loss is: In the formula, For pixels ( The true label for () is a value of 0 for background and 1 for target. The probability that the pixel is the target is predicted by the model, and the value range is [0,1]. The full-image boundary loss is the average of the losses of all pixels: In the formula, For feature map height, The width of the feature map. This represents the boundary loss value.

4. The wind turbine blade damage identification method based on the Faster R-SU model according to claim 1, characterized in that: The structure awareness module is located at the output end of the segmentation branch and is composed of a convolutional block attention module and a context guidance module connected in series. The convolutional block attention module sequentially performs weighted calibration on the input features through channel attention and spatial attention; The context guidance module aggregates multi-scale contextual information through parallel branches of local feature extraction, surrounding context extraction, edge feature optimization, and global feature extraction, thereby enhancing the feature representation of the damaged area.

5. The wind turbine blade damage identification method based on the Faster R-SU model according to claim 1, characterized in that: The data flow of the Faster R-SU model is as follows: the input image is processed by the Faster R-CNN backbone network to extract features, which are then optimized by the small target enhancement module. A damage candidate region is generated by the region candidate network, and after the feature size is unified by the RoI pooling layer, the initial detection box and preliminary classification result of the damage target are output. The region of interest corresponding to the initial detection box is cropped and input into the Swin-Unet segmentation branch. The segmentation accuracy is optimized by the edge-guided attention mechanism, and a pixel-level segmentation mask of the damage is output. The segmentation mask and the feature map of the corresponding region are input into the structure perception module, and after feature enhancement, the fine classification and recognition of the damage are completed.

6. The wind turbine blade damage identification method based on the Faster R-SU model according to claim 1, characterized in that: The efficient channel attention module adopts a dimensionless local cross-channel interaction structure. It generates channel attention weights through one-dimensional convolution. The size of the convolution kernel is adaptively determined according to the number of input channels. The attention weight of each channel is jointly determined by its k neighboring channels, where k is the convolution kernel size.

7. The wind turbine blade damage identification method based on the Faster R-SU model according to claim 1, characterized in that: The progressive feature pyramid adopts a progressive multi-stage fusion mechanism, starting from the deepest feature layer and gradually fusing it with adjacent shallow features, and then passing the fusion result to shallower layers. The fusion process adopts an adaptive spatial weighting strategy, dynamically learning the fusion weights of different levels of features based on the content of each spatial location.

8. The wind turbine blade damage identification method based on the Faster R-SU model according to claim 1, characterized in that: The dilated spatial pyramid pooling module applies multiple dilated convolutions with different dilation rates in parallel on the input feature map. It also sets a 1×1 convolution branch and a global average pooling branch. After capturing multi-scale contextual information, the outputs of each branch are spliced ​​and fused in the channel dimension without changing the spatial resolution of the feature map.

9. The wind turbine blade damage identification method based on the Faster R-SU model according to claim 1, characterized in that: The calculation process of the spatial attention mechanism is as follows: global average pooling and global max pooling are performed on the input feature map in the channel dimension respectively, and then two single-channel feature maps are concatenated along the channel dimension. A spatial attention weight matrix is ​​generated by 7×7 convolution and sigmoid activation function. The weight matrix is ​​then multiplied element-wise with the input feature map to obtain the spatially weighted feature map.

10. The wind turbine blade damage identification method based on the Faster R-SU model according to claim 1, characterized in that: The joint loss function in step S3 is composed of a weighted sum of the detection branch loss and the segmentation branch loss, wherein the detection branch loss includes the classification loss and the normalized Wasserstein distance loss, and the segmentation branch loss includes the cross-entropy loss and the boundary loss.

Images