Method for detecting surface defects of blades of wind driven generator

Abstract

The invention discloses a method for detecting surface defects of a blade of a wind driven generator. The method comprises the following steps: acquiring a high-resolution image of the blade of the wind driven generator acquired by an unmanned aerial vehicle; the method comprises the steps of pre-constructing a neural network model based on a deep learning target detection framework, and processing an image; the neural network model comprises a backbone network, a neck network and a detection head; a double-attention mechanism module is embedded in the backbone network and the neck network and is formed by connecting a channel-space attention mechanism and a parallel attention mechanism in series; a multi-scale feature fusion and enhancement operator is introduced into the backbone network, and a detection head operator of an optimization model is adopted to process a feature map output by the neck network; and training the neural network model by using the data set, deploying the neural network model on an unmanned aerial vehicle computing platform after training is completed, and performing real-time defect detection and recognition on an input blade image. According to the method, high-precision recognition of fine and small target defects and high-efficiency operation can be met at the same time, and real-time, accurate and automatic defect detection of the blade image is realized.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and artificial intelligence, and more specifically to a method for detecting surface defects on wind turbine blades. Background Technology

[0002] Currently, with the continuous growth of global demand for clean energy, wind power has become an important pillar of the new energy sector. Wind turbines operate for extended periods in harsh natural environments such as high altitudes, where their core component—the blades—is constantly subjected to the erosion and challenges of wind, sand, rain, snow, lightning strikes, ultraviolet radiation, and enormous aerodynamic loads. Therefore, various defects inevitably appear on the blade surface, such as cracks, leading-edge corrosion, coating peeling, lightning damage, and pinholes. If these defects are not detected and repaired in a timely manner, they can range from affecting the aerodynamic efficiency of the turbine and causing a decrease in power generation, to potentially leading to structural damage or even breakage of the blades, resulting in major safety accidents and economic losses. Therefore, regular, efficient, and accurate defect detection of wind turbine blades is a crucial step in ensuring the safe and stable operation of wind farms and achieving cost reduction and efficiency improvement.

[0003] Traditional blade inspection methods primarily rely on manual labor. One method involves "Spider-Man" style high-altitude rope work, where specialized technicians are suspended from the blades for close-range visual inspection. While this method offers high accuracy, it has several drawbacks: First, it carries extremely high operational risks, posing a serious threat to personnel safety; second, it is inefficient and cannot meet the needs of large-scale inspections in large wind farms; third, the inspection results depend on personnel experience, are highly subjective, and are difficult to standardize and manage with data; and finally, it requires prolonged turbine shutdown, resulting in significant power generation losses. Another method involves using ground-based telescopes or high-magnification cameras for long-distance observation. While this method is safer, its accuracy is severely insufficient, making it difficult to detect early-stage micro-cracks or internal damage, and it is easily limited by weather conditions and observation angles.

[0004] To overcome the drawbacks of manual inspection, automated inspection technologies have emerged. Early attempts at automation were primarily based on traditional computer vision algorithms, such as edge detection (Canny and Sobel operators), thresholding, morphological processing, and feature descriptors like SIFT and SURF. These methods can identify some obvious defects under specific, ideal conditions. However, their robustness is extremely poor. In practical applications, the background of blade images is extremely complex and variable, including the sky, clouds, mountains, and other wind turbines, while lighting conditions, shooting angles, and shadows change drastically. Traditional algorithms are prone to misclassifying background textures or shadows as defects, or missing detections due to low contrast between defect features and the blade's base color. Furthermore, these algorithms typically require tedious manual feature design and parameter tuning for specific scenes, have weak generalization ability, and are difficult to adapt to complex real-world environments.

[0005] In recent years, deep learning technologies, represented by convolutional neural networks (CNNs), have achieved revolutionary breakthroughs in the field of image recognition, bringing new solutions for defect detection. Object detection models, such as the two-stage Faster R-CNN series and the single-stage YOLO and SSD series, can automatically learn deep features of defects from massive amounts of data, significantly improving detection accuracy and robustness. However, when these general object detection models are directly applied to wind turbine blade defect detection scenarios, certain limitations still exist:

[0006] 1) Significant scale differences and detection range: Wind turbine blades are enormous (up to hundreds of meters in size), while critical defects (such as early cracks) may only be a few centimeters in size. This results in defect targets occupying a very small proportion of pixels in a high-resolution panoramic image of the blade, creating a typical challenge for small target detection. General-purpose models have limited feature extraction capabilities for small targets, easily leading to missed detections;

[0007] 2) Weak and easily confused defect features: Many defects, especially fine cracks and early corrosion, have colors and textures that are not much different from the surface material of the blade itself, resulting in low feature contrast. In addition, problems such as fog and blurring may occur during image acquisition and transmission, further weakening the defect features and making them easily confused with stains, watermarks, etc. in the background, which places extremely high demands on the feature recognition ability of the model.

[0008] 3) Limitations of Real-Time Detection and Airborne Deployment: Using unmanned aerial vehicles (UAVs) for inspection is currently the most efficient solution. However, UAV platforms are typically equipped with embedded devices with limited computing power (such as the NVIDIA Jetson series) and are constrained by battery life. This requires the detection model to be lightweight, capable of minimizing computational resource consumption while ensuring detection speed (meeting real-time requirements). Traditional high-precision models (such as Faster R-CNN) are computationally intensive and difficult to deploy; while standard lightweight models (such as YOLO-tiny) are fast, their accuracy often fails to meet detection requirements.

[0009] Therefore, existing wind turbine blade inspection technologies, whether traditional manual methods, early computer vision algorithms, or general deep learning models, are all unable to perfectly solve the core contradictions of "large background, small target, weak features, and limited resources" in blade defect detection tasks. Summary of the Invention

[0010] The purpose of this invention is to overcome the shortcomings of existing technologies and provide a method for detecting surface defects on wind turbine blades. This method, through an innovative network structure, is optimized specifically for the characteristics of blade defects based on a lightweight design. It enhances the ability to represent features of weak and small-scale targets and suppresses interference from complex backgrounds, thereby achieving the best balance between accuracy and efficiency on UAV platforms with limited computing resources.

[0011] To achieve the above objectives, the present invention adopts the following technical solution:

[0012] This invention provides a method for detecting surface defects on wind turbine blades, comprising the following steps:

[0013] Acquire high-resolution images of wind turbine blades captured by drones, the images containing surface defects of the wind turbine blades as: defect targets with huge scale differences and weak features and complex natural background;

[0014] A neural network model based on a deep learning object detection framework is pre-constructed to process the image; the neural network model sequentially includes a backbone network, a neck network, and a detection head.

[0015] A dual attention mechanism module is embedded in the backbone network and / or neck network to adaptively refine the input feature map, thereby enhancing the model's perception of defect features and suppressing interference from complex backgrounds; the dual attention mechanism module is composed of a channel-space attention mechanism CSAM and a parallel attention mechanism PAM connected in series.

[0016] Multi-scale feature fusion and enhancement operators are introduced into the backbone network to learn the offset of the sampling points of the convolution kernel, so that the receptive field of the model can dynamically fit the irregular geometry and variable spatial position of the defects on the blade surface.

[0017] The detection head operator, which employs an optimized model, processes the feature map output from the neck network. The detection head operator includes a localization branch and a classification branch set in parallel, which are used to perform bounding box regression and category recognition tasks for the target, respectively.

[0018] The neural network model is trained using a dataset containing images of surface defects on wind turbine blades to optimize its learnable parameters.

[0019] The trained model is deployed on an unmanned aerial vehicle (UAV) computing platform to perform real-time defect detection and recognition on the input blade images.

[0020] Furthermore, the Channel-Spatial Attention (CSAM) mechanism processes the input feature map in the following manner:

[0021] The input feature map is fed into the parallel channel attention branch and spatial attention branch respectively;

[0022] The channel attention branch is processed sequentially through a global average pooling layer, a first linear layer, a first activation function, a second linear layer, and a second activation function, and then element-wise multiplied with the input feature map to generate channel attention weights.

[0023] The spatial attention branch is processed by a 7×7 convolutional layer and an activation function, and multiplied with the input feature map to generate spatial attention weights.

[0024] The channel attention weights and spatial attention weights are added together to obtain a preliminary refined feature map;

[0025] The preliminary refined feature map is added to the original input feature map through residual connection to obtain the output feature map of the CSAM module.

[0026] Furthermore, the Parallel Attention Mechanism (PAM) processes the input feature map in the following manner:

[0027] The input feature map is simultaneously input into the Global Channel Attention (GCA) submodule, the Local Channel Attention (LCA) submodule, and the Spatial Attention (SA) submodule;

[0028] The GCA submodule processes the data sequentially through a global average pooling layer and a 1×1 convolutional layer to extract cross-channel global information.

[0029] The LCA submodule is processed sequentially through a one-dimensional convolutional layer and a deep convolutional layer, preserving local channel interaction information;

[0030] The SA submodule is processed by a convolutional layer with a kernel size of 7×7 to preserve key spatial details.

[0031] The output feature maps of the three sub-modules GCA, LCA, and SA are concatenated along the channel dimension.

[0032] The concatenated feature map is passed sequentially through the first 1×1 convolutional layer, an activation function, and the second 1×1 convolutional layer, and then residually connected with the output feature map of the channel-spatial attention mechanism CSAM to obtain the final output feature map of the PAM module.

[0033] Furthermore, the multi-scale feature fusion and enhancement operator is a deformable convolution operator, and its execution process includes:

[0034] For the input feature map, an offset is learned through an additional convolutional layer, which predicts a two-dimensional offset vector for each sampling point in the standard convolutional grid.

[0035] Based on the offset vector, the fixed sampling position of the standard convolution is adaptively moved to the critical region of the target defect;

[0036] Feature values ​​are obtained at the non-integer coordinate positions after the shift using a bilinear interpolation algorithm;

[0037] The acquired feature values ​​are weighted and summed with the convolution kernel weights to complete the deformable convolution operation, thereby enabling feature extraction of irregular geometric defects.

[0038] Furthermore, an offset is learned through an additional convolutional layer, expressed by the following formula:

[0039] Δp m,n (i,j)=f(X)(i×s+mp,j×s+np)

[0040] Where, Δp m,n (i,j) is a two-dimensional vector representing the offset learned at position (i,j) for the sampling point at position (m,n) in the convolution kernel; f represents the convolution operation, and the learned offset can make the sampling point position adapt to the image content; X represents the input feature map; s represents the stride; p represents the padding amount.

[0041] Furthermore, the obtained feature values ​​are weighted and summed with the convolution kernel weights, as shown in the following formula:

[0042]

[0043] Where Y(i,j) represents the value of the output feature map at position (i,j); k represents the size of the convolution kernel; g(·) represents the bilinear interpolation operation, used to obtain feature values ​​at non-integer coordinate positions; W(m,n) represents the weight of the convolution kernel, corresponding to the weight at position (m,n) in the convolution kernel.

[0044] Furthermore, the localization branch and classification branch in the detection head operator of the optimized model are decoupled structures:

[0045] The localization branch consists of several stacked standard convolutional layers, used to predict the bounding box coordinates of the defective target; the localization loss is calculated as follows:

[0046]

[0047] in, L smoothL1 Calculate the loss of the predicted bounding box relative to the ground truth bounding box; (x i ,y i ,w i ,h i () represents the coordinates of the predicted bounding box. The coordinates of the ground truth bounding box; smooth L1 The (x) function represents the smoothed L1 loss, which measures the difference between the bounding box coordinates predicted by the model and the ground truth box.

[0048] The classification branch is composed of several depthwise separable convolutional layers and is used to predict the classification of defective targets; the comprehensive loss for localization is calculated.

[0049]

[0050] Where IOU is the intersection-union ratio between the predicted bounding box and the ground truth bounding box; ρ 2 (b,b gt ) represents the squared Euclidean distance between the center points of the predicted bounding box and the ground truth bounding box; c represents the diagonal length of the minimum closure region containing both boxes; and Used to measure aspect ratio differences; where w and h are the width and height of the predicted bounding box, w gt h gt These are the width and height of the actual bounding box, and arctan(·) is used to measure the difference in aspect ratio.

[0051] Furthermore, the deep learning object detection framework is the YOLOv11 framework.

[0052] As can be seen from the above technical solution, compared with the prior art, this invention discloses an automated and intelligent safety inspection and health assessment method for wind turbine blades. This method can ensure wind turbine safety, improve power generation efficiency, and reduce operation and maintenance costs while guaranteeing task completion.

[0053] The core of this invention lies in constructing a deep learning model that integrates three key innovative modules:

[0054] 1) A novel dual attention mechanism is disclosed: This invention utilizes advanced object detection-based algorithms to enhance the model's ability to correlate global features by combining channel attention and spatial attention, enabling more efficient location and identification of defect areas on wind turbine blades from complex backgrounds (such as fog).

[0055] 2) A multi-scale feature fusion and enhancement operator is disclosed: To address the problem of varying blade defect sizes, an operator is designed to effectively fuse and enhance feature maps at different levels in the network, thereby improving the model's ability to detect defects of different sizes and solving the problem of difficult feature extraction.

[0056] 3) Disclose an optimized detection head operator: This method is better adapted to the variability of the shape and scale of the defect target, and improves the flexibility and accuracy of detection.

[0057] Compared with existing industrial target detection methods, this invention proposes a lightweight model specifically optimized for wind turbine blade defect detection, achieving a better balance between accuracy and real-time performance. This method, through innovative feature enhancement and attention mechanisms, effectively addresses the core contradiction of general-purpose methods on resource-constrained platforms such as UAVs, which struggle to simultaneously achieve high-precision identification of minute, small target defects and high-efficiency operation.

[0058] This invention integrates the three innovative modules mentioned above into high-performance object detection frameworks such as YOLOv11, forming a proprietary, end-to-end training model. This model is trained on a wind turbine blade defect dataset and ultimately deployed on a drone platform to achieve real-time, accurate, and automated defect detection of blade images. Attached Figure Description

[0059] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0060] Figure 1 The present invention provides a flowchart of a method for detecting surface defects on wind turbine blades.

[0061] Figure 2 The overall framework diagram of the method for detecting surface defects of wind turbine blades provided by this invention.

[0062] Figure 3 The structural diagram of the dual attention mechanism provided by this invention.

[0063] Figure 4 The present invention provides three parallel attention mechanism structure diagrams.

[0064] Figure 5 The structure diagram of the detection head operator of the optimized model provided by this invention. Detailed Implementation

[0065] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0066] This invention is designed for the development of surface defect detection for wind turbine blades, particularly for real-time detection applications using unmanned aerial vehicles (UAVs). Through this invention, the detection of surface defects on wind turbine blades is achieved by analyzing the acquired blade images, enabling efficient and accurate detection of small, multi-scale defects against complex backgrounds.

[0067] This invention discloses a method for detecting surface defects on wind turbine blades, referring to... Figure 1 As shown, it includes the following steps S1 to S7:

[0068] S1. Acquire a high-resolution image of the wind turbine blades collected by the UAV. The image contains surface defects of the wind turbine blades, which are: defect targets with huge scale differences and weak features, and complex natural background.

[0069] S2. A neural network model based on a deep learning object detection framework is pre-constructed to process the image; the neural network model includes a backbone network, a neck network, and a detection head in sequence.

[0070] S3. Embed a dual attention mechanism module in the backbone network and / or neck network to adaptively refine the input feature map, thereby enhancing the model's perception of defect features and suppressing interference from complex backgrounds; the dual attention mechanism module is composed of a channel-space attention mechanism CSAM and a parallel attention mechanism PAM connected in series.

[0071] S4. Introduce multi-scale feature fusion and enhancement operators into the backbone network to learn the offset of the sampling points of the convolution kernel, so that the receptive field of the model can dynamically fit the irregular geometry and variable spatial position of the defects on the blade surface.

[0072] S5. The detection head operator of the optimized model processes the feature map output by the neck network. The detection head operator includes a localization branch and a classification branch set in parallel, which are used to perform bounding box regression and category recognition tasks of the target, respectively.

[0073] S6. The neural network model is trained using the acquired dataset containing images of surface defects on wind turbine blades to optimize its learnable parameters.

[0074] S7. Deploy the trained model on the UAV computing platform to perform real-time defect detection and recognition on the input blade images.

[0075] This method, through innovative network structure design, significantly improves the ability to extract features of small and weak defects and suppress complex backgrounds while maintaining a lightweight design, achieving high-precision real-time detection on embedded platforms such as UAVs.

[0076] The following is a detailed explanation of each of the above steps:

[0077] The first step is dataset construction: acquiring high-resolution images of wind turbine blades collected by drones, including defects with significant scale differences and subtle features, as well as complex natural backgrounds. This constructed dataset prepares for subsequent model training, with the ultimate goal of achieving real-time and accurate detection of minute, subtle, and irregular defects in large-scale, complex-background wind turbine blade images on a drone platform.

[0078] The second step is model framework construction. Addressing the challenges of large target scale differences, weak features, and complex backgrounds in wind turbine blade defect detection, this invention selects YOLOv11, which excels in both training efficiency and detection accuracy, as the baseline framework. Based on this, the final detection method architecture is constructed by integrating three core innovations—a novel dual-attention mechanism, a multi-scale feature fusion and enhancement operator, and an optimized detection head operator (as shown in the attached figure). Figure 2 (As shown). This includes: the backbone network, the neck network, and the head.

[0079] 1. The backbone network, as the core component of feature extraction, is typically composed of multiple layers of convolutional neural networks (such as Conv, DAM, MFFEM, and C2PSA modules shown in the figure). Its core function is to perform convolution and downsampling on the input image layer by layer, progressively extracting features from low-level edge and texture features to high-level semantic features.

[0080] 2. The neck network connects the backbone network and the detection head, and its core function is to perform multi-scale feature fusion. It integrates feature maps of different depths and scales from the backbone network through operations such as upsampling and concatenation.

[0081] 3. The head is the final output of the network, responsible for decoding the multi-scale feature maps sent from the neck network and completing the two core tasks of target localization (Bounding Box Regression) and classification.

[0082] The third step is global feature enhancement and background suppression. Because convolutional neural network-based methods tend to ignore crucial global information when processing images with complex context or fog components, thus reducing detection accuracy, this invention proposes a dual-attention mechanism combining multi-channel feature fusion and parallel attention. This innovation aims to utilize local and global contextual information to enhance the network framework's ability to perceive blade surface defects and improve its global correlation capabilities across channels and spaces. The specific steps are as follows:

[0083] A dual attention mechanism is embedded in the model's backbone and neck. First, this dual attention mechanism consists of a channel-spatial attention mechanism (CSAM) and a parallel attention mechanism (PAM) connected in series, adaptively refining the feature maps sequentially. Second, CSAM, by combining channel attention and spatial attention, reduces model complexity while more accurately detecting and suppressing interfering components such as fog in the image. Finally, PAM innovatively connects three mechanisms in parallel: global channel attention (GCA), local channel attention (LCA), and spatial attention (SA), effectively integrating cross-channel global information, local interaction information, and spatial detail information. This allows the model to comprehensively acquire global features of the leaf image, thereby achieving high-precision defect detection.

[0084] Because wind turbine blade images contain rich spatial and color information, it is difficult to accurately identify fog components. CSAM, while reducing model complexity, combines channel and spatial information from channel-spatial attention, enabling more accurate detection of fog components.

[0085] like Figure 3 As shown in the figure, (a) is the overall framework diagram of the dual attention mechanism, (b) is the structural diagram of the channel-space attention mechanism, and (c) is the structural diagram of the parallel attention mechanism. This innovation aims to address the neglect of key information caused by complex backgrounds (such as fog).

[0086] The specific execution process is as follows:

[0087] 1. Channel-Spatial Attention (CSAM) Mechanism: To achieve fast convergence and prevent overfitting, the input feature map X∈R... b ×m×n×c First, batch normalization is performed using CSAM. This mechanism extracts global channel information and extensive spatial information through parallel channel branches and spatial branches, respectively. The channel branch uses global average pooling and linear layers to capture inter-channel dependencies, while the spatial branch uses a 7x7 large convolutional kernel to capture spatial context. The combination of the two (as shown in Equation 1) enhances the ability to identify interfering components such as fog, and residual connections (as shown in Equation 2) preserve the original features.

[0088]

[0089] Here, GAP represents the GAP function, and Linear represents a linear layer. Furthermore, Sigmoid and GELU represent the Sigmoid and GELU activation functions, respectively. Conv represents element-wise multiplication. 7×7 This represents a convolution with a kernel size of 7; This represents the original input feature map after batch normalization.

[0090] X CSAM =Conv 1×1 (Mish(Conv 1×1 (X CSA )))+X (2)

[0091] Among them, Conv 1×1 denoted as 1×1 convolution, Mish represents the Mish activation function; X represents the original input feature map.

[0092] 2. Parallel Attention (PAM) Mechanism: The output of CSAM is then fed into the PAM module. This module connects three different attention mechanisms in parallel: Global Channel Attention (GCA), Local Channel Attention (LCA), and Spatial Attention (SA) (see attached). Figure 4 (as shown), Figure 4 (a) is a structural diagram of the Global Channel Attention (GCA) mechanism, (b) is a structural diagram of the Local Channel Attention (LCA) mechanism, and (c) is a structural diagram of the Spatial Attention (SA) mechanism.

[0093] GCA extracts cross-channel global information through global average pooling and 1x1 convolution (as shown in Equation 3); LCA preserves local channel interaction information through a combination of 1D convolution and depthwise convolution (as shown in Equation 4); SA preserves image details through 7x7 convolution (as shown in Equation 5). Finally, the outputs of the three are concatenated (as shown in Equation 6) and combined with a multilayer perceptron and residual connections (as shown in Equation 7) to effectively preserve image details and improve dehazing performance, thereby enhancing the model's adaptability to complex scenes.

[0094]

[0095] After compressing the information of each channel in GAP, GCA can extract global information across channels to reduce the loss of channel information.

[0096]

[0097] Among them, 1DConv and DWConv 3×3 These are 3×3 one-dimensional convolution and depthwise convolution, respectively.

[0098]

[0099] X c =concat(X) GCA ,X LCA ,X SA (6)

[0100] X PAM =Conv 1×1 (Mish(Conv 1×1 (X c )))+X CSAM (7)

[0101] The fourth step is multi-scale and adaptive extraction of deformation features. Because defects on wind turbine blades are irregular in shape and varied in location, the fixed receptive field of traditional convolution is insufficient for effectively extracting their features. To flexibly capture the geometric deformation and complex location of defects in images, this invention introduces a multi-scale feature fusion and enhancement operator into the backbone network. The core idea of ​​this operator is to learn an additional variable "offset" for each sampling point of the convolution kernel. Unlike traditional convolution which samples at fixed grid points, the multi-scale feature fusion and enhancement operator allows sampling points to move adaptively according to the image content, thus focusing on the actual location and irregular shape of blade defects. This dynamic receptive field enables the model to extract defect features with geometric deformation more accurately, significantly enhancing the model's feature extraction capability and generalization performance for multi-scale, diverse targets.

[0102] Specifically, a multi-scale feature fusion and enhancement operator is introduced into the backbone network of the model. This operator is used to flexibly capture the irregular geometry and variable location of defects. Its execution process is as follows:

[0103] 1. Offset Learning: For the input feature map X, an additional convolutional layer learns an offset Δp in parallel. m,n (As shown in Equation 8). This offset makes the sampling points p of the standard convolution... n It can adaptively move to a new position p based on local image features. n +Δp n .

[0104] 2. Feature Sampling and Convolution: At the learned offset positions, feature values ​​are obtained through bilinear interpolation, and then weighted and summed with the convolution kernel weights (as shown in Equation 9). In this way, the model can more accurately extract the features of irregularly shaped defects, improving its adaptability to geometric deformations.

[0105] Δp m,n (i,j)=f(X)(i×s+mp,j×s+np) (8)

[0106] Where, Δp m,n (i,j) is a two-dimensional vector representing the offset learned at position (i,j) for the sampling point at position (m,n) in the convolution kernel; f represents the convolution operation, and the learned offset allows the sampling point position to adapt to the image content; X represents the input feature map; s represents the stride; and p represents the padding amount.

[0107]

[0108] Where Y(i,j) represents the value of the output feature map at position (i,j); g(·) represents the bilinear interpolation operation; X represents the input feature map; W(m,n) represents the weight of the convolution kernel, corresponding to the weight at position (m,n) in the convolution kernel.

[0109] The fifth step is to decouple and optimize the target localization and classification tasks. Because blade surface defects are diverse and easily confused with the background, a single detection head cannot simultaneously achieve accurate localization and classification, easily leading to false alarms. To address this issue, this invention innovatively combines the detection head with the dynamic characteristics of deformable convolutional networks, proposing an optimized detection head operator with more efficient and flexible feature extraction capabilities.

[0110] This operator employs two parallel branches, focusing on the core tasks of target localization and classification, respectively. The localization branch utilizes stacked convolutions for bounding box regression, aiming to accurately pinpoint defect locations; while the classification branch uses depthwise separable convolutional combinations to identify defect categories, accurately distinguishing defects from the background. This decoupling of localization and classification tasks enhances the detection head's feature extraction capabilities for specific tasks and optimizes the model's adaptability to changes in target shape and scale, thereby improving overall detection performance.

[0111] Specifically, in the detection head part of the model, the detection head operator of the optimized model proposed in this invention (as shown in the attached diagram) is used. Figure 5 (As shown). This operator aims to solve the false alarm problem caused by feature similarity and improve the comprehensive processing capabilities for target localization and classification. Its execution process is as follows:

[0112] 1. Localization Branch Processing: The input feature map first passes through a branch consisting of stacked ordinary convolutions (Conv), specifically for bounding box regression. Its output is used to calculate the localization loss (as shown in Equation 10), aiming to accurately determine the location and size of the defect.

[0113]

[0114] in, LsmoothL1 Calculate the loss of the predicted bounding box relative to the ground truth bounding box; (x i ,y i ,w i ,h i For each predicted bounding box coordinate, For each ground truth bounding box coordinate; smooth L1 The (x) function represents the smooth L1 loss, which is specifically designed to measure the difference between the bounding box coordinates predicted by the model and the ground truth box. It can avoid gradient explosion and converge more smoothly when approaching the optimum.

[0115] 2. Classification Branch Processing: Simultaneously, the input feature map also flows through another branch consisting of depthwise separable convolutions (DWConv and Conv), specifically for target classification. Its output is used to calculate the combined loss for classification and localization (as shown in Equation 12), aiming to accurately distinguish defect categories and suppress background interference. By decoupling the localization and classification tasks, this detection head operator enhances the feature extraction capability of the detection head and its adaptability to changes in target shape and scale, thereby optimizing the final detection performance.

[0116]

[0117] Where IOU is the intersection-union ratio between the predicted bounding box and the ground truth bounding box; ρ 2 (b,b gt ) represents the squared Euclidean distance between the center points of the predicted bounding box and the ground truth bounding box; c represents the diagonal length of the minimum closure region containing both boxes; and Used to measure aspect ratio differences. Where w and h are the width and height of the predicted bounding box, w... gt h gt These are the width and height of the actual bounding box, and arctan(·) is used to measure the difference in aspect ratio.

[0118] Step 6: Model Training and Defect Detection. The constructed method framework is trained end-to-end on a common wind turbine blade surface defect detection dataset. The model, through backpropagation, simultaneously optimizes all learnable parameters in the network based on localization and classification losses, including convolutional kernel weights in each module and offsets in the variable convolutional network. After training, the method is deployed on platforms such as drones to perform efficient and accurate real-time defect detection on new blade images.

[0119] The technical problem solved by this invention is how to efficiently and accurately detect surface defects of varying sizes and subtle features on wind turbine blades on a UAV platform with limited computing resources, while effectively suppressing interference from complex backgrounds (such as fog). Its technical advantages mainly lie in the methodological architecture design and algorithmic innovations tailored to this specific scenario, including:

[0120] 1) Synergistic optimization of detection accuracy and efficiency: Unlike current general-purpose detection models that struggle to balance high accuracy and high efficiency on resource-constrained equipment, the proposed method, through a lightweight modular design, achieves high-precision identification of blade defects (mAP50 reaching 0.903) while maintaining real-time detection speed. It focuses on exploring the optimal balance between accuracy and efficiency, resolving the contradiction between the inability to deploy traditional high-precision models and the insufficient accuracy of lightweight models.

[0121] 2) Enhancement of weak features in complex backgrounds: A novel dual-attention mechanism is innovatively designed, which achieves deep mining of global contextual information through a serial channel-space attention mechanism and a parallel multi-attention mechanism. This enables the method framework to effectively suppress background interference such as fog and illumination changes, while enhancing the expression of subtle defect features, significantly improving robustness and accuracy in complex and harsh detection environments.

[0122] 3) Adaptability to Multi-Scale and Irregular Targets: Addressing the challenge of large size ranges and irregular shapes of blade defects, this invention introduces a multi-scale feature fusion and enhancement operator and an optimized model detection head operator. The former utilizes deformable convolution to give the model the ability to dynamically adjust its receptive field to adapt to the geometric deformation of the defect; the latter enhances adaptability to changes in target shape and scale by decoupling the localization and classification tasks. The combination of these two innovations enables this invention to effectively handle various defects ranging from micro-cracks to large-area corrosion.

[0123] In summary, the comprehensive technological innovations of this invention not only present a highly efficient and lightweight solution specifically tailored for wind turbine blade defect detection, but also offer a novel feature enhancement framework that integrates multiple attention mechanisms and dynamic convolution. This framework can effectively serve the current field of intelligent operation and maintenance, and also shows development potential in other industrial defect detection scenarios with challenges of "large background, small target, and weak features." These advantages make this invention an important technical support for UAV intelligent inspection, significantly improving the accuracy and practicality of automated detection.

[0124] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0125] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for detecting surface defects in wind turbine blades, characterized in that, Includes the following steps: Acquire high-resolution images of wind turbine blades captured by drones, the images containing surface defects of the wind turbine blades as: defect targets with huge scale differences and weak features and complex natural background; A neural network model based on a deep learning object detection framework is pre-constructed to process the image; the neural network model sequentially includes a backbone network, a neck network, and a detection head. A dual attention mechanism module is embedded in the backbone network and / or neck network to adaptively refine the input feature map, thereby enhancing the model's perception of defect features and suppressing interference from complex backgrounds. The dual attention mechanism module is composed of a channel-space attention mechanism (CSAM) and a parallel attention mechanism (PAM) connected in series. Multi-scale feature fusion and enhancement operators are introduced into the backbone network to learn the offset of the sampling points of the convolution kernel, so that the receptive field of the model can dynamically fit the irregular geometry and variable spatial position of the defects on the blade surface. The detection head operator, which employs an optimized model, processes the feature map output from the neck network. The detection head operator includes a localization branch and a classification branch set in parallel, which are used to perform bounding box regression and category recognition tasks for the target, respectively. The neural network model is trained using a dataset containing images of surface defects on wind turbine blades to optimize its learnable parameters. The trained model is deployed on an unmanned aerial vehicle (UAV) computing platform to perform real-time defect detection and recognition on the input blade images.

2. The method according to claim 1, characterized in that, The Channel-Spatial Attention (CSAM) mechanism processes the input feature map in the following way: The input feature map is fed into the parallel channel attention branch and spatial attention branch respectively; The channel attention branch is processed sequentially through a global average pooling layer, a first linear layer, a first activation function, a second linear layer, and a second activation function, and then element-wise multiplied with the input feature map to generate channel attention weights. The spatial attention branch is processed by a 7×7 convolutional layer and an activation function, and multiplied with the input feature map to generate spatial attention weights. The channel attention weights and spatial attention weights are added together to obtain a preliminary refined feature map; The preliminary refined feature map is added to the original input feature map through residual connection to obtain the output feature map of the CSAM module.

3. The method according to claim 1, characterized in that, The Parallel Attention Mechanism (PAM) processes the input feature map in the following way: The input feature map is simultaneously input into the Global Channel Attention (GCA) submodule, the Local Channel Attention (LCA) submodule, and the Spatial Attention (SA) submodule; The GCA submodule processes the data sequentially through a global average pooling layer and a 1×1 convolutional layer to extract cross-channel global information. The LCA submodule is processed sequentially through a one-dimensional convolutional layer and a deep convolutional layer, preserving local channel interaction information; The SA submodule is processed by a convolutional layer with a kernel size of 7×7 to preserve key spatial details. The output feature maps of the three sub-modules GCA, LCA, and SA are concatenated along the channel dimension. The concatenated feature map is passed sequentially through the first 1×1 convolutional layer, an activation function, and the second 1×1 convolutional layer, and then residually connected with the output feature map of the channel-spatial attention mechanism CSAM to obtain the final output feature map of the PAM module.

4. The method according to claim 1, characterized in that, The multi-scale feature fusion and enhancement operator is a deformable convolution operator, and its execution process includes: For the input feature map, an offset is learned through an additional convolutional layer, which predicts a two-dimensional offset vector for each sampling point in the standard convolutional grid. Based on the offset vector, the fixed sampling position of the standard convolution is adaptively moved to the critical region of the target defect; Feature values ​​are obtained at the non-integer coordinate positions after the shift using a bilinear interpolation algorithm; The acquired feature values ​​are weighted and summed with the convolution kernel weights to complete the deformable convolution operation, thereby enabling feature extraction of irregular geometric defects.

5. The method according to claim 4, characterized in that, An offset is learned through an additional convolutional layer, expressed by the following formula: Δp m,n (i,j)=f(X)(i×s+m-p,j×s+n-p) Where, Δp m,n (i,j) is a two-dimensional vector representing the offset learned at position (i,j) for the sampling point at position (m,n) in the convolution kernel; f represents the convolution operation, and the learned offset can make the sampling point position adapt to the image content; X represents the input feature map; s represents the stride; p represents the padding amount.

6. The method according to claim 5, characterized in that, The obtained feature values ​​are weighted and summed with the convolution kernel weights, as shown in the following formula: Where Y(i,j) represents the value of the output feature map at position (i,j); k represents the size of the convolution kernel; g(·) represents the bilinear interpolation operation, used to obtain feature values ​​at non-integer coordinate positions; W(m,n) represents the weight of the convolution kernel, corresponding to the weight at position (m,n) in the convolution kernel.

7. The method according to claim 1, characterized in that, The localization and classification branches in the detection head operator of the optimized model are decoupled structures: The localization branch consists of several stacked standard convolutional layers, used to predict the bounding box coordinates of the defective target; the localization loss is calculated as follows: in, L smoothL1 Calculate the loss of the predicted bounding box relative to the ground truth bounding box; (x i ,y i ,w i ,h i () represents the coordinates of the predicted bounding box. The coordinates of the ground truth bounding box; smooth L1 The (x) function represents the smoothed L1 loss, which measures the difference between the bounding box coordinates predicted by the model and the ground truth box. The classification branch is composed of several depthwise separable convolutional layers and is used to predict the classification of defective targets; the comprehensive loss for localization is calculated. Where IOU is the intersection-union ratio between the predicted bounding box and the ground truth bounding box; ρ 2 (b,b gt ) represents the squared Euclidean distance between the center points of the predicted bounding box and the ground truth bounding box; c represents the diagonal length of the minimum closure region containing both boxes; and Used to measure aspect ratio differences; where w and h are the width and height of the predicted bounding box, w gt h gt These are the width and height of the actual bounding box, and arctan(·) is used to measure the difference in aspect ratio.

8. The method according to claim 1, characterized in that, The deep learning object detection framework is the YOLOv11 framework.

Images