Deep learning-based fan blade surface small-size feature detection method and system

Abstract

The invention belongs to the technical field of fan blade detection, and provides a fan blade surface small-size feature detection method and system based on deep learning, and the system comprises an MS-FEM module, a high-resolution feature pyramid module, and a small target optimization prediction head. The method comprises the following steps: preprocessing an input image, inputting the image into a YOLOv11 network, and generating a feature map; the MS-FEM module enhances the feature map, the high-resolution feature pyramid enhances the fusion feature of the spatial information flow, and a high-resolution feature map is generated; and the small target optimization prediction head predicts and outputs the high-resolution feature map. According to the method, through multi-scale feature fusion and global context modeling, the problem that tiny target information in a shallow feature map is lost is effectively solved; a high-resolution feature pyramid is introduced to reserve more small target space information, a small target optimization prediction head is improved to improve small target detection sensitivity, and a small target sensitive loss function is improved to strengthen small target gradient return.

Description

Technical Field

[0001] This invention belongs to the field of wind turbine blade inspection technology. Specifically, it relates to a method and system for detecting small-sized features on the surface of wind turbine blades based on deep learning. In particular, it relates to an improved YOLOv11 target detection model that is particularly suitable for high-precision identification of small-sized defects on the surface of wind turbine blades in the wind power industry. Background Technology

[0002] Feature classification and segmentation techniques based on the YOLO model have been extensively researched and applied in various fields in recent years, and their basic models have achieved significant improvements in various accuracy indicators for classification and segmentation. However, specifically in the field of wind turbine blade inspection, due to the influence of factors such as the lighting environment under the special operating conditions of wind farms, blade surface texture, and feature types, its performance in detecting small and medium-sized features on the surface of wind turbine blades still needs improvement.

[0003] The surface defects of wind turbine blades are small in size, resulting in significant information loss in the high-level feature maps of standard YOLOv11. Defects on the wind turbine blade surface (such as cracks, corrosion, scratches, pits, and coating peeling) constitute a very small proportion of the large-size image captured by an industrial camera. The backbone network of the object detection model downsamples the input image through a series of convolution and pooling operations, gradually extracting higher-level, more abstract semantic features. However, this downsampling process significantly reduces the spatial resolution of the feature maps. Under these conditions, key details representing minute defects (such as edges, texture anomalies, and the direction of micro-cracks) may be completely lost, and the response on the feature map becomes extremely weak or even disappears. Although high-level feature maps possess a larger receptive field and stronger semantic information, their ability to represent the spatial location and fine structure of small targets is severely insufficient, causing the model to be unable to "see" or effectively identify these small defects in high-level feature maps.

[0004] Meanwhile, complex backgrounds (such as blade texture and light reflection) lead to low signal-to-noise ratios for small targets. The surface of wind turbine blades is not a uniform background; their composite materials (such as glass fiber reinforced plastic GFRP) themselves have complex textures, such as weave patterns, graininess, and slight unevenness left from the manufacturing process. These inherent textures appear as high-frequency details and intensity variations in images, easily confused with the morphological features of minute defects. Furthermore, blades are typically in open environments with complex and variable lighting conditions. Direct sunlight, cloud cover, and light from different angles can create strong reflections, highlights, and shadows on the smooth blade surface. These lighting variations can not only mask real defects (such as highlights obscuring shallow cracks) but may also produce artifacts resembling defects. For minute defects, their visual signals are already very weak. Under such complex and noisy background interference, the signal-to-noise ratio becomes extremely low, greatly increasing the risk of false positives and false negatives.

[0005] During YOLO training, pre-defined anchor boxes or locations on feature maps need to be assigned to positive samples or negative background samples. Existing models exhibit an imbalance in the allocation of positive and negative samples for small targets. Mainstream allocation strategies (such as IoU thresholding, ATSS, SimOTA, etc.) typically rely on the intersection-over-union (IoU) ratio or center distance between the predicted and ground truth boxes. However, small targets face inherent disadvantages, such as difficulty in anchor box matching, high positional sensitivity, and a small number of positive samples, all of which ultimately affect the accuracy of the detection model.

[0006] Meanwhile, traditional loss functions are insufficient for optimizing the localization accuracy of small targets. Even small positional shifts have a significant impact on the IoU value of small targets, causing drastic oscillations in the loss value and unstable training. When calculating gradients, the rate of change of IoU caused by small positional changes in small targets may not be smooth, affecting optimization efficiency. Small targets themselves are represented vaguely on feature maps, providing weak regression signals and exacerbating optimization difficulties. The design of traditional loss functions fails to fully consider the scale and sensitivity of localization errors for small targets, resulting in generally lower prediction accuracy for the bounding boxes of small targets compared to large targets.

[0007] In the YOLOv11 base model's Feature Pyramid (FPN), deep features dominate, resulting in insufficient sensitivity of the prediction head for small targets. The FPN was designed to fuse features from different levels; however, in standard implementations, deep features often possess stronger semantic information and tend to dominate the feature fusion process. Meanwhile, shallow, high-resolution features containing key spatial details of small targets may have their information diluted or suppressed during fusion, failing to be effectively transferred and enhanced into the final feature map used for detection. This leads to insufficient representational power of the final feature map for small targets. Furthermore, issues such as receptive field mismatch and insufficient feature representation power also make the prediction head less effective against small targets.

[0008] The patent document "A Method and System for Detecting Surface Damage to Wind Turbine Blades" (CN119887790A) discloses a method combining pyramid enhancement and patch enhancement algorithms. The pyramid enhancement algorithm provides global multi-scale information, while the patch enhancement algorithm provides local detail information. Combined with an attention mechanism and a polar coordinate feature transformation module, an improved YOLOv8 model is constructed to enhance the accuracy of damage detection. This method can learn from information at different levels, improving the accuracy of wind turbine blade damage detection and reducing missed and false detections. However, it has poor feature selection capabilities, a large computational load and parameter count, making it unsuitable for edge deployment.

[0009] The patent document "A Product Small Target Defect Detection Method Based on YoloV8" (CN118628885A) discloses a method that uses the EM-YOLO model based on YoloV8, combined with an efficient multi-scale attention network module (EMA) and a multi-scale feature comparison module. It enhances the representation of fine defect features through hierarchical downsampling and feature fusion networks, suppressing noise interference and enhancing fine-grained feature representation. This achieves accurate identification and location of small-sized defects, avoiding missed and false detections, and improving detection accuracy and robustness. However, it also suffers from poor feature selection capabilities, high computational load and parameter count, making it unsuitable for edge deployment.

[0010] The patent document "A Multi-Scale Feature Fusion Small Target Detection Algorithm for UAV Aerial Images" (CN118262256A) discloses a multi-scale feature fusion small target detection algorithm. By modifying the detection head scale and adding a small object detection head, it uses a multi-scale feature extraction module to extract feature information from fine-grained to high-level, and enhances the information flow between feature layers through a bidirectional dense feature pyramid network to ensure full utilization of features from shallow to deep layers. However, it abandons the original large object detection head, which limits its ability to detect defects of various scales. At the same time, it lacks gradient backpropagation and sensitivity for small targets.

[0011] In summary, detecting minute defects on the surface of wind turbine blades is an extremely challenging task. The difficulty lies in the small physical size of the target itself and the complex environmental background. A new target detection model is needed that can fully adapt to the characteristics of small targets at multiple levels, such as structural design and training mechanism. Summary of the Invention

[0012] In open outdoor environments, lighting and weather conditions vary greatly, and the surface texture of wind turbine blades is also quite complex. Under the influence of these factors, the accuracy of existing YOLO models in detecting small-sized defects on the surface of wind turbine blades has decreased significantly.

[0013] To address the shortcomings of existing technologies, the purpose of this invention is to provide a method and system for detecting small-sized features on the surface of wind turbine blades based on deep learning.

[0014] The present invention provides a deep learning-based system for detecting small-sized features on the surface of wind turbine blades, comprising: an MS-FEM module, a high-resolution feature pyramid module, and a small target optimization prediction head;

[0015] The input image is preprocessed and then fed into the YOLOv11 network to generate feature maps;

[0016] The MS-FEM module enhances the feature map, and the high-resolution feature pyramid enhances the spatial information flow fusion features to generate a high-resolution feature map.

[0017] Small target optimization prediction head predicts output for high-resolution feature maps.

[0018] Preferably, the preprocessing includes uniformly increasing the resolution of the input image to 1280×1280, using adaptive histogram homogenization, and randomly copying and pasting minor defects on the blade surface.

[0019] The MS-FEM module is inserted in the shallow phase of the YOLOv11 backbone network.

[0020] The high-resolution feature pyramid module replaces the FPN / PANet module with a bidirectional cross-scale connection, and includes a top-down path, a bottom-up enhancement module, and a high-resolution output layer.

[0021] The small target optimization prediction head is a multi-receptive field prediction head with a dual-branch structure.

[0022] Preferably, the highest resolution of the network output layer is 256×256.

[0023] One branch of the multi-receptive-field prediction head uses a 3×3 small convolution kernel, while the other branch dynamically adjusts the receptive field size. The anchor box size is [8×8, 16×16, 24×24], and the positive sample matching threshold is 0.3.

[0024] Preferably, it also includes a small target sensitive loss function module;

[0025] The small target sensitive loss function module initializes the loss, calculates the classification loss, and then obtains the predicted value and the ground truth bounding box in sequence.

[0026] Calculate the target area, gradient amplification factor, center point distance d, normalization constant C, and normalized Wasserstein distance;

[0027] If it is not a small target, the weighted regression loss is accumulated; if it is a small target, the IoU is calculated and the small target penalty term is accumulated.

[0028] Normalized regression loss and small objective penalty term;

[0029] Calculate the total loss and form the loss function;

[0030] The gradient values ​​are calculated using the loss function, and the parameters of the high-resolution feature pyramid module and the backbone network of the YOLOv11 network are updated after backpropagation.

[0031] Preferably, the small target sensitive loss function module employs a gradient amplification mechanism, incorporating a gradient amplification coefficient β:

[0032]

[0033] A = wgt ×h gt

[0034] Where A represents the target area;

[0035] 'a' represents the magnification factor;

[0036] k represents the size sensitivity coefficient;

[0037] w gt ,h gt These represent the width and height of the GT frame, respectively.

[0038] Normalized Wasserstein distance, then calculate the distance between the center points:

[0039]

[0040] Among them, (c x ,c y () indicates the center coordinates of the ground plane bounding box;

[0041] (c x ′, c y ′) represents the center coordinates of the prediction box.

[0042] Establish the normalization constant C:

[0043]

[0044] Perform NWD calculations, with values ​​ranging from [0,1]:

[0045]

[0046] Where NWD represents the normalized Wasserstein distance.

[0047] The regression loss is:

[0048] L reg =β·(1-NWD)

[0049] The penalty function for the small target penalty term is:

[0050]

[0051] Where, N small This indicates the number of small targets in a batch;

[0052] T represents the Intersection over Union (IoU) threshold;

[0053] i represents the ordinal number of the smaller target in the batch.

[0054] The loss function is:

[0055] L total =λ cls ·L cls +λ reg ·L reg +λ st ·L st

[0056] Among them, L cls This represents the classification loss.

[0057] A method for detecting small-sized features on the surface of wind turbine blades based on deep learning, according to the present invention, includes:

[0058] Preprocessing steps: Preprocess the input image;

[0059] Image detection steps: The preprocessed image is processed by the MS-FEM module and high-resolution feature pyramid in the YOLOv11 network, and the small target optimization prediction head predicts the output.

[0060] Preferably, the preprocessing step includes uniformly increasing the resolution of the input image to 1280×1280, using adaptive histogram homogenization, and randomly copying and pasting minor defects on the blade surface.

[0061] In the image detection step, the preprocessed image generates a feature map in the YOLOv11 network for detection processing, including:

[0062] Multi-scale feature enhancement steps: The MS-FEM module reduces the dimensionality of the feature map, extracts multi-scale features, constructs a global context model, fuses and enhances the features, and outputs the enhanced feature map through residual connections;

[0063] Spatial flow enhancement steps: The high-resolution feature pyramid receives feature maps from different stages, conveys high-level semantic information, enhances spatial details, and generates high-resolution feature maps.

[0064] Prediction steps: The small target optimization prediction head receives high-resolution feature maps, performs dual-branch feature extraction, and then performs feature fusion to output the prediction.

[0065] Preferably, the multi-scale feature enhancement step includes:

[0066] Step S1.1: Input feature map F in ∈R C×H×W Channel compression is performed using 1×1 convolution:

[0067] F compressed =σ(BN(Conv) 1x1 (F in )))

[0068] Where C represents the number of channels;

[0069] H represents altitude;

[0070] W represents the width;

[0071] R represents the real number space;

[0072] Conv 1×1 This means compressing the number of channels to 1 / 4 of the original input;

[0073] BN represents the ReLU activation function;

[0074] F compressed This represents the feature map after channel compression;

[0075] σ() represents the ReLU activation function.

[0076] Step S1.2: Extract multi-scale features using dilated convolutions with different receptive fields in parallel:

[0077]

[0078] The void ratio is set to a basic receptive field of 3×3 for d=1, a medium receptive field of 7×7 for d=2, and a large receptive field of 13×13 for d=3; d represents the receptive field number.

[0079] F d1 F d2 F d3 These represent the multi-scale features extracted from the 1st, 2nd, and 3rd receptive fields, respectively.

[0080] Step S1.3: Capture image semantic information through global average pooling:

[0081] F gap =GAP(F compressed )

[0082] Perform feature transformation:

[0083] F global =σ(BN(Conv) 1×1 (F gap )))

[0084] Recovering spatial dimension through bilinear interpolation upsampling:

[0085] F upsampled =Upsample bilinear (F global )

[0086] Where GAP(·) is the global average pooling operation;

[0087] F gapThis represents the semantic features after the global pooling operation;

[0088] F global Represents local features that are relevant to the global context;

[0089] F upsampled These represent the features after upsampling;

[0090] Upsample represents the upsampling operation;

[0091] The subscript bilinear indicates bilinear interpolation.

[0092] Step S1.4: Concatenate the multi-scale features and global features along the channel dimension:

[0093] F concat =Concat(F d1 ,F d2 ,F d3 ,F upsampled )

[0094] Feature fusion using 1×1 convolution:

[0095] F fused =σ(BN(Conv) 1×1 (F concat )))

[0096] Enhanced features through residual connection output:

[0097] F out =F in +F fused

[0098] Among them, F concat F fuse F out These represent the concatenated feature map, the feature fused feature map, and the output feature map, respectively.

[0099] Preferably, the spatial flow enhancement step includes:

[0100] Step S2.1: Receive feature maps of different stages of the backbone network, including shallow high-resolution features C2, mid-layer features C3, mid-deep features C4, and deep low-resolution features C5.

[0101] Step S2.2: Passing high-level semantic information from top to bottom:

[0102] P5 = Conv 3×3 (C5)

[0103] P4 = Conv 3×3 (C4)+U2×(Conv 3×3 (P5)

[0104] P3 = Conv 3×3 (C3)+U2×(Conv 3×3 (P4)

[0105] Among them, U 2× This indicates bilinear interpolation with a 2x upsampling.

[0106] Conv 3×3 This represents a 3×3 convolution operation;

[0107] P 3’ P4 and P5 represent the 3rd, 4th and 5th layers of the feature pyramid, respectively.

[0108] Step S2.3: Enhance spatial details from bottom to top:

[0109] N4 = Conv 3×3 (ReLU(P4+D 2× (P5)))

[0110] N3 = Conv 3×3 (ReLU(C2+P 3′ +D 2× (N4)))

[0111] Among them, D 2× This indicates a 3×3 convolution downsampling with a stride of 2;

[0112] ReLU represents the modified linear unit activation function.

[0113] Step S2.4: Generate the final multi-scale feature map:

[0114] P3 = Conv 3×3 (N3)(256×256)

[0115] P4 = Conv 3×3 (N4)(128×128)

[0116] P5 = Conv 3×3 (P5)(64×64)

[0117] Where N4 and N5 represent the enhanced outputs of the 4th and 5th layers of the feature pyramid, respectively.

[0118] Preferably, the prediction step includes:

[0119] Step S2.1: Receive the high-resolution feature map and perform dual-branch feature extraction. For the small receptive field branch:

[0120] Y small =Conv 1×1 (GN(Conv3×3 (X)))

[0121]

[0122] Where C represents the number of channels;

[0123] X represents the feature map to be input;

[0124] Indicates feature dimension;

[0125] Conv 3×3 This represents a 3×3 convolution operation;

[0126] GN indicates group normalization;

[0127] Conv 1×1 This represents a 1×1 convolution operation.

[0128] For the adaptive receptive field branch:

[0129] Y RFB =Conv 1×1 (RFB(X))

[0130] RFB(X)=Concat(Branch1(X),Branch2(X),Branch3(X))

[0131] Branch1(X) = Conv 1×1 (X)

[0132]

[0133] Here, Branch1, Branch2, and Branch3 represent the 1st, 2nd, and 3rd branches of the receptive field, respectively;

[0134] RFB() represents the receptive field branch operation;

[0135] Concat means concatenating along the channel dimension.

[0136] Step S3.2: Perform feature fusion.

[0137] Y fused =Conv 1β1 (Concat(Y small ,Y RFB ))

[0138] Y final =ReLU(GN(Y) fused ))

[0139] Wherein, Concat indicates concatenation along the channel dimension;

[0140] Y small Y RFB Y final Y fused These represent the small receptive field, adaptive receptive field, output receptive field, and fused receptive field branches, respectively.

[0141] Step S3.3, Predicted Output:

[0142]

[0143] Where A represents the number of anchor frames;

[0144] 5 indicates the bounding box parameters, which include 4 coordinates and 1 confidence level;

[0145] C represents the number of defect types.

[0146] Compared with the prior art, the present invention has the following beneficial effects:

[0147] 1. This invention introduces an improved multi-scale feature enhancement module based on the existing YOLO model. Through multi-scale feature fusion and global context modeling, it prevents the loss of small defect features in early downsampling and effectively solves the problem of loss of small target information in shallow feature maps.

[0148] 2. This invention introduces a high-resolution feature pyramid to retain more spatial information of small targets. It adopts a bidirectional cross-scale connection and a high-resolution feature preservation mechanism to effectively solve the problem of spatial information loss when traditional FPN processes small targets, thereby achieving high-resolution feature preservation.

[0149] 3. This invention improves the small target optimization prediction head to enhance the sensitivity of small target detection. By establishing a dual-branch structure and an adaptive receptive field mechanism, it effectively solves the problem of insufficient response of traditional prediction heads to small target features.

[0150] 4. This invention improves the small target sensitive loss function to enhance gradient backpropagation for small targets. By adding a gradient amplification mechanism and normalized Wasserstein distance, it effectively solves the problem of insufficient gradient backpropagation for small targets by the traditional loss function. Attached Figure Description

[0151] Other features, objects, and advantages of the invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0152] Figure 1 This is a schematic diagram of a small-size defect detection system for wind turbine blade surfaces based on YOLOv11;

[0153] Figure 2 A schematic diagram of the Multi-Scale Feature Enhancement Module (MS-FEM);

[0154] Figure 3 This is a schematic diagram of a high-resolution feature pyramid (HR-FPN).

[0155] Figure 4 A schematic diagram of optimizing the prediction head for small-sized targets. Detailed Implementation

[0156] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the protection scope of the present invention.

[0157] To address the issues of small physical size of surface defects on wind turbine blades, reduced signal-to-noise ratio due to complex environmental backgrounds, and the inadequacy of existing YOLO models in terms of structural design and training mechanisms to fully adapt to the needs of small target detection, this invention provides a deep learning-based method and system for detecting small-sized features on wind turbine blade surfaces. This improves the accuracy of small-sized feature detection on wind turbine blade surfaces and enables high-precision detection of small-sized features in open outdoor environments with diverse lighting conditions and complex blade surface textures. This solves the problem of insufficient applicability and accuracy of existing models for detecting wind turbine blade defects in complex environments.

[0158] The introduction of modules such as C3K2 and C2PSA in YOLOv11 provides stronger feature selection capabilities. The introduction of depthwise separable convolution in the classification branch can reduce the amount of computation and parameters while maintaining accuracy, making it more suitable for edge deployment.

[0159] Based on the YOLOv11 model, improvements have been made in multiple aspects, including the detection backbone, classification prediction head, and loss function. An improved Multi-Scale Feature Enhancement Module (MS-FEM) is introduced to enhance the preservation of small target information in shallow features. High-Resolution Feature Pyramid Networks (HR-FPN) are introduced to retain more spatial information of small targets. The small target optimization prediction head is improved to enhance the sensitivity of small target detection. The small target sensitive loss function is improved to strengthen gradient backpropagation for small targets, thereby improving the accuracy of the YOLO model in detecting small features on wind turbine blade surfaces. Compared to the traditional YOLOv11 model, the improvements are more comprehensive and the expected results are better. Figure 1 For example, specifically including:

[0160] Preprocessing steps: Upgrade the resolution of the input images to 1280×1280.

[0161] To adapt to the characteristics of blade images and to provide a data foundation for improving blade surface defect detection algorithms, the resolution of the model input images was uniformly increased to 1280×1280 to preserve details of small targets as much as possible. Simultaneously, adaptive histogram homogenization was employed to suppress uneven illumination, and random copying and pasting of minute defects on the blade surface was used to enhance the data for small targets.

[0162] The preprocessed image is input into the improved YOLOv11 network for image detection. The improvements include:

[0163] Backbone network improvement: In the shallow stages of the backbone network, that is, after Stage 1 generates the C1 feature map and Stage 2 generates the C2 feature map, a multi-scale feature enhancement module (MS-FEM) is added.

[0164] To enhance the detection capability of small-sized defects on wind turbine blade surfaces, a multi-scale feature enhancement module was added to the YOLOv11 model. The MS-FEM module captures contextual information from different receptive fields in the shallow network. Through multi-scale feature fusion and global context modeling, it prevents the loss of minute defect features in early downsampling, effectively solving the problem of information loss of small targets in shallow feature maps. The enhanced C2 feature map and C3, C4, and C5 feature maps are then input into a high-resolution pyramid for further processing.

[0165] Specifically, the MS-FEM module is inserted into the shallow stages of the YOLOv11 backbone network to improve the network's ability to extract small-sized features of wind turbine blades. Figure 2 For example, the steps include:

[0166] Step SA1, Input Feature Dimensionality Reduction:

[0167] Input feature map F in ∈R C×H×W Where C represents the number of channels, H represents the height, W represents the width, and R represents the real space. Channel compression is performed using 1×1 convolution, which reduces computational cost while preserving key information.

[0168] F compressed =σ(BN(Conv) 1×1 (F in )))

[0169] Among them, Conv 1×1 Its function is to compress the number of channels to 1 / 4 of the original input. BN is a batch normalization layer, representing the ReLU activation function. σ() represents the ReLU activation function.

[0170] Step SA2, Multi-scale Feature Extraction:

[0171] Parallel use of dilated convolutions with different receptive fields to extract multi-scale features:

[0172]

[0173] The void ratio is set as follows:

[0174] d = 1: basic receptive field (3×3); d = 2: intermediate receptive field (7×7); d = 3: large receptive field (13×13).

[0175] d represents the receptive field number; F d1 F d2 F d3 These represent the multi-scale features extracted from the 1st, 2nd, and 3rd receptive fields, respectively. By using convolutional kernels with different dilation rates, local details, regional contextual relationships, and global relationships are captured simultaneously, covering a large receptive field range, thus enabling the network to acquire multi-scale perceptual capabilities.

[0176] Step SA3: Global Context Modeling.

[0177] Capturing semantic information of images through global average pooling:

[0178] F gap =GAP(F compressed )

[0179] Here, GAP(·) is the global average pooling operation. GAP provides scene-level semantic information, guiding the network to focus on globally relevant local features, thereby achieving global context guidance.

[0180] Perform feature transformation:

[0181] F global =σ(BN(Conv) 1x1 (F gap )))

[0182] Recovering spatial dimension through bilinear interpolation upsampling:

[0183] F upsampled =Upsample bilinear (F global )

[0184] Among them, F gap F represents the semantic features after the global pooling operation. global F represents a local feature that is relevant to the global context. upsampled These represent the features after upsampling, where Upsample indicates the upsampling operation and the subscript bilinear indicates the bilinear interpolation.

[0185] Step SA4: Feature fusion and enhancement.

[0186] Multi-scale features and global features are concatenated along the channel dimension:

[0187] F concat =Concat(F d1 ,F d2 ,F d3 ,F upsampled )

[0188] Then, feature fusion is performed using a 1×1 convolution as follows:

[0189] F fused =σ(BN(Conv) 1×1 (F concat )))

[0190] Step SA5: Residual connection output.

[0191] Enhanced features through residual connection output:

[0192] F out =F in +F fused

[0193] Residual connections ensure that original features are not lost while adding augmenting information. Gradient pass-through improves training stability, thus achieving an information preservation mechanism.

[0194] High-resolution feature pyramid: Replace the FPN / PANet module in the original model with a high-resolution feature pyramid (HR-FPN).

[0195] A bidirectional cross-scale connection is employed, using a top-down approach to transmit semantic information and a bottom-up approach to enhance spatial information flow. The highest resolution of the network output layer is increased from 80×80 to 256×256, and the resolution of small target feature maps is improved by 4 times. Its input consists of the C2 feature map enhanced by MS-FEM and the C3, C4, and C5 feature maps of the backbone network, and the outputs are P3 (256×256), P4 (128×128), and P5 (64×64), thus achieving high-resolution feature preservation.

[0196] Specifically, the improved HR-FPN is a feature fusion architecture proposed for detecting small-sized defects on the surface of wind turbine blades. Through bidirectional cross-scale connections and a high-resolution feature preservation mechanism, it effectively solves the problem of spatial information loss in traditional FPN when processing small-sized targets.

[0197] The output feature map resolutions are P3: 256×256, P4: 128×128, and P5: 64×64, significantly improving the ability to detect small-sized defects on the wind turbine blade surface. The P3 layer is specifically responsible for detecting small targets. Figure 3 For example, specifically including:

[0198] Step SO1: Input feature preparation. HR-FPN receives feature maps from different stages of the backbone network.

[0199] C2: Shallow high-resolution features (256×256)

[0200] C3: Mid-layer features (128×128)

[0201] C4: Mid-to-deep features (64×64)

[0202] C5: Deep low-resolution features (32×32)

[0203] Among them, C2 is enhanced by MS-FEM, which can preserve the fine details of small targets.

[0204] Step SO2: The top-down path transmits high-level semantic information from top to bottom.

[0205] P5 = Conv 3×3 (C5)

[0206] P4 = Conv 3×3 (C4)+U2×(Conv 3×3 (P5)

[0207] P3 = Conv 3×3 (C3)+U2×(Conv 3×3 (P4)

[0208] Among them, U 2× For bilinear interpolation, upsampled by 2, Conv 3×3 For a 3×3 convolution, the upsampling operation means:

[0209]

[0210] Here, F, x, and y all represent intermediate variables. This injects deep semantic information into shallow features, thus preparing for the P3 output.

[0211] Step SO3: Bottom-up enhancement module strengthens spatial details from the bottom up, as expressed by the following formula:

[0212] N4 = Conv 3×3 (ReLU(P4+D 2× (P5)))

[0213] N3 = Conv 3×3 (ReLU(C2+P 3’ +D 2× (N4)))

[0214] Among them, D 2× This indicates a 3×3 convolution downsampling with a stride of 2. ReLU represents the modified linear unit activation function, P 3’ P4 and P5 represent the 3rd, 4th, and 5th layers of the feature pyramid, respectively, while N4 and N5 represent the enhanced outputs of the 4th and 5th layers. This process transfers high-resolution details from shallow layers to deeper layers, ensuring that spatial information is not lost.

[0215] Step SO4: The high-resolution output layer generates the final multi-scale feature map, expressed by the following formula:

[0216] P3 = Conv 3×3 (N3)(256×256)

[0217] P4 = Conv 3×3 (N4)(128×128)

[0218] P5 = Conv 3×3 (P5)(64×64)

[0219] The high-resolution feature pyramid achieves high resolution preservation, cross-layer feature reuse, and symmetric resolution architecture, providing optimal feature representation for small-size feature detection on wind turbine blade surfaces while maintaining the model's computational speed characteristics.

[0220] Small target optimization prediction head: A multi-receptive field prediction head (MRF-Head) is built using a dual-branch structure.

[0221] One branch of the multi-receptive-field prediction head uses a 3×3 small convolutional kernel to focus on local features of small targets, while the second branch dynamically adjusts the receptive field size. Simultaneously, the anchor boxes are optimized, with anchor box sizes specifically designed for small targets: [8×8, 16×16, 24×24]. The positive sample matching threshold is adjusted from 0.5 to 0.3 to increase the matching rate for small targets. In the network, it is only connected to the highest resolution output P3, meaning its input is the P3 feature map output by HR-FPN.

[0222] Specifically, the Small Target Optimized Multi-Receptive Field Prediction Head (MRF-Head) is a dedicated prediction module designed for small-sized features on wind turbine blade surfaces. By establishing a dual-branch structure and an adaptive receptive field mechanism, it effectively solves the problem of insufficient response of traditional prediction heads to small-sized target features, directly acting on the P3 feature map output by HR-FPN to detect small-sized feature defects on the wind turbine blade surface. Figure 4 For example, specifically including:

[0223] Step SI1: Input feature preparation, receiving high-resolution feature maps from HR-FPN:

[0224]

[0225] Where C is the number of channels and X represents the feature map to be input.

[0226] Indicates the feature dimension.

[0227] Step SI2: Dual-branch feature extraction.

[0228] The small receptive field branch can be expressed by the following formula:

[0229] Y small =Conv 1×1 (GN(Conv 3×3 (X)))

[0230] Among them, Conv 3×3 This is a 3×3 convolution operation with padding of 1, resulting in 256 output channels; GN is group normalization, which divides the input feature map into 8 groups for normalization; Conv 3×3 This is a 3×3 convolution operation with 256 output channels. This function captures local detail features of small targets.

[0231] The adaptive receptive field branch is established, expressed by the following formula:

[0232] Y RFB =Conv 1×1 (RFB(X))

[0233] Where RFB(X) = Concat(Branch1(X),Branch2(X),Branch3(X)). The receptive field size is adaptively adjusted to cover small target features of different sizes. The expressions for each branch structure are as follows:

[0234] Branch1(X)λConv 1×1 (X)

[0235]

[0236] Here, Branch1, Branch2, and Branch3 represent branches 1, 2, and 3 of the receptive field, respectively, and RFB() represents the receptive field branch operation.

[0237] Step SI3, feature fusion, mathematically expressed as:

[0238] Y fused =Conv 1×1 (Concat(Y small ,Y RFB ))

[0239] Y final =ReLU(GN(Y) fused ))

[0240] Where Concat represents concatenation along the channel dimension; Conv 1×1 Compress the channel to 256. Y small Y RFB Y final Y fused These represent the small receptive field, adaptive receptive field, output receptive field, and fused receptive field branches, respectively.

[0241] Step SI4, predict the output, mathematically expressed as:

[0242]

[0243] Where A is the number of anchor boxes, 5 is the bounding box parameter, which includes 4 coordinates and 1 confidence level; C is the number of categories, i.e. the number of defect types.

[0244] The small target optimization prediction head has been improved to enhance the sensitivity of small targets, while the large object detection head has been retained, thus ensuring the ability to detect defects of various scales.

[0245] By incorporating features such as dual-path feature extraction, dynamic receptive field adjustment, and lightweight normalization, the detection capability for small-sized targets is significantly improved.

[0246] The present invention also provides a deep learning-based system for detecting small-size features on the surface of wind turbine blades. The deep learning-based system for detecting small-size features on the surface of wind turbine blades can be implemented by executing the process steps of the deep learning-based method for detecting small-size features on the surface of wind turbine blades. That is, those skilled in the art can understand the deep learning-based method for detecting small-size features on the surface of wind turbine blades as a preferred embodiment of the deep learning-based system for detecting small-size features on the surface of wind turbine blades.

[0247] The present invention provides a deep learning-based small-size feature detection system for wind turbine blade surfaces, comprising: an MS-FEM module, a high-resolution feature pyramid module, and a small target optimized prediction head.

[0248] The input image is preprocessed and then fed into the YOLOv11 network to generate feature maps;

[0249] The MS-FEM module enhances the feature map, and the high-resolution feature pyramid enhances the spatial information flow fusion features to generate a high-resolution feature map.

[0250] Small target optimization prediction head predicts output for high-resolution feature maps.

[0251] Specifically, the preprocessing includes uniformly increasing the resolution of the input image to 1280×1280, using adaptive histogram homogenization, and randomly copying and pasting minor defects on the blade surface.

[0252] The MS-FEM module is inserted in the shallow phase of the YOLOv11 backbone network.

[0253] The high-resolution feature pyramid module replaces the FPN / PANet module with a bidirectional cross-scale connection, and includes a top-down path, a bottom-up enhancement module, and a high-resolution output layer.

[0254] The small target optimization prediction head is a multi-receptive field prediction head with a dual-branch structure.

[0255] In more preferred embodiments, the highest resolution of the network output layer is 256×256;

[0256] One branch of the multi-receptive-field prediction head uses a 3×3 small convolution kernel, while the other branch dynamically adjusts the receptive field size. The anchor box size is [8×8, 16×16, 24×24], and the positive sample matching threshold is 0.3.

[0257] In more preferred examples, the loss function is improved by establishing a small-target sensitive loss function (STS-Loss).

[0258] A gradient amplification mechanism is introduced, which makes the regression loss weight of small targets increase exponentially as the target size decreases; Normalized Wasserstein Distance (NWD) is used instead of IoU (Intersection over Union) to make the model more sensitive to the offset of the center point of small targets, thereby enhancing the positioning accuracy.

[0259] Specifically, the small target sensitive loss function is a dedicated loss function optimized for detecting small target features on the surface of wind turbine blades. By incorporating a gradient amplification mechanism and normalized Wasserstein distance, it effectively addresses the problem of insufficient gradient backpropagation for small targets in traditional loss functions. Specifically, it includes:

[0260] Initialize the loss and calculate the classification loss;

[0261] Iterate through each target box:

[0262] Obtain the predicted value and the ground truth bounding box sequentially; calculate the target area, gradient amplification factor, center point distance d, normalization constant C, and NWD;

[0263] The weighted regression loss is accumulated. If the objective is small, the IoU is calculated and a small objective penalty term is accumulated.

[0264] Specifically, the gradient amplification mechanism mainly involves adding a target size perception factor, i.e., the gradient amplification coefficient:

[0265]

[0266] Where A = w gt ×h gt Let represent the target area, 'a' be the magnification intensity coefficient, and 'k' be the size sensitivity coefficient. gt ,h gt Here are the width and height of the ground truth bounding box. This exponential decay function automatically adapts to the target size, enabling significant gradient enhancement for small targets.

[0267] The normalized Wasserstein distance (NWD) is first calculated by determining the distance between the center points, using the following formula:

[0268]

[0269] Among them, (c x ,c y ) represents the center coordinates of the ground truth bounding box, (c x ′, c y ′) represents the center coordinates of the predicted bounding box. A normalization constant C is established as follows:

[0270]

[0271] Perform NWD calculation:

[0272]

[0273] The NWD value ranges from [0,1]. A larger value indicates more accurate localization, which can make the model more robust to the offset of the center point and make gradient calculation smoother.

[0274] The loss function was optimized to enhance gradient backpropagation for small targets. The regression loss is as follows:

[0275] L reg =β·(1-NWD)

[0276] The penalty term for the small target is defined by the following penalty function:

[0277]

[0278] Where, N small This represents the number of small targets in the batch. The IoU threshold is used. This adaptive penalty mechanism only applies to small targets and can suppress high-confidence positive targets.

[0279] Normalized regression loss, which is divided by the number of target boxes; normalized small target penalty term.

[0280] Calculate the total loss using the small target sensitive loss function:

[0281] L total =λ cls ·L cls +λ reg ·L reg +λ st ·L st

[0282] Where cls is the classification loss, reg is the regression loss, and st is the small target penalty term.

[0283] Parameter Update: The network parameters are updated using the loss function. Specifically, the gradient values ​​are calculated using STS-Loss, and then updated after backpropagation to update the HR-FPN and backbone network parameters.

[0284] The system output after the above improvements and training is used for subsequent blade surface defect detection.

[0285] Those skilled in the art will understand that, besides implementing the system and its various devices, modules, and units provided by this invention in the form of purely computer-readable program code, the same functions can be achieved entirely through logical programming of the method steps, making the system and its various devices, modules, and units of this invention function in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, the system and its various devices, modules, and units provided by this invention can be considered as a hardware component, and the devices, modules, and units included therein for implementing various functions can also be considered as structures within the hardware component; alternatively, the devices, modules, and units for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0286] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.

Claims

1. A deep learning-based system for detecting small-sized features on the surface of wind turbine blades, characterized in that, include: MS-FEM module, high-resolution feature pyramid module, small target optimized prediction head; The input image is preprocessed and then fed into the YOLOv11 network to generate feature maps; The MS-FEM module enhances the feature map, and the high-resolution feature pyramid enhances the spatial information flow fusion features to generate a high-resolution feature map. Small target optimization prediction head predicts output for high-resolution feature maps.

2. The deep learning-based small-size feature detection system for wind turbine blade surfaces according to claim 1, characterized in that, The preprocessing includes uniformly increasing the resolution of the input image to 1280×1280, using adaptive histogram homogenization, and randomly copying and pasting minor defects on the blade surface. The MS-FEM module is inserted in the shallow phase of the YOLOv11 backbone network; The high-resolution feature pyramid module replaces the FPN / PANet module with a bidirectional cross-scale connection, and includes a top-down path, a bottom-up enhancement module, and a high-resolution output layer. The small target optimization prediction head is a multi-receptive field prediction head with a dual-branch structure.

3. The deep learning-based small-size feature detection system for wind turbine blade surfaces according to claim 2, characterized in that, The highest resolution of the network output layer is 256×256; One branch of the multi-receptive-field prediction head uses a 3×3 small convolution kernel, while the other branch dynamically adjusts the receptive field size. The anchor box size is [8×8, 16×16, 24×24], and the positive sample matching threshold is 0.

3.

4. The deep learning-based small-size feature detection system for wind turbine blade surfaces according to claim 1, characterized in that, It also includes a small target sensitive loss function module; The small target sensitive loss function module initializes the loss, calculates the classification loss, and then obtains the predicted value and the ground truth bounding box in sequence. Calculate the target area, gradient amplification factor, center point distance d, normalization constant C, and normalized Wasserstein distance; If it is not a small target, the weighted regression loss is accumulated; if it is a small target, the IoU is calculated and the small target penalty term is accumulated. Normalized regression loss and small objective penalty term; Calculate the total loss and form the loss function; The gradient values ​​are calculated using the loss function, and the parameters of the high-resolution feature pyramid module and the backbone network of the YOLOv11 network are updated after backpropagation.

5. The deep learning-based small-size feature detection system for wind turbine blade surfaces according to claim 4, characterized in that, The small target sensitive loss function module employs a gradient amplification mechanism, incorporating a gradient amplification coefficient β: A=w gt xh gt Where A represents the target area; 'a' represents the magnification factor; k represents the size sensitivity coefficient; w gt ,h gt These represent the width and height of the GT frame, respectively. Normalized Wasserstein distance, then calculate the distance between the center points: Among them, (c x ,c y () indicates the center coordinates of the ground plane bounding box; (c x ′, c y ′) represents the center coordinates of the prediction box; Establish the normalization constant C: Perform NWD calculations, with values ​​ranging from [0,1]: Where NWD represents the normalized Wasserstein distance; The regression loss is: 50 reg =β·(1-NWD) The penalty function for the small target penalty term is: Where, N small This indicates the number of small targets in a batch; τ represents the Intersection over Union (IoU) threshold; i represents the ordinal number of the smaller target in the batch; The loss function is: L total =λ cls ·L cls +λ reg ·L reg +λ st ·L st Among them, L cls This represents classification loss.

6. A method for detecting small-size features on the surface of wind turbine blades based on deep learning, wherein the detection is performed using the deep learning-based wind turbine blade small-size feature detection system described in any one of claims 1-5, characterized in that, include: Preprocessing steps: Preprocess the input image; Image detection steps: The preprocessed image is processed by the MS-FEM module and high-resolution feature pyramid in the YOLOv11 network, and the small target optimization prediction head predicts the output.

7. The method for detecting small-size features on the surface of wind turbine blades based on deep learning according to claim 6, characterized in that, The preprocessing steps include uniformly increasing the resolution of the input image to 1280×1280, using adaptive histogram homogenization, and randomly copying and pasting minor defects on the blade surface. In the image detection step, the preprocessed image generates a feature map in the YOLOv11 network for detection processing, including: Multi-scale feature enhancement steps: The MS-FEM module reduces the dimensionality of the feature map, extracts multi-scale features, constructs a global context model, fuses and enhances the features, and outputs the enhanced feature map through residual connections; Spatial flow enhancement steps: The high-resolution feature pyramid receives feature maps from different stages, conveys high-level semantic information, enhances spatial details, and generates high-resolution feature maps. Prediction steps: The small target optimization prediction head receives high-resolution feature maps, performs dual-branch feature extraction, and then performs feature fusion to output the prediction.

8. The method for detecting small-size features on the surface of wind turbine blades based on deep learning according to claim 7, characterized in that, The multi-scale feature enhancement step includes: Step S1.1: Input feature map F in ∈R C×H×W Channel compression is performed using 1×1 convolution: F compressed =σ(BN(Conv 1×1 (F in ))) Where C represents the number of channels; H represents altitude; W represents the width; R represents the real number space; Conv 1×1 This means compressing the number of channels to 1 / 4 of the original input; BN represents the ReLU activation function; F compressed This represents the feature map after channel compression; σ() represents the ReLU activation function; Step S1.2: Extract multi-scale features using dilated convolutions with different receptive fields in parallel: The void ratio is set to a basic receptive field of 3×3 for d=1, a medium receptive field of 7×7 for d=2, and a large receptive field of 13×13 for d=3; d represents the receptive field number. F d1 F d2 F d3 These represent the multi-scale features extracted from the 1st, 2nd, and 3rd receptive fields, respectively. Step S1.3: Capture image semantic information through global average pooling: F gap =GAP(F compressed ) Perform feature transformation: F global =σ(BN(Conv 1×1 (F gap ))) Recovering spatial dimension through bilinear interpolation upsampling: F upsampled =Upsample bilinear (F global ) Where GAP(·) is the global average pooling operation; F gap This represents the semantic features after the global pooling operation; F global Represents local features that are relevant to the global context; F upsampled These represent the features after upsampling; Upsample represents the upsampling operation; The subscript bilinear indicates bilinear interpolation; Step S1.4: Concatenate the multi-scale features and global features along the channel dimension: F concat =Concat(F d1 ,F d2 ,F d3 ,F upsampled ) Feature fusion using 1×1 convolution: F fused =σ(BN(Conv 1×1 (F concat ))) Enhanced features through residual connection output: F out =F in +F fused Among them, F concat F fuse F out These represent the concatenated feature map, the feature fused feature map, and the output feature map, respectively.

9. The method for detecting small-size features on the surface of wind turbine blades based on deep learning according to claim 7, characterized in that, The spatial flow enhancement step includes: Step S2.1: Receive feature maps of different stages of the backbone network, including shallow high-resolution features C2, mid-layer features C3, mid-deep features C4, and deep low-resolution features C5. Step S2.2: Passing high-level semantic information from top to bottom: P5=Conv 3＝3 (C5) P4=Conv 3×3 (C4)+U 2× Conv 3×3 (P5)) P3=Conv 3×3 (C3)+U 2× (Conv 3×3 (P4)) Among them, U 2× This indicates bilinear interpolation with a 2x upsampling. Conv 3×3 This represents a 3×3 convolution operation; P3', P4, and P5 represent the 3rd, 4th, and 5th layers of the feature pyramid, respectively. Step S2.3: Enhance spatial details from bottom to top: N4=Conv 3×3 (ReLU(P4+D 2× (P5))) N3=Conv 3×3 (ReLU(C2+P 3′ +D 2× (N4))) Among them, D 2× This indicates a 3×3 convolution downsampling with a stride of 2; ReLU represents the modified linear unit activation function; Step S2.4: Generate the final multi-scale feature map: P3=Conv 3×3 (N3)(256×256) P4=Conv 3×3 (N4)(128×128) P5=Conv 3×3 (P5)(64×64) Where N4 and N5 represent the enhanced outputs of the 4th and 5th layers of the feature pyramid, respectively.

10. The method for detecting small-size features on the surface of wind turbine blades based on deep learning according to claim 7, characterized in that, The prediction steps include: Step S3.1: Receive the high-resolution feature map and perform dual-branch feature extraction. For the small receptive field branch: Y small =Conv 1×1 (GN(Conv 3×3 (X))) Where C represents the number of channels; X represents the feature map to be input; Indicates feature dimension; Conv 3×3 This represents a 3×3 convolution operation; GN indicates group normalization; Conv 1×1 This represents a 1×1 convolution operation; For the adaptive receptive field branch: Y RFB =Conv 1×1 (RFB(X)) RFB(X)=Concat(Branch1(X),Branch2(X),Branch3(X)) Branch1(X)=Conv 1×1 (X) Here, Branch1, Branch2, and Branch3 represent the 1st, 2nd, and 3rd branches of the receptive field, respectively; RFB() represents the receptive field branch operation; Concat indicates concatenation along the channel dimension; Step S3.2: Perform feature fusion. AND fused =Conv 1×1 (Concat(Y small ,AND RFB )) AND final =ReLU(GN(Y fused )) Wherein, Concat indicates concatenation along the channel dimension; Y small Y RFB Y final Y fused These represent the small receptive field, adaptive receptive field, output receptive field, and fused receptive field branches, respectively. Step S3.3, Predicted Output: Where A represents the number of anchor frames; 5 indicates the bounding box parameters, which include 4 coordinates and 1 confidence level; C represents the number of defect types.

Images