A wind power generation blade surface defect detection method and system based on a TFPN-YOLO model

Abstract

The present application relates to a kind of wind power generation blade surface defect detection method and system based on TFPN-YOLO model, improved defect detection network is constructed, the improved defect detection network is based on YOLOv12 network as basic network;Acquire wind power generation blade surface image dataset, construct dataset after pre-processing, train the improved defect detection network with dataset;Collect wind power generation blade surface graph, input the improved defect detection network after training, obtain defect class, location and confidence information;With method the setting of system is realized.The present application effectively solves the key problems in the field of wind power generation blade surface defect detection, significantly improves the detection accuracy and efficiency, reduces the consumption of computing resources, improves the applicability and stability of model, provides a new solution for the blade surface defect detection of wind power industry, and promotes the technical progress and development in this field.

Description

Technical Field

[0001] This invention relates to the technical field of general image data processing or generation, and in particular to a method and system for detecting surface defects in wind turbine blades based on the TFPN-YOLO model. Background Technology

[0002] Wind power, as a crucial component of clean energy, plays a key role in the global energy structure transformation. Wind turbine blades are the core component of wind turbine units, and their structural integrity directly affects power generation efficiency, equipment lifespan, and operational safety. However, due to the long-term exposure of blades to complex natural environments such as strong winds, rain erosion, ultraviolet radiation, and salt spray corrosion, coupled with limitations in manufacturing processes, blade surfaces are prone to defects such as cracks, peeling, bulging, coating delamination, and lightning strike damage. If these defects are not detected and addressed in a timely manner, they can rapidly expand, leading to blade breakage, turbine shutdown, or even safety accidents, resulting in significant economic losses.

[0003] Currently, defect detection in wind turbine blades mainly relies on manual visual inspection, UAV-mounted visible light / thermal imager inspection, acoustic or vibration signal-based inspection, and ultrasonic testing. Manual visual inspection involves technicians observing the blade surface at close range using telescopes, aerial work platforms, or suspended baskets; this method suffers from low accuracy and high cost. UAV-mounted visible light / thermal imager inspection acquires images of the blade through aerial photography and combines image processing technology to identify defects. However, this method is susceptible to environmental interference, thermal imaging is only effective for specific defects such as internal debonding, and image resolution and flight stability directly limit detection accuracy. Acoustic or vibration signal-based inspection uses sensors to capture excited vibrations or sound wave reflections from the blade to diagnose internal damage. It is insensitive to small defects, has a high rate of missed detections, and requires complex sensor networks, making implementation difficult and costly. While ultrasonic testing can identify minute blade defects and has broad coverage, it requires external force to operate and cannot accurately reflect the extent of the defect being detected.

[0004] With the rapid development of artificial intelligence technology, deep learning-based methods for detecting surface defects on wind turbine blades have gradually become a research hotspot, especially with the widespread application of deep learning technology in the field of target detection. These methods construct deep learning models and train them on a large number of images of surface defects on wind turbine blades to achieve accurate identification and localization of defects. However, existing deep learning-based algorithms for detecting surface defects on wind turbine blades still have some problems. For example, some algorithms suffer from high computational cost and slow detection speed due to complex model structures, making it difficult to meet the needs of real-time detection. Furthermore, the detection accuracy of some algorithms still needs improvement, particularly for defects in small targets and complex backgrounds.

[0005] With the development of deep learning technology, the YOLO series of algorithms has been widely used due to its high efficiency and accuracy. YOLOv12, as one of the latest versions of the YOLO series, performs excellently in object detection tasks. However, when applied to wind turbine blade surface defect detection, it still suffers from insufficient detection accuracy and a large number of parameters. Specifically, the existing YOLOv12 network, when processing wind turbine blade surface defect datasets, struggles to fully extract the subtle defect features of the blade surface due to limitations in the model's mesh structure, resulting in low detection accuracy. Simultaneously, the large number of parameters increases the model's complexity and computational resource requirements, hindering its deployment and widespread adoption in practical applications.

[0006] Traditional detection methods suffer from low efficiency and insufficient accuracy, while existing deep learning-based detection methods are limited by model complexity and slow detection speed. Addressing these issues and current technological shortcomings, a lightweight, high-precision algorithm for detecting surface defects on wind turbine blades is of great significance. Summary of the Invention

[0007] This invention addresses the problems existing in the prior art by providing a method and system for detecting surface defects of wind turbine blades based on the TFPN-YOLO model. It solves the problem of insufficient detection accuracy in the field of wind turbine blade surface defect detection, accurately identifies small target defects on the surface of wind turbine blades, and improves detection efficiency.

[0008] The technical solution adopted in this invention is a method for detecting surface defects of wind turbine blades based on the TFPN-YOLO model. The method constructs an improved defect detection network, which is based on the YOLOv12 network.

[0009] A dataset of images of the surface of wind turbine blades was acquired, preprocessed, and then used to train the improved defect detection network.

[0010] A surface image of a wind turbine blade is collected and input into the improved defect detection network after training to obtain defect category, location, and confidence information.

[0011] Preferably, the improved defect detection network includes a Backbone module, a Neck module, and a Head module arranged in sequence;

[0012] Replace all feature extraction blocks of the base network with feature extraction blocks modified by Fourier-gated bottleneck convolutions;

[0013] A self-supervised visual backbone network is set after the feature extraction block modified by the second Fourier gated bottleneck convolution of the Backbone module, and the output of the self-supervised visual backbone network is associated with the Neck module;

[0014] Set up a dynamic multi-scale feature fusion module after A2C2f in the Backbone module and output it to the Neck module;

[0015] All fully connected blocks of the base network are replaced with a bidirectional feature pyramid network, and several convolutional layers are set up in conjunction with the bidirectional feature pyramid network for cross-channel information integration.

[0016] A detection head with multi-scale feature fusion is set up in conjunction with the Head module.

[0017] Preferably, the feature extraction block modified by the Fourier gated bottleneck convolution includes a CBS block and a split block connected in sequence. The split block is followed by a parallel identity branch and a refined branch. The identity branch and the refined branch are fused by a fully connected layer and then output after passing through a spatial attention network and a CBS block.

[0018] Preferably, the refined branch includes three sequentially connected Fourier-gated bottleneck convolutional modules. Each Fourier-gated bottleneck convolutional module includes a Fourier-gated convolutional block arranged in coordination. The Fourier-gated convolutional block includes a bottleneck convolutional layer, an activation function layer, and a Fourier enhancement block. In the Fourier enhancement block, the output of the Fourier transform layer is output to the channel splicing layer after passing through an average pooling layer and a channel dimension averaging layer in the amplitude spectrum branch and the phase spectrum branch, respectively. A convolutional block is provided after the channel splicing layer, and the output of the convolutional block is multiplied element-wise with the output of the Fourier transform layer.

[0019] Preferably, the self-supervised visual backbone network includes a reconstructed layer, an upsampling layer, and a convolutional layer connected in sequence.

[0020] Preferably, the dynamic multi-scale feature fusion module includes a multi-scale receptive field branch, a cascaded large kernel attention branch, and a vertical-horizontal context branch arranged in parallel;

[0021] The multi-scale receptive field branch includes sequentially connected CBS blocks and three max-pooling layers;

[0022] The cascaded large kernel attention branch includes four large kernel attention blocks connected in sequence. The output of the CBS block is the input of the first large kernel attention block. The input of each other large kernel attention block is the sum of the output of the corresponding max pooling layer and the output of the previous large kernel attention block.

[0023] The vertical-horizontal context branch includes sequentially connected average pooling layers, dynamic convolutional layers, grouped vertical-horizontal convolutional groups, and bilinear upsampling layers;

[0024] The outputs of the multi-scale receptive field branch, the cascaded large kernel attention branch, and the vertical-horizontal global context branch are concatenated and used as the output of the dynamic multi-scale feature fusion module.

[0025] Preferably, a TFPN feature fusion channel is established based on the bidirectional feature pyramid network;

[0026] The TFPN feature fusion channel fuses the outputs of the self-supervised visual backbone network and subsequent A2C2f from the first bidirectional feature pyramid network in the Neck module; fuses the outputs of the self-supervised visual backbone network and the feature extraction block modified by the first Fourier gated bottleneck convolution from the second bidirectional feature pyramid network in the Neck module; fuses the outputs of the first bidirectional feature pyramid network and the two A2C2f outputs from the second bidirectional feature pyramid network from the third bidirectional feature pyramid network in the Neck module; and fuses the outputs of the dynamic multi-scale feature fusion module and the A2C2f output from the third bidirectional feature pyramid network from the fourth bidirectional feature pyramid network in the Neck module.

[0027] Preferably, the multi-scale feature fusion detection head includes a bounding box regression branch, a classification branch, and a bounding box prediction branch arranged in parallel;

[0028] The bounding box regression branch, classification branch, and bounding box prediction branch all include sequentially arranged convolutional layers and multiple parallel CBS blocks. The outputs of the multiple parallel CBS blocks are added together and then output through the CBS block and the convolutional layer.

[0029] Preferably, a loss function is constructed. Train the improved defect detection network.

[0030]

[0031] Where K is the total number of scales used in the multiscale analysis;

[0032] If K=1, it degenerates into a single-scale SIoU;

[0033] This represents the fusion weight at the i-th scale. , For learnable parameters, Indicated in scale The intersection and union ratio below, Indicated in scale Distance matching cost Indicated in scale The cost of shape matching below.

[0034] A surface defect detection system for wind turbine blades based on the TFPN-YOLO model includes:

[0035] At least one processor; and

[0036] A memory that is communicatively connected to at least one of the processors;

[0037] The memory stores instructions that can be executed by the processor to implement the wind turbine blade surface defect detection method based on the TFPN-YOLO model.

[0038] This invention relates to a method and system for detecting surface defects in wind turbine blades based on the TFPN-YOLO model. The method involves constructing an improved defect detection network, with the YOLOv12 network as its base network; acquiring a dataset of wind turbine blade surface images, preprocessing them to construct a new dataset, and using this dataset to train the improved defect detection network; collecting surface images of the wind turbine blades and inputting them into the trained improved defect detection network to obtain defect category, location, and confidence information; and implementing the system setup using this method.

[0039] The beneficial effects of this invention are as follows: for YOLOv12, innovative improvements are introduced, such as the DINOv3 frozen backbone, the combination of dynamic multi-scale feature fusion module, and the use of dynamic convolution and grouped convolution to reduce the computational burden. These improvements address the shortcomings of the YOLOv12 model in the detection of surface defects on wind turbine blades, reduce model parameters and computational load, improve the model's detection accuracy and perception ability of small targets, and improve the model's feature capture ability by replacing the original decoupled detection head of YOLOv12 with the lightweight and multi-scale feature fusion detection head CE-Head.

[0040] This invention effectively solves key problems in the field of wind turbine blade surface defect detection, significantly improves detection accuracy and efficiency, reduces computational resource consumption, enhances the applicability and stability of the model, provides a new solution for wind turbine blade surface defect detection, and promotes technological progress and development in this field. Attached Figure Description

[0041] Figure 1 This is a flowchart of the method of the present invention;

[0042] Figure 2 This is a schematic diagram of the improved defect detection network in this invention;

[0043] Figure 3 This is a schematic diagram of the feature extraction block modified by Fourier gated bottleneck convolution in this invention;

[0044] Figure 4This is a schematic diagram of the dynamic multi-scale feature fusion module in this invention;

[0045] Figure 5 This is a schematic diagram of the detection head for multi-scale feature fusion in this invention;

[0046] Figure 6 This is a schematic diagram of establishing a TFPN feature fusion channel based on the bidirectional feature pyramid network in this invention;

[0047] Figure 7 The mAP and recall of the improved defect detection network in this invention for detecting surface defects of wind turbine blades are shown in (a) and (b), respectively.

[0048] Figure 8 This is an example diagram of the detected defect portion output by the improved defect detection network in this invention. Detailed Implementation

[0049] The present invention will be further described in detail below with reference to embodiments, but the scope of protection of the present invention is not limited thereto.

[0050] like Figure 1 As shown, this invention relates to a method for detecting surface defects of wind turbine blades based on the TFPN-YOLO model. The method constructs an improved defect detection network, which is based on the YOLOv12 network.

[0051] A dataset of images of the surface of wind turbine blades was acquired, preprocessed, and then used to train the improved defect detection network.

[0052] A surface image of a wind turbine blade is collected and input into the improved defect detection network after training to obtain defect category, location, and confidence information.

[0053] The steps are explained below with reference to specific embodiments. The experimental hardware configuration is a Xeon(R) Silver 4214R processor, an NVIDIA GeForce RTX4060 graphics card, Python 3.11, CUDA version 12.4, and PyTorch version 1.14.

[0054] (1) Construct an improved defect detection network, wherein the improved defect detection network is based on the YOLOv12 network;

[0055] The improved defect detection network includes a Backbone module, a Neck module, and a Head module arranged in sequence; such as Figure 2 As shown, the specific improvements include:

[0056] All feature extraction blocks in the base network are replaced with feature extraction blocks C3K2_FGBC modified by Fourier gated bottleneck convolution. Fourier gated convolution is used to make the features of small targets sharper, more stable, and easier for the detection head to acquire.

[0057] A self-supervised visual backbone network DINOv3Backbone is set after the feature extraction block modified by the second Fourier gated bottleneck convolution of the Backbone module to enhance the feature extraction capability of the model. The output of the self-supervised visual backbone network is associated with the Neck module.

[0058] A dynamic multi-scale feature fusion module is set up after A2C2f in the Backbone module (after P5 layer) and output to the Neck module to further enhance the feature extraction capability of small targets;

[0059] The bidirectional feature pyramid network replaces all fully connected blocks of the base network to enhance the multi-scale perception capability of the target detection model. Several convolutional layers are set in conjunction with the bidirectional feature pyramid network for cross-channel information integration.

[0060] The detection head, which incorporates multi-scale feature fusion in conjunction with the Head module, is more lightweight.

[0061] (1-1) Feature extraction block C3K2_FGBC modified by Fourier gated bottleneck convolution

[0062] This invention addresses the technical challenges of existing target detection algorithms, such as insufficient extraction of small target defect features and significant background interference, by introducing the C3K2_FGBC multi-branch bottleneck-channel-space triple attention module. This significantly enhances the model's ability to focus on multi-scale channel correlation and spatial details, improves the sensitivity and positioning accuracy of small target defects, and enables the detection head to receive more detailed small target defect features, thereby achieving high-precision and robust detection in complex environments.

[0063] like Figure 3 As shown, the feature extraction block modified by the Fourier gated bottleneck convolution includes a sequentially connected CBS block and a split block. After the split block, there are parallel identity branches and refined branches. The identity branches and refined branches are fused and concatenated by a fully connected layer and then output after passing through the spatial attention network SPA and the CBS block.

[0064] The refined branch includes three sequentially connected Fourier-gated bottleneck convolutional modules (FGBCs). Each Fourier-gated bottleneck convolutional module includes a coordinated Fourier-gated convolutional block (Block1~Block4). The Fourier-gated convolutional block includes a bottleneck convolutional layer (BotConv), an activation function layer (ReLU), and a Fourier enhancement block (SpeAtt). In the Fourier enhancement block, the output of the Fourier transform layer is passed through an average pooling layer and a channel-dimensional averaging layer in the amplitude spectrum branch and the phase spectrum branch, respectively, and then output to the channel splicing layer. After the channel splicing layer, there is a convolutional block. The output of the convolutional block is multiplied element-wise with the output of the Fourier transform layer.

[0065] The initial convolution doubles the number of input channels and splits them into two branches. The identity branch retains the original features, while the refinement branch sequentially passes through multiple Fourier-gated bottleneck convolutional modules (FGBC) for multi-stage depthwise separable bottleneck transformation. The input features are first fed into parallel Block 1 and Block 3, respectively. The features output from Block 3 are fed into Block 4 for further refinement. The enhanced features from Block 4 are then multiplied element-wise with the gate weights generated by Block 1 and output to Block 2, reprojecting the features back to the original dimension and completing feature integration. Specifically, the refinement branch internally employs a bottleneck convolution (BotConv) architecture, projecting features into a low-rank subspace through pointwise convolution (PointConv) and using depthwise convolution (DepthConv) for lightweight spatial construction. The model is then projected back to the original dimension after modeling, thus significantly reducing computational complexity while preserving key features. The processed features are then passed through the ReLU activation layer and entered into the SpeAtt Fourier enhancement block. The Fourier transform converts the input features from the spatial domain to the frequency domain, decomposing them into the amplitude spectrum representing frequency intensity and the phase spectrum representing spatial location. Global average pooling and channel-dimensional averaging are then performed on both to obtain global frequency domain features. The amplitude and phase information are then concatenated and the frequency domain importance weights of each channel are learned through convolutional layers. Finally, these weights are multiplied element-wise with the original input to enhance important frequency domain features and suppress unimportant features, thereby improving the model's feature representation ability. Subsequently, all branch outputs are concatenated along the channel dimension to form multi-scale feature fusion. The feature response of important spatial regions is enhanced through a spatial attention mechanism, and Dropout regularization is used to prevent overfitting. Finally, a 1×1 convolution is used to adjust the number of channels to the target output dimension, completing the refinement and integration of features.

[0066] In this invention, the entire forward process maintains a constant spatial resolution, while aggregating multi-level receptive fields, channel-space dual attention representations, and dense residual gradient flow to achieve efficient and lightweight multi-scale feature enhancement.

[0067] (1-2) Self-supervised visual backbone network DINOv3Backbone

[0068] This invention addresses the technical challenges of existing detectors in complex backgrounds, scenes with drastic lighting changes, and small cross-domain targets—namely, the need for large-scale fine-tuning, memory overload, and NaN loss—by introducing a cascaded structure of "DINOv3 frozen backbone - delayed loading - normalization repair - channel alignment interpolation." This significantly enhances the model's ability to capture high-resolution dense features, numerical stability, and multi-scale alignment, improving sensitivity and localization accuracy for small target defects. This achieves zero-fine-tuning, low-cost, and high-precision detection in complex environments. By freezing the weights and preventing their participation in training and inference, the extracted features are used to enhance the model's expressive power. Further enhancement of small target defect feature extraction can be achieved after the P3 layer in the backbone layer.

[0069] The self-supervised visual backbone network includes a sequentially connected recombination layer, an upsampling layer, and a convolutional layer.

[0070] Specifically, in the training instructions, freeze=[5] represents training. During inference, the DINOv3Backbone is frozen, and gradient updates and parameter training are not performed. The rich feature information contained in this weight is used to enhance the expressive power of the model, thereby improving detection accuracy and reducing model training time, thus improving detection efficiency. After placing the self-supervised visual backbone network into the P3 feature layer, the version of the pre-trained weights used is dinov3_vits16. For a given input image of 80×80×256 after the P3 feature layer... When the improved DINOv3Backbone is input with a patch size of 16 and intermediate features of 80×80×256, the module detects the 256-channel input and compresses the features into a 3-channel representation through 3×3 and 1×1 convolutions. This is then upsampled to 224×224 and processed by ViT on DINOv3-vits16, generating a 14×14 384-dimensional patch feature. This feature is then passed through 12 Transformer encoding blocks, projected to 256 dimensions via linear layers, and upsampled back to 80×80 resolution. This approach leverages the advantages of both CNN and DINOv3Backbone to enhance feature extraction capabilities and improve detection accuracy.

[0071] (1-3) Dynamic Multi-Scale Feature Fusion Module Dy-MFFM

[0072] In this invention, the introduction of the Dy-MFFM multi-receptive field-cascaded LKA-local-global parallel pooling module can solve the technical problems of insufficient fusion of multi-scale features and loss of contextual information in the neck part of existing detection under complex backgrounds, drastic scale changes and dense arrangement. It significantly enhances the model's ability to capture multi-receptive field spatial-channel coupling features and global-local contextual information, improves the scale adaptability and localization accuracy of small target defects, and thus achieves high-precision, multi-scale and robust detection in complex environments.

[0073] like Figure 4 As shown, the dynamic multi-scale feature fusion module includes parallel multi-scale receptive field branches, cascaded large kernel attention branches, and vertical and horizontal-global context branches;

[0074] The multi-scale receptive field branch includes sequentially connected CBS blocks and three max-pooling layers (Maxpool).

[0075] The cascaded large kernel attention branch includes four sequentially connected large kernel attention blocks (LKAs). The output of the CBS block is the input of the first large kernel attention block. The input of each other large kernel attention block is the sum of the output of the corresponding max pooling layer and the output of the previous large kernel attention block.

[0076] The vertical-horizontal context branch includes sequentially connected average pooling layer AvgPool, dynamic convolutional layer DyConv, grouped vertical-horizontal convolutional group DWConv, and bilinear upsampling layer Upsample.

[0077] The outputs of the multi-scale receptive field branch, the cascaded large kernel attention branch, and the vertical-horizontal global context branch are concatenated and used as the output of the dynamic multi-scale feature fusion module.

[0078] Specifically, after adding Dy-MFFM to A2C2f (the feature extraction module that introduces region attention), Dy-MFFM first performs a 1×1 convolution on the input... Compress to Preserve all spatial information; then... Perform a three-level cascaded 5×5 max pooling with a stride of 1 to generate Y=[Y0, ​​Y1, Y2, Y3] without reducing resolution, equivalently obtaining a multi-scale receptive field from 5×5 to 13×13; then construct four depthwise separable large kernel-dilated convolution branches LKA1-LKA4 (dilation rates 4, 3, 2, 1), and use a residual progression method out=LKA(out) -1 +Y -1The system refines and integrates multi-scale features step by step; simultaneously, it introduces horizontal and vertical global context branches in parallel. Y0 is first downsampled using 7×7 average pooling and then dynamically convolved using Dyconv with a 4×4 kernel. Then, it performs intra-module group convolutions in both horizontal and vertical directions, serially using (1,11) / (11,1) grouped horizontal and vertical convolutions to capture long-range horizontal and vertical dependencies. Finally, it performs bilinear upsampling to restore resolution, resulting in X2. The original Y0, max-pooled Y3, LKA-refined out4, and global context X2 are densely concatenated along the channels to form... After being convolved pointwise by 1×1 and projected onto the target C2, the output is... This enables efficient fusion of multi-scale, multi-path, and multi-attention approaches.

[0079] (1-4) Establish TFPN feature fusion channel based on the bidirectional feature pyramid network BiFPN.

[0080] In this invention, the TFPN feature fusion network enhances the multi-scale perception capability of the target detection model, enabling the P2 and P3 feature layers to be effectively fused with higher-level feature maps, reducing feature loss in the P2 and P3 feature layers. Through this fusion method, detailed information in the P2 and P3 feature layers can be effectively transferred to higher-level feature maps, thereby enhancing the detection capability of these feature maps for targets at different scales.

[0081] Specifically, the TFPN feature fusion channel uses the first bidirectional feature pyramid network in the Neck module to fuse the output of the self-supervised visual backbone network DINOv3Backbone and its subsequent A2C2f (feature extraction module with region attention) in the Backbone module; uses the second bidirectional feature pyramid network in the Neck module to fuse the output of the self-supervised visual backbone network DINOv3Backbone and the first Fourier-gated bottleneck convolution modified feature extraction block in the Backbone module; uses the third bidirectional feature pyramid network in the Neck module to fuse the output of the first bidirectional feature pyramid network and the two A2C2f (feature extraction modules with region attention) after the second bidirectional feature pyramid network; and uses the fourth bidirectional feature pyramid network in the Neck module to fuse the output of the dynamic multi-scale feature fusion module Dy-MFFM and the A2C2f (feature extraction module with region attention) after the third bidirectional feature pyramid network.

[0082] The TFPN feature fusion process involves compressing the P2 feature by 1×1 and upsampling it with the nearest neighbor, and then injecting it into the P3 feature all at once using learnable fast normalized fusion weights.

[0083] Referring to the arrows in the network diagram, which indicate the specific sequence directions, in this invention, the B2, B3, B4, and B5 feature layers are deeply fused through an improved TFPN feature fusion network. B2 and B3, after being processed by CBS downsampling and 1×1 convolutional channel adjustment, are added as fusion input branches in two different BiFPNs, realizing the full utilization of shallow to deep features. This enhances the model's ability to capture small feature defects while also improving its global semantic understanding of large target defects, thereby better achieving defect localization.

[0084] like Figure 6 As shown, the specific implementation process is as follows:

[0085] First, the feature maps of layers B2 and B3 are fused. After upsampling and downsampling, cross-fusion is performed. Taking a 640*640 image as input, the input of layer B2 is 160*160, and layer P3 is 80*80. The output sizes of the two are mismatched. Considering the channel consistency requirement of each fusion branch of the BiFPN fusion module, a downsampling layer and a channel adjustment layer are added after layer B2 to match the size and number of channels of the output features of layer P3. Then, the feature maps of layers B3 and B4 are fused. After upsampling and downsampling, cross-fusion is performed. The input of layer B3 is 80*80, and layer P4 is 40*40. Again, the output sizes of the two are mismatched. Similarly, there is a problem of channel and size mismatch at the output of layer B5. Therefore, a downsampling layer and a channel adjustment layer are added after layers B3 and B5 to match the size and number of channels of the output features of layer P4.

[0086] (1-5) CE_Head, a detection head for multi-scale feature fusion

[0087] By introducing the CE_Head three-branch large receptive field-group convolutional enhanced detection head, the technical challenges of inaccurate regression of small target defect bounding boxes and low classification confidence in existing YOLO series under complex backgrounds, high-density arrangements and extreme lighting conditions are solved. This significantly enhances the model's ability to capture multi-scale receptive field features and channel-space coupling information, improves the localization accuracy and classification robustness of small target defects, and thus achieves high-precision, high-recall and low-false-detection detection in complex environments.

[0088] like Figure 5 As shown, the multi-scale feature fusion detection head includes a bounding box regression branch, a classification branch, and a bounding box prediction branch arranged in parallel;

[0089] The bounding box regression branch, classification branch, and bounding box prediction branch all include sequentially arranged convolutional layers (CBS blocks) and multiple parallel CBS blocks. The outputs of the multiple parallel CBS blocks are added together and then output through the CBS block and the convolutional layer Conv.

[0090] Specifically, CE_Head includes three parallel branches responsible for bounding box regression (Reg), classification (Cls), and bounding box prediction (Bbox), respectively. Each branch first processes the input feature map through a convolutional layer (k=1), then further extracts features through multiple convolutional layers (k=3 and k=5) containing Batch Normalization and SiLU activation functions, and adds the feature map to the original feature map through residual connections to preserve information and promote gradient flow.

[0091] Finally, the Reg branch outputs bounding box regression feature maps, the Cls branch outputs classification feature maps, and the Bbox branch outputs bounding box prediction feature maps. The entire structure achieves efficient feature fusion and multi-task prediction through parallel processing and residual connections, making it suitable for application scenarios that require lightweight and efficient detection.

[0092] (2) Obtain a dataset of images of the surface of wind turbine blades, preprocess them to construct a dataset, and use the dataset to train the improved defect detection network.

[0093] In this embodiment, publicly available wind turbine blade surface defect datasets were collected from the Robflows network platform, including 7 categories: burns, cracks, deformation, dirt, oil stains, peeling, and rust, totaling 4787 images. The collected wind turbine blade data were labeled using the open-source software LabelImg.

[0094] Furthermore, the dataset was augmented using Mosaic and Cutmix. Mosaic data augmentation uses four images and stitches them together. Each image has its corresponding bounding box. After stitching the four images together, a new image is obtained. Cutmix randomly selects two images and randomly cuts out a rectangular region from each image. Then, the cut regions of the two images are swapped and merged into a new image.

[0095] The processed dataset was randomly divided into a training set, a validation set, and a test set in a ratio of 8:1:1. The training set contained 3766 images, the validation set contained 560 images, and the test set contained 461 images.

[0096] During training, the hyperparameters in this embodiment include 300 training iterations, a batch size of 16, a maximum learning rate of 0.01, and a minimum learning rate of 0. The learning rate is dynamically adjusted using a cosine annealing algorithm.

[0097] In this invention, a loss function is constructed. Train the improved defect detection network.

[0098]

[0099] Where K is the total number of scales used in the multiscale analysis;

[0100] If K=1, it degenerates into a single-scale SIoU;

[0101] This represents the fusion weight at the i-th scale. , For learnable parameters, Indicated in scale The intersection and union ratio below, Indicated in scale Distance matching cost Indicated in scale The cost of shape matching below.

[0102] All of these are marked The parameters with subscripts all represent the i-th scale. The values ​​obtained from the calculation are as follows, where each scale corresponds to a scale factor. , =1,2,3, corresponding to large, medium, and small scales respectively, with values ​​of 0.5, 1.0, and 2.0; through multi-scale analysis and adaptive weight fusion, the scale robustness and detection accuracy of bounding box regression are significantly improved.

[0103] For the improved defect detection network that has been trained, precision (P), recall (R), and mean precision (mAP) are used as evaluation metrics.

[0104] (3) Collect surface images of wind turbine blades, input them into the trained improved defect detection network, and obtain defect category, location and confidence information.

[0105] In this invention, the three detection terminals will output defect category, location and confidence level information respectively, which makes it easier to feed back the detected surface defect information of wind turbine blades to the enterprise in real time, so that the enterprise can take corresponding maintenance or repair measures in a timely manner.

[0106] This invention also relates to a surface defect detection system for wind turbine blades based on the TFPN-YOLO model, comprising:

[0107] At least one processor; and

[0108] A memory that is communicatively connected to at least one of the processors;

[0109] The memory stores instructions that can be executed by the processor to implement the wind turbine blade surface defect detection method based on the TFPN-YOLO model.

[0110] The same wind turbine blade images were processed using the original YOLOV12S model and TFPN-YOLO, respectively. The experimental results are shown in Table 1. The experimental results of the improved defect detection network in this invention are shown in Table 1. Figure 7 As shown, the corresponding defect detection result diagram is as follows. Figure 8 As shown, Figure 8 (a) and (b) show different positions of the wind turbine blades;

[0111] Table 1 Experimental Results

[0112] Accuracy P (%) Recall rate R (%) mAP@0.5(%) mAP@0.5-0.95(%) YOLOV12S 75.5 82 85.1 55.3 TFPN-YOLO 79.7 87 89.7 57.4

[0113] Experimental results demonstrate that, compared to the original model, the improved TPFN-YOLO model, obtained by combining all modules into the original model, achieves a 4.2% improvement in precision, a 5.0% improvement in recall, a 4.6% improvement in mAP@0.5, and a 2.1% improvement in mAP@0.5-0.95. This shows that each innovative module slightly enhances the detection capabilities of the original model, and when combined with the original model, they significantly improve detection precision, recall, and mAP, proving the synergistic effect of the modules.

[0114] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0115] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0116] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0117] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0118] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0119] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for detecting surface defects in wind turbine blades based on the TFPN-YOLO model, characterized in that: An improved defect detection network is constructed, wherein the improved defect detection network is based on the YOLOv12 network; The improved defect detection network includes a Backbone module, a Neck module, and a Head module arranged in sequence. All feature extraction blocks of the base network are replaced with feature extraction blocks modified by Fourier-gated bottleneck convolutions. These feature extraction blocks include sequentially connected CBS blocks and split blocks. Each split block is followed by parallel identity and refined branches. The identity and refined branches are fused by a fully connected layer and then output through a spatial attention network and a CBS block. Each refined branch includes three sequentially connected Fourier-gated bottleneck convolution modules. Each Fourier-gated bottleneck convolution module includes a coordinated Fourier-gated convolution block, which comprises a bottleneck convolution layer, an activation function layer, and a Fourier enhancement block. In the Fourier enhancement block, the output of the Fourier transform layer is output to the channel splicing layer after passing through an average pooling layer and a channel-dimensional averaging layer in the amplitude spectrum branch and phase spectrum branch, respectively. A convolution block follows the channel splicing layer, and the output of the convolution block is element-wise multiplied by the output of the Fourier transform layer. A self-supervised visual backbone network is set after the feature extraction block modified by the second Fourier gated bottleneck convolution of the Backbone module, and the output of the self-supervised visual backbone network is associated with the Neck module; Set up a dynamic multi-scale feature fusion module after A2C2f in the Backbone module and output it to the Neck module; All fully connected blocks of the base network are replaced with a bidirectional feature pyramid network, and several convolutional layers are set up in conjunction with the bidirectional feature pyramid network for cross-channel information integration. A detection head with multi-scale feature fusion is configured in conjunction with the Head module; A dataset of images of the surface of wind turbine blades was acquired, preprocessed, and then used to train the improved defect detection network. A surface image of a wind turbine blade is collected and input into the improved defect detection network after training to obtain defect category, location, and confidence information.

2. The method for detecting surface defects of wind turbine blades based on the TFPN-YOLO model according to claim 1, characterized in that: The self-supervised visual backbone network includes a sequentially connected recombination layer, an upsampling layer, and a convolutional layer.

3. The method for detecting surface defects of wind turbine blades based on the TFPN-YOLO model according to claim 1, characterized in that: The dynamic multi-scale feature fusion module includes parallel multi-scale receptive field branches, cascaded large kernel attention branches, and vertical and horizontal-global context branches. The multi-scale receptive field branch includes sequentially connected CBS blocks and three max-pooling layers; The cascaded large kernel attention branch includes four large kernel attention blocks connected in sequence. The output of the CBS block is the input of the first large kernel attention block. The input of each other large kernel attention block is the sum of the output of the corresponding max pooling layer and the output of the previous large kernel attention block. The vertical-horizontal context branch includes sequentially connected average pooling layers, dynamic convolutional layers, grouped vertical-horizontal convolutional groups, and bilinear upsampling layers; The outputs of the multi-scale receptive field branch, the cascaded large kernel attention branch, and the vertical-horizontal global context branch are concatenated and used as the output of the dynamic multi-scale feature fusion module.

4. The method for detecting surface defects of wind turbine blades based on the TFPN-YOLO model according to claim 1, characterized in that: A TFPN feature fusion channel is established based on the aforementioned bidirectional feature pyramid network; The TFPN feature fusion channel fuses the outputs of the self-supervised visual backbone network and subsequent A2C2f from the first bidirectional feature pyramid network in the Neck module; fuses the outputs of the self-supervised visual backbone network and the feature extraction block modified by the first Fourier-gated bottleneck convolution from the backbone network in the second bidirectional feature pyramid network in the Neck module; fuses the outputs of the first bidirectional feature pyramid network and the two A2C2f outputs from the second bidirectional feature pyramid network in the third bidirectional feature pyramid network in the Neck module; and fuses the outputs of the dynamic multi-scale feature fusion module and the A2C2f output from the third bidirectional feature pyramid network in the fourth bidirectional feature pyramid network in the Neck module.

5. The method for detecting surface defects of wind turbine blades based on the TFPN-YOLO model according to claim 1, characterized in that: The multi-scale feature fusion detection head includes a bounding box regression branch, a classification branch, and a bounding box prediction branch arranged in parallel; The bounding box regression branch, classification branch, and bounding box prediction branch all include sequentially arranged convolutional layers and multiple parallel CBS blocks. The outputs of the multiple parallel CBS blocks are added together and then output through the CBS block and the convolutional layer.

6. The method for detecting surface defects of wind turbine blades based on the TFPN-YOLO model according to claim 1, characterized in that: Constructing the loss function Train the improved defect detection network. ， Where K is the total number of scales used in the multiscale analysis; If K=1, it degenerates into a single-scale SIoU; This represents the fusion weight at the i-th scale. , For learnable parameters, Indicated in scale The intersection and union ratio below, Indicated in scale Distance matching cost Indicated in scale The cost of shape matching below.

7. A surface defect detection system for wind turbine blades based on the TFPN-YOLO model, characterized in that: include: At least one processor; and A memory that is communicatively connected to at least one of the processors; The memory stores instructions that can be executed by the processor to implement the wind turbine blade surface defect detection method based on the TFPN-YOLO model as described in any one of claims 1 to 6.

Images