Wind driven generator blade defect detection method based on RT-DETR
By improving the backbone and neck networks of the RT-DETR model and adopting partial convolution and multi-scale fusion diffusion networks, the problems of high computational complexity and insufficient detection accuracy of RT-DETR in wind turbine blade inspection are solved, and efficient and fast blade defect detection is achieved.
Patent Information
- Application Number
- CN202511794298.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-02
- Publication Date
- 2026-02-24
AI Technical Summary
Existing RT-DETR models suffer from high computational complexity and slow inference speed in wind turbine blade defect detection, and their detection accuracy is limited for targets of different scales. In particular, they are prone to missing small targets or inaccurate positioning of the bounding boxes of large targets in complex scenarios.
By improving the use of partial convolution in the backbone network and designing a multi-scale fusion diffusion network model in the neck network, combined with multi-scale fusion modules and decoders, the model architecture is optimized to reduce computational complexity and improve the detection capability of defects at different scales.
It achieves reduced computational complexity, increased inference speed, and efficient detection of defects of different scales. It can be deployed on embedded devices and improve the detection accuracy of small and large targets in complex scenarios.
Smart Images
Figure CN121564508A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of motor blade inspection technology, specifically to a method for detecting defects in wind turbine blades based on RT-DETR. Background Technology
[0002] In the global new energy revolution, wind energy has gained attention due to its abundant resources, low cost, and high safety. The inspection and maintenance of wind turbine blades is crucial, accounting for one-third of the total operation and maintenance cost. Traditional visual methods are inefficient, while methods based on vibration signals, mechanical waves, SCADA systems, and electromagnetic waves, although improved, suffer from problems such as long processing time, high cost, or poor results. With the development of machine vision and drone technology, drones equipped with high-precision image acquisition equipment can quickly and accurately acquire images of various wind turbine blade surfaces and efficiently identify defects through digital image processing and machine learning technologies, thereby improving inspection efficiency and reducing costs.
[0003] Currently, wind turbine blade defect detection mainly relies on target detection technology. Common models include the YOLO series, Faster R-CNN (Faster Region-based Convolutional Neural Network), and RT-DETR (DETRs Beat YOLOs on Real-time Object Detection). YOLO (You Only Look Once) is a single-stage target detection model that treats detection as a regression problem. After grid generation, bounding box and class prediction, the result is output in a single forward propagation. It is a real-time target detection model, but its adaptability to targets of different scales is poor, and small targets (such as fine cracks) are easily missed. At the same time, its generalization ability in complex environments is significantly limited. Faster R-CNN... R-CNN employs a two-stage detection mechanism, first generating candidate regions using RPN and then classifying and regressing them. While achieving high detection accuracy, it is relatively slow, making it suitable for scenarios with high accuracy requirements and low real-time demands. RT-DETR, an end-to-end target detection model based on Transformer, achieves a certain balance between detection speed and accuracy. However, its high computational complexity results in low inference efficiency, making it difficult to deploy in actual production. Furthermore, RT-DETR has limitations in feature utilization, failing to fully capture and fuse feature information at different scales. This limits its detection accuracy for targets with large size differences (such as small objects or very large targets), especially in complex scenarios, where small targets are easily missed or large targets have inaccurate bounding box localization. Therefore, we propose a wind turbine blade defect detection method based on RT-DETR. Summary of the Invention
[0004] The purpose of this invention is to provide a method for detecting defects in wind turbine blades based on RT-DETR, thereby addressing the problem mentioned in the background art where network information security incidents may lead to service interruptions and significant economic losses. To achieve the above objective, this invention provides a method for detecting defects in wind turbine blades based on RT-DETR, wherein the management method includes:
[0005] Step 1: Overall architecture of the LMFD-DETR model;
[0006] Step 2: Backbone network improvement;
[0007] Step 3: Neck network improvement;
[0008] Step 4: Multi-scale fusion module (MSFBlock);
[0009] Step 5: Multi-scale fusion and diffusion;
[0010] Preferably, the proposed LMFD-DETR (Light Multi-scale Fusion Diffusion DETR) model is an improvement on RT-DETR, aiming to improve the efficiency and accuracy of wind turbine blade defect detection. This model mainly improves the backbone network and neck network, thereby reducing computational complexity, increasing inference speed, and enhancing the ability to detect defects at different scales. The overall architecture includes a backbone network, a neck network (Multi-scale Fusion Diffusion Network model MFDN), and a decoder, etc., and completes the blade defect detection task through feature extraction, multi-scale fusion diffusion, and other processes.
[0011] Preferably, to address the issues of high computational complexity and slow inference speed of RT-DETR, the backbone network is improved by replacing the standard convolution in BasicBlock with partial convolution (PConv). In practical applications, there is high similarity between different channels of feature maps, leading to feature redundancy and increased model cost. PConv only applies regular convolution to a portion of the input channels for spatial feature extraction, keeping the remaining channels unchanged. According to relevant formulas, when the number of channels processed is 1 / 4c, the computational cost (FLOPs) of PConv is only 1 / 16 of that of standard convolution, and the memory access cost (MAC) is 1 / 4 of that of standard convolution, effectively reducing computational overhead while maintaining high feature representation capability.
[0012] Preferably, to address the issue of RT-DETR missing detections of small targets, an efficient multi-scale fusion diffusion network (MFDN) model is designed in the neck network to replace the original feature fusion network. The backbone network outputs feature maps of three different scales: s3, s4, and s5. S5 is processed by AIFI Attention-based Intra-scale Feature Interaction to generate F5. The outputs of the backbone network, s3 and s4, are collectively referred to as F3 and F4. F3, F4, and F5 are input to the multi-scale fusion module (MSFBlock), which is an important component of MFDN. Through the fusion and processing of multi-scale features, the model's ability to detect defects of different scales is improved.
[0013] Preferably, MSFBlock allows input of feature maps at three different scales. The specific operation is as follows: s5 is upsampled, and s3 is downsampled using the lightweight downsampler ADown; the upsampled s5, downsampled s3, and s4 are subjected to a ConCat operation; then, a set of parallel depthwise separable convolutions (DWConv) are used to extract semantic information of different sizes. Considering that wind turbine blade defects exist at different scales, four convolution kernel sizes are set: 5×5, 7×7, 9×9, and 11×11. The 5×5 convolution kernel has a small receptive field and can identify microscopic defects such as fine cracks and small holes; the 9×9 and 11×11 convolution kernels expand the receptive field and can obtain complete contextual information, helping to determine the defect type and severity, and enabling the model to learn richer and more comprehensive feature representations.
[0014] Preferably, to detect targets at different scales, the multi-scale fusion module (MSFBlock) is diffused to different sizes. The specific process is as follows: the feature information output by the feature focusing module is subjected to 3×3 convolution and downsampling, and then ConCat operation is performed with s5 and s3 respectively, and then input into the RepC3 module; this process is repeated twice, and finally the processed features are input into the decoder. Through multiple rounds of diffusion and fusion, the detection accuracy of the model for targets at different scales is further enhanced, especially the detection effect of small and large targets in complex scenes.
[0015] Compared with the prior art, the beneficial effects of the present invention are as follows:
[0016] In this invention, addressing the high computational complexity and slow inference speed of RT-DETR, the invention replaces the original standard convolutions in the backbone network with partial convolutions. These partial convolutions perform grouped convolution operations on the input feature maps, effectively avoiding redundant computations in standard convolutions while preserving key semantic information and detailed features. This significantly reduces computational overhead while maintaining high feature representation capabilities. From a practical application perspective, on the one hand, the model's dependence on hardware resources is significantly reduced, eliminating the need for long-term support from high-performance GPUs and enabling smooth deployment on embedded devices. On the other hand, the reduction in computational complexity directly translates into increased inference speed. In fields with high real-time requirements, such as dynamic object detection, real-time image segmentation, and online quality detection, the model can quickly process input data and output results, avoiding decision lag caused by computational delays. Furthermore, the reduced number of parameters lowers the model training cost, enabling stable training results even with small sample data.
[0017] In this invention, to address the issue of missed detection of small targets in RT-DETR, a highly efficient Multi-scale Fusion Diffusion Network (MFDN) model is designed. RT-DETR uses a CNN-based Cross-scale Feature Fusion (CCFF) in the encoder. This structure is a bidirectional cascaded process of "bottom-up + top-down": starting from shallow features, it gradually incorporates deeper semantic information, and then uses the fused deep features as a guide to enhance the semantic expression of shallow features in reverse. However, since CCFF's bidirectional fusion is based on the global interaction of the "whole layer feature map", it does not distinguish between "effective regions (such as small targets, foreground)" and "ineffective regions (such as blank background)" in the features, resulting in the dilution of small target features, thus affecting the model's accuracy. The feature aggregation and diffusion network designed in this invention, by first aggregating feature maps and fully extracting semantic information, uses a diffusion mechanism to diffuse features with rich contextual information to various detection scales, improving the model's ability to detect different targets in complex backgrounds, and significantly improving the model's accuracy. Attached Figure Description
[0018] Figure 1 This is a schematic diagram of the process structure of the present invention;
[0019] Figure 2 This is a flowchart of the model training process of the present invention;
[0020] Figure 3 This is a flowchart of the LMFD-DETR model of the present invention;
[0021] Figure 4 This is a flowchart of the PConv_Block process of the present invention;
[0022] Figure 5 This is a diagram of the multi-scale fusion module of the present invention;
[0023] Figure 6 This is a diagram of the multi-scale fusion diffusion model of the present invention;
[0024] Figure 7 This is a schematic diagram of the RepC3 structure of the present invention;
[0025] Figure 8 This is a schematic diagram of the RepC3_Block structure of the present invention. Detailed Implementation
[0026] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0027] Please see Figure 1 This invention provides a technical solution: a method for detecting defects in wind turbine blades based on RT-DETR, the management method of which includes:
[0028] Step 1: Overall architecture of the LMFD-DETR model;
[0029] Step 2: Backbone network improvement;
[0030] Step 3: Neck network improvement;
[0031] Step 4: Multi-scale fusion module (MSFBlock);
[0032] Step 5: Multi-scale fusion and diffusion;
[0033] In step one, the proposed LMFD-DETR (Light Multi-scale Fusion Diffusion DETR) model is an improvement on RT-DETR, aiming to improve the efficiency and accuracy of wind turbine blade defect detection. This model mainly achieves reduced computational complexity, increased inference speed, and enhanced defect detection capabilities at different scales by improving the backbone network and neck network. The overall architecture includes a backbone network, a neck network (Multi-scale Fusion Diffusion Network model MFDN), and a decoder, etc., and completes the blade defect detection task through feature extraction, multi-scale fusion diffusion, and other processes.
[0034] In step two, to address the issues of high computational complexity and slow inference speed of RT-DETR, the backbone network is improved by replacing the standard convolutions in BasicBlock with partial convolutions (PConv). In practical applications, there is high similarity between different channels of feature maps, leading to feature redundancy and increased model cost. PConv only applies regular convolutions to a portion of the input channels for spatial feature extraction, keeping the remaining channels unchanged. According to relevant formulas, when the number of channels processed is 1 / 4c, the computational cost (FLOPs) of PConv is only 1 / 16 of that of standard convolution, and the memory access cost (MAC) is 1 / 4 of that of standard convolution, effectively reducing computational overhead while maintaining high feature representation capability.
[0035] In step three, to address the issue of RT-DETR missing detections of small targets, an efficient multi-scale fusion diffusion network (MFDN) model was designed in the neck network to replace the original feature fusion network. The backbone network outputs feature maps of three different scales: s3, s4, and s5. S5 is processed by AIFI (Attention-based Intra-scale Feature Interaction) to generate F5. The outputs of the backbone network, s3 and s4, are collectively referred to as F3 and F4. F3, F4, and F5 are input to the multi-scale fusion module (MSFBlock), which is an important component of MFDN. Through the fusion and processing of multi-scale features, the model's ability to detect defects of different scales is improved.
[0036] In step four, MSFBlock allows input of feature maps at three different scales. The specific operations are as follows: s5 is upsampled, and s3 is downsampled using the lightweight downsampler ADown; the upsampled s5, downsampled s3, and s4 are subjected to a ConCat operation; then, a set of parallel depthwise separable convolutions (DWConv) are used to extract semantic information of different sizes. Considering that wind turbine blade defects exist at different scales, four convolution kernel sizes of 5×5, 7×7, 9×9, and 11×11 are set. The 5×5 convolution kernel has a small receptive field and can identify microscopic defects such as fine cracks and small holes; the 9×9 and 11×11 convolution kernels expand the receptive field and can obtain complete contextual information, helping to determine the defect type and severity, and enabling the model to learn richer and more comprehensive feature representations.
[0037] In step five, to detect targets at different scales, the multi-scale fusion module (MSFBlock) is diffused to different sizes. Specifically, the feature information output by the feature focusing module is subjected to 3×3 convolution and downsampling, and then ConCat operations are performed with s5 and s3 respectively, followed by input to the RepC3 module. This process is repeated twice, and finally the processed features are input into the decoder. Through multiple rounds of diffusion and fusion, the detection accuracy of the model for targets at different scales is further enhanced, especially the detection effect of small and large targets in complex scenes.
[0038] The method of use and advantages of this invention: The working process of this RT-DETR-based wind turbine blade defect detection method is as follows:
[0039] First, the backbone network is improved by replacing the standard convolutions in BasicBlock with partial convolutions. In practical applications, feature maps have high similarity across different channels, and the redundancy of feature maps further increases model cost. PConv only needs to apply regular Convolutions to a portion of the input channels for spatial feature extraction, while keeping the remaining channels unchanged. The computational cost (FLOPs) and memory access cost (MAC) of PConv are shown in Equations 1 and 2.
[0040]
[0041] Standard convolution is shown in Equations 3 and 4:
[0042] F PConv =h×w×k 2 ×c 2 (3)
[0043] M PConv =h×w×2c+k 2 ×c 2 ≈h×w×2c, (4)
[0044] Where h and w correspond to the height and width of the feature map, respectively, k represents the size of the convolution kernel, and c represents the number of channels. As seen in the above formula, when c... pWhen the frequency is 1 / 4c, PConv's FLOPs are only 1 / 16 of standard convolutions, and the memory access is 1 / 4 of that of convolutions. In the neck network, this invention designs an efficient multi-scale fusion diffusion network (MFDN) model to replace the original feature fusion network. The specific steps are as follows: the backbone network outputs feature maps of three different scales, labeled as s3, s4, and s5 respectively. s5 is processed by AIFI (Attention-based Intra-scale Feature Interaction) to generate F5. (For ease of consistent terminology, s3 and s4 output by the backbone network are changed to F3 and F4). F3, F4, and F5 are then input together into the multi-scale fusion module (Multi-Scale Fusion). The Block (MSFBlock) is a crucial component of the Multi-Scale Fusion Diffusion Network (MFDN) model. It allows input of feature maps at three different scales. It upsamples s5, downsamples s3 using a lightweight downsampler ADown, and then performs a CanCat operation on the upsampled s5, downsampled s3, and s4. Finally, it passes these inputs through a set of parallel depthwise separable convolutions. This invention employs a convolutional model (DWConv) to extract semantic information at different sizes. Since wind turbine blade defects exist at various scales, this invention designs DW convolutions of different sizes, with kernel sizes set to 5, 7, 9, and 11 respectively. A 5×5 depthwise separable convolution kernel can effectively identify local details in blade defects, such as microscopic defects like fine cracks and small holes, due to its smaller receptive field, allowing it to focus on each pixel and its neighborhood in the feature map, thus exhibiting strong representation of local features. Subsequently, this paper uses 9×9 and 11×11 convolution kernels to expand the receptive field, obtaining complete contextual information. The model can better understand the changes in the overall blade structure, thereby more accurately determining the type and severity of defects. In summary, this paper sets up a set of parallel depthwise separable convolutions in the feature fusion module, enabling the model to learn richer and more comprehensive feature representations. To detect targets at different scales, this invention designs a multi-scale fusion diffusion network model, incorporating a multi-scale fusion module (Multi-scalefusion...) The block is diffused to different sizes. First, the feature information output by the feature focusing module is convolved and downsampled by 3×3 respectively. Then, it is fed into the RepC3 module after being fed into the CanCat operation with s5 and s3 respectively. This process is repeated twice and finally fed into the Decoder.
[0045] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely preferred examples and are not intended to limit the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.
Claims
1. A method for detecting defects in wind turbine blades based on RT-DETR, characterized in that: The management methods include: Step 1: Overall architecture of the LMFD-DETR model; Step 2: Backbone network improvement; Step 3: Neck network improvement; Step 4: Multi-scale fusion module (MSFBlock); Step 5: Multi-scale fusion and diffusion; In step one, the proposed LMFD-DETR (Light Multi-scale Fusion Diffusion DETR) model is an improvement on RT-DETR, aiming to improve the efficiency and accuracy of wind turbine blade defect detection. This model mainly achieves reduced computational complexity, increased inference speed, and enhanced defect detection capabilities at different scales by improving the backbone network and neck network. The overall architecture includes a backbone network, a neck network (Multi-scale Fusion Diffusion Network model MFDN), and a decoder, etc., and completes the blade defect detection task through feature extraction, multi-scale fusion diffusion, and other processes.
2. The method for detecting defects in wind turbine blades based on RT-DETR according to claim 1, characterized in that: In step two, to address the issues of high computational complexity and slow inference speed of RT-DETR, the backbone network is improved by replacing the standard convolutions in BasicBlock with partial convolutions (PConv). In practical applications, there is high similarity between different channels of the feature map, leading to feature redundancy and increased model cost. PConv only applies regular convolutions to a portion of the input channels for spatial feature extraction, keeping the remaining channels unchanged. According to relevant formulas, when the number of channels processed is 1 / 4c, the computational cost (FLOPs) of PConv is only 1 / 16 of that of standard convolution, and the memory access cost (MAC) is 1 / 4 of that of standard convolution, effectively reducing computational overhead while maintaining high feature representation capability.
3. The method for detecting defects in wind turbine blades based on RT-DETR according to claim 2, characterized in that: In step three, to address the issue of RT-DETR missing detections of small targets, an efficient multi-scale fusion diffusion network (MFDN) model was designed in the neck network to replace the original feature fusion network. The backbone network outputs feature maps of three different scales: s3, s4, and s5. S5 is processed by AIFI (Attention-based Intra-scale Feature Interaction) to generate F5. The outputs of the backbone network, s3 and s4, are collectively referred to as F3 and F4. F3, F4, and F5 are input to the multi-scale fusion module (MSFBlock), which is an important component of MFDN. Through the fusion and processing of multi-scale features, the model's ability to detect defects of different scales is improved.
4. The method for detecting defects in wind turbine blades based on RT-DETR according to claim 3, characterized in that: In step four, MSFBlock allows input of feature maps at three different scales. The specific operations are as follows: s5 is upsampled, and s3 is downsampled using the lightweight downsampler ADown; the upsampled s5, downsampled s3, and s4 are subjected to a ConCat operation; then, a set of parallel depthwise separable convolutions (DWConv) are used to extract semantic information of different sizes. Considering that wind turbine blade defects exist at different scales, four convolution kernel sizes are set: 5×5, 7×7, 9×9, and 11×11. The 5×5 convolution kernel has a small receptive field and can identify microscopic defects such as fine cracks and small holes; the 9×9 and 11×11 convolution kernels expand the receptive field and can obtain complete contextual information, helping to determine the defect type and severity, and enabling the model to learn richer and more comprehensive feature representations.
5. The method for detecting defects in wind turbine blades based on RT-DETR according to claim 4, characterized in that: In step five, to detect targets at different scales, the multi-scale fusion module (MSFBlock) is diffused to different sizes. Specifically, the feature information output by the feature focusing module is subjected to 3×3 convolution and downsampling, and then ConCat operations are performed with s5 and s3 respectively, followed by input to the RepC3 module. This process is repeated twice, and finally the processed features are input into the decoder. Through multiple rounds of diffusion and fusion, the detection accuracy of the model for targets at different scales is further enhanced, especially the detection effect of small and large targets in complex scenes.