Image processing method for detecting defects of wind power blade in real time

Abstract

The invention provides an image processing method for wind power blade defect real-time detection, and relates to the technical field of wind power equipment operation and maintenance and machine vision crossing. An unmanned aerial vehicle carries a high-speed camera to shoot a wind power blade rotating at a high speed, and a blade surface image is obtained; constructing a U-net semantic segmentation model, segmenting the obtained blade surface image, and extracting a wind power blade contour feature image; establishing a fan blade defect data set based on the wind power blade contour feature image; yOLOv7 is adopted as a fan blade defect detection model, an attention mechanism module is introduced, and the defect detection model is optimized and improved; and based on the established fan blade defect data set, training the improved model, inputting the wind power blade contour feature image into the trained model, and outputting the blade defect type and position information. According to the method, the defect recognition rate is not lower than 90%, and the requirements of real-time and high-precision detection and efficient data management in a wind power blade operation and maintenance scene can be met.

Description

Technical Field

[0001] This invention relates to the field of wind power equipment operation and maintenance and machine vision technology. Specifically, it relates to an image processing method for real-time detection of defects in wind turbine blades, which is particularly suitable for real-time detection and data management of defects such as surface corrosion, gel coat peeling, and protective film damage on wind turbine blades in wind farms. Background Technology

[0002] With the rapid development of the wind power industry, wind turbine blades, as the core component of wind turbine units, directly affect the power generation efficiency and safety of the units. Wind turbine blades are exposed to complex outdoor environments (such as strong winds, rain, snow, and ultraviolet radiation) for extended periods, making them prone to surface defects such as corrosion, gel coat peeling, and damage to the protective film. If these defects are not detected and maintained in a timely manner, they may expand, leading to serious accidents such as blade breakage.

[0003] Currently, wind turbine blade defect detection mainly relies on two methods:

[0004] Manual inspection: Maintenance personnel observe the blade surface at close range by climbing the tower or operating drones. This method is inefficient, labor-intensive, and poses safety risks for working at heights, making it difficult to meet the maintenance needs of large-scale wind farms.

[0005] Machine vision inspection: Automated inspection based on image processing and deep learning technologies;

[0006] However, existing technologies have the following shortcomings:

[0007] (1) Complex background interference: The original image of the blade contains background information such as the sky and the tower. Traditional models have difficulty effectively extracting blade features, resulting in a high false detection rate of defects.

[0008] (2) Insufficient feature extraction capability: Existing detection models (such as the basic YOLOv7) have weak feature capture capability for subtle defects on the blade surface (such as early corrosion), and the detection accuracy is usually less than 80%.

[0009] (3) Poor real-time performance: The model parameters are redundant and the training efficiency is low, making it difficult to meet the needs of real-time wind field inspection.

[0010] Therefore, there is an urgent need for an image processing method that can achieve accurate extraction of blade contours, high-precision defect detection, real-time model inference, and efficient data management to solve the above-mentioned technical pain points. Summary of the Invention

[0011] To address the aforementioned technical problems, this invention provides an image processing method for real-time detection of defects in wind turbine blades. The invention uses a UAV equipped with a high-speed camera to capture images of the wind turbine blade surface, extracts blade contour features using the U-net semantic segmentation model, and combines this with an improved YOLOv7 model incorporating a CBAM attention mechanism module for defect detection, thereby achieving real-time, high-precision detection of wind turbine blade defects.

[0012] The technical means employed in this invention are as follows:

[0013] An image processing method for real-time detection of defects in wind turbine blades, comprising:

[0014] S1. Use a drone equipped with a high-speed camera to photograph the high-speed rotating wind turbine blades and obtain images of the blade surface;

[0015] S2. Construct the U-net semantic segmentation model to segment the acquired blade surface image and extract the wind turbine blade contour feature image;

[0016] S3. Based on the contour feature images of wind turbine blades, establish a dataset of wind turbine blade defects;

[0017] S4. YOLOv7 is used as the defect detection model for wind turbine blades. At the same time, the CBAM attention mechanism module is introduced to optimize and improve the YOLOv7 defect detection model.

[0018] S5. Based on the established wind turbine blade defect dataset, the improved YOLOv7 defect detection model is trained. The wind turbine blade contour feature image is input into the trained model, and the blade defect type and location information are output.

[0019] Further, step S2 includes:

[0020] S21. In the encoding stage, the U-net semantic segmentation model extracts different semantic and spatial features from the blade surface image layer by layer.

[0021] S22. In the decoding stage, the different semantic and spatial features extracted from the blade surface image are continuously upsampled to reconstruct resolution details close to the original image, resulting in accurate image segmentation results.

[0022] S23. The feature information between the encoding and decoding stages is connected by horizontal jumps to integrate features at different semantic levels and obtain the wind turbine blade outline feature image.

[0023] Furthermore, the U-net semantic segmentation model introduces a cross-entropy loss function in the image segmentation task. When processing each pixel location in the image, the output result is treated as a binary classification problem, that is, determining whether the pixel belongs to the target category (foreground) or the non-target category (background). The cross-entropy loss function is as follows:

[0024]

[0025] Where N represents the total number of pixels; y i The label representing reality, y i The value is 0 or 1; Loss represents the output result. This represents the predicted probability.

[0026] Further, step S3 includes:

[0027] S31. Classify the surface defects and damage characteristics of wind turbine blades according to geometric features, texture features, and color features;

[0028] S32. Three types of wind turbine defects and damages are used as the detection objects: corrosion, blade gel coat peeling, and protective film damage.

[0029] S33. Use the Labelme annotation tool to annotate the wind turbine blade defect data to obtain a wind turbine blade damage image dataset with category labels;

[0030] S34. Based on image enhancement technology, the wind turbine blade damage image dataset with category labels is expanded to obtain a wind turbine blade defect dataset.

[0031] Furthermore, in step S31:

[0032] The geometric features include crack-like features, hole-like features, and deformation-like features. Crack-like features include linear cracks, bifurcated cracks, and network cracks; hole-like features include circular holes and irregular holes; and deformation-like features include bending deformation, torsional deformation, and local depressions.

[0033] The texture features include surface roughness features, layering features, and texture disorder features. Surface roughness features include wear and corrosion; layering features include internal layering and surface coating layering; texture disorder features include repairing texture disorder and manufacturing defects.

[0034] The color characteristics include discoloration characteristics, color spot characteristics, and color unevenness characteristics. Among them, discoloration characteristics include overall discoloration and local discoloration; color spot characteristics include pollution spots and corrosion spots; and color unevenness characteristics include those caused by manufacturing processes and those caused by damage repair.

[0035] Further, step S34 includes:

[0036] The leaf image data is defined as a function of two mathematical variables, represented by the horizontal and vertical coordinates of the image region, as follows:

[0037] Ω = {p = [x, y]} T , I(x,y):0≤x<W,0≤y<H}

[0038] Where W represents the width; H represents the height; p = [x, y] T Representing a pixel, I(x,y)=I(p)=[r,g,b] p The color value at image point p is represented by homogeneous coordinates p = [x, y, 1]. T Represents point p in the image;

[0039] Define the affine transformation in the image as follows:

[0040]

[0041] in, Let A represent the transformed second coordinates, and let A represent the affine transformation matrix. M controls the adjustment of image enhancement, and t controls the translation transformation.

[0042] Furthermore, in step S4, the CBAM attention mechanism module is added to the backbone of YOLOv7, so that when the model enters the backbone architecture network from the input end, it first selects the region of interest that the model is interested in.

[0043] Furthermore, the CBAM attention mechanism module includes a channel attention module and a spatial attention module, wherein:

[0044] The calculation formula for the channel attention module is as follows:

[0045]

[0046] For a feature map of size H×W×C, the channel attention module first receives the output features of the previous convolutional layer, i.e., the data of the feature map in the 1×1×C channel dimension, and performs a weighted operation on it. Then, it adaptively assigns weights to the input features through two convolutional operations: max pooling and average pooling, to extract global feature information. After the pooling operation, the feature map is converted into two one-dimensional vectors, removing spatial dimensional redundancy while retaining channel dimension information. Finally, the two one-dimensional vectors are input into a multilayer perceptron for processing. The output features are then fused through element-wise addition to generate a new feature vector. The fused feature vector is activated by the Sigmoid activation function to generate a channel-connected feature map. In the above formula, σ represents the Sigmoid function; W0 and W1 represent the weights of the fully connected layer and the output layer, respectively; MLP represents the multilayer perceptron, which implements non-linear function fitting of the output. and To obtain two C-dimensional pooled feature maps by performing average pooling and max pooling on the input feature layer respectively; M c (F) By obtaining the channel attention vector, the weight parameters of the multiple channels are adjusted appropriately;

[0047] The calculation formula for the spatial attention module is as follows:

[0048]

[0049] For a spatial feature map of size H×W×1, a spatial attention module is used to implement an adaptive weight allocation mechanism in the image to highlight key quality regions. The weighting coefficients generated by the channel attention mechanism are multiplied element-wise with the original input feature map. Then, the processed feature map is further processed through max pooling and average pooling to generate two independent feature maps, which are then stacked and convolved to reduce the feature dimension and output an H×W×1 feature map. Finally, a spatial attention feature map is generated through an activation function, and it is multiplied element-wise with the original input features again to obtain the final output feature.

[0050] Furthermore, in step S5, during the training of the improved YOLOv7 defect detection model, loss functions are designed, including localization loss, classification loss, and target confidence loss, which are used to evaluate bounding box prediction, classification prediction, and confidence prediction, respectively. The process is as follows:

[0051] The localization loss function is designed as follows:

[0052] Loss box =(xx) * ) 2 +(yy * )2 +(ww * ) 2 +(hh * ) 2

[0053] Where x, y, w, and h represent the coordinates of the top-left corner of the prediction box and the length and width of the prediction box, respectively;

[0054] The classification loss function is designed as follows:

[0055]

[0056] Among them, y i The predicted class probability after the Sigmoid function. The true value (0 or 1) for the category;

[0057] The target confidence loss function is designed as follows:

[0058]

[0059] Where K represents the network feature output, divided into a K×K grid; M is the number of anchor boxes in each grid. For a target anchor box, For anchor boxes without target objects, λ noobj These are the weighting coefficients;

[0060] Based on the localization loss, classification loss, and target confidence loss designed in the design, the total loss function is designed as follows:

[0061] Loss = Loss box +Loss cla +Loss obj

[0062] Among them, Loss box This represents the localization loss function; Loss cla Represents the classification loss function; Loss obj This represents the target confidence loss function.

[0063] Compared with the prior art, the present invention has the following advantages:

[0064] 1. The present invention provides an image processing method for real-time detection of defects in wind turbine blades. By using a drone equipped with a high-speed camera to capture images of the surface of the wind turbine blades, high-definition images of the blade surface can be obtained quickly and efficiently. This avoids the low efficiency and high risk of manually climbing the tower or operating a drone to observe the blade surface at close range, and significantly improves detection efficiency and safety.

[0065] 2. The present invention provides an image processing method for real-time detection of wind turbine blade defects. It utilizes the U-net semantic segmentation model to extract blade contour features. This model can extract different semantic and spatial features from the blade surface image layer by layer, and integrate different semantic level features through lateral jump connections to accurately reconstruct resolution details close to the original image, thereby achieving accurate extraction of wind turbine blade contours, effectively solving the problem of complex background interference, and reducing the false detection rate of defects.

[0066] 3. The image processing method for real-time detection of wind turbine blade defects provided by this invention classifies the surface defect damage features of wind turbine blades according to geometric features, texture features and color features, and uses Labelme annotation tool for annotation. Then, the dataset is expanded through image enhancement technology, which provides rich and accurate annotation data for model training, and improves the generalization ability and detection accuracy of the model.

[0067] 4. The image processing method for real-time detection of wind turbine blade defects provided by the present invention adopts an improved YOLOv7 model with the introduction of the CBAM attention mechanism module for defect detection. The CBAM attention mechanism module assigns weights to the feature map through channel attention and spatial attention, which improves the model’s attention to key feature areas, enhances the feature capture ability of the blade surface for minor defects, and significantly improves the detection accuracy, enabling the effective detection of minor defects such as early corrosion.

[0068] 5. The present invention provides an image processing method for real-time detection of defects in wind turbine blades. It designs a total loss function that includes localization loss, classification loss and target confidence loss, which is used to evaluate bounding box prediction, classification prediction and confidence prediction. This optimizes the model training process and further improves the accuracy and reliability of the model.

[0069] In summary, this invention enables real-time, high-precision detection of wind turbine blade defects, which not only improves detection efficiency and accuracy and reduces false detection rate, but also meets the needs of real-time wind farm inspection. It provides strong technical support for the operation and maintenance management of wind turbine blades, helps to promptly detect and handle blade defects, ensures the safe operation of wind turbine units, improves power generation efficiency, and reduces operation and maintenance costs. Attached Figure Description

[0070] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0071] Figure 1 This is a flowchart of the method of the present invention.

[0072] Figure 2 The diagram shows the structure of the U-net semantic segmentation model provided in this embodiment of the invention.

[0073] Figure 3 This is a classification diagram of wind turbine blade defect damage features provided in an embodiment of the present invention.

[0074] Figure 4 Geometric feature class defect diagrams provided for embodiments of the present invention.

[0075] Figure 5 This is a texture feature class defect map provided in an embodiment of the present invention.

[0076] Figure 6 This is a coating peeling diagram in the color features provided in an embodiment of the present invention.

[0077] Figure 7 The Labelme annotation interface provided in this embodiment of the invention.

[0078] Figure 8 This is a schematic diagram of the defect labeling process provided in an embodiment of the present invention.

[0079] Figure 9 This is a schematic diagram of blade data enhancement provided in an embodiment of the present invention.

[0080] Figure 10 This is a diagram of the improved YOLOv7 network structure provided in an embodiment of the present invention.

[0081] Figure 11 The channel attention mechanism and spatial attention mechanism provided in the embodiments of the present invention.

[0082] Figure 12 This is a CBAM network architecture diagram provided for an embodiment of the present invention.

[0083] Figure 13 The loss function curve provided for an embodiment of the present invention. Detailed Implementation

[0084] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0085] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0086] like Figure 1 As shown, the present invention provides an image processing method for real-time detection of defects in wind turbine blades, including:

[0087] S1. Use a drone equipped with a high-speed camera to photograph the high-speed rotating wind turbine blades and obtain images of the blade surface;

[0088] S2. Construct the U-net semantic segmentation model to segment the acquired blade surface image and extract the wind turbine blade contour feature image;

[0089] S3. Based on the contour feature images of wind turbine blades, establish a dataset of wind turbine blade defects;

[0090] S4. YOLOv7 is used as the defect detection model for wind turbine blades. At the same time, the CBAM attention mechanism module is introduced to optimize and improve the YOLOv7 defect detection model.

[0091] S5. Based on the established wind turbine blade defect dataset, the improved YOLOv7 defect detection model is trained. The wind turbine blade contour feature image is input into the trained model, and the blade defect type and location information are output.

[0092] In a specific implementation, as a preferred embodiment of the present invention, step S2 includes:

[0093] S21. In the encoding stage, the U-net semantic segmentation model extracts different semantic and spatial features from the blade surface image layer by layer.

[0094] S22. In the decoding stage, the different semantic and spatial features extracted from the blade surface image are continuously upsampled to reconstruct resolution details close to the original image, resulting in accurate image segmentation results.

[0095] S23. The feature information between the encoding and decoding stages is connected by horizontal jumps to integrate features at different semantic levels and obtain the wind turbine blade outline feature image.

[0096] In this embodiment, as Figure 2 As shown, U-net, a fully convolutional neural network model, employs a unique topological structure combining encoders and decoders with skip connections, enabling effective feature fusion through concatenation. This structure boasts end-to-end training capabilities, is not only simple and easy to implement, but also demonstrates good stability in practical engineering applications. This characteristic allows U-net to achieve relatively accurate segmentation results even when processing small image datasets.

[0097] In a specific implementation, as a preferred embodiment of the present invention, the U-net semantic segmentation model introduces a cross-entropy loss function in the image segmentation task. When processing each pixel position of the image, the output result is treated as a binary classification problem, that is, determining whether the pixel belongs to the target category (foreground) or the non-target category (background). The cross-entropy loss function is as follows:

[0098]

[0099] Where N represents the total number of pixels; y i The label representing reality, y i The value is 0 or 1; Loss represents the output result. This represents the predicted probability.

[0100] In a specific implementation, as a preferred embodiment of the present invention, step S3 includes:

[0101] S31, such as Figure 3 As shown, the surface defect damage characteristics of wind turbine blades are classified according to geometric features, texture features, and color features;

[0102] S32. Three types of wind turbine defects and damages are used as the detection objects: corrosion, blade gel coat peeling, and protective film damage.

[0103] S33. Use the Labelme annotation tool to annotate the wind turbine blade defect data to obtain a wind turbine blade damage image dataset with category labels;

[0104] S34. Based on image enhancement technology, the wind turbine blade damage image dataset with category labels is expanded to obtain a wind turbine blade defect dataset.

[0105] In this embodiment, a total of 4,500 images of wind turbine blades were collected, which were then expanded to 7,000 images using image enhancement technology. Finally, the Labelme annotation tool was used to annotate 5,000 images, resulting in 5,269 bounding boxes, categorized into three types: corrosion, gelcoat off, and protective film damage. The annotated images were then divided into training and testing sets in a 7:3 ratio. Table 1 shows the data for the three types of defects and damage in the training and testing sets, and the statistics for the wind turbine blade dataset.

[0106] Table 1. Statistics of Wind Turbine Blade Dataset

[0107]

[0108] In a specific implementation, as a preferred embodiment of the present invention, in step S31:

[0109] The geometric features include crack-like features, void-like features, and deformation-like features. Crack-like features include linear cracks, bifurcated cracks, and network cracks; void-like features include circular voids and irregular voids; deformation-like features include bending deformation, torsional deformation, and localized depressions. Figure 4 The image shows crack-type features and hole-type features.

[0110] The texture features include surface roughness features, layering features, and texture disorder features. Surface roughness features include wear and corrosion; layering features include internal layering and surface coating layering; texture disorder features include repairing texture disorder and manufacturing defects. Figure 5 As shown, this is a texture feature type defect.

[0111] The color characteristics include discoloration characteristics, color spot characteristics, and color unevenness characteristics. Discoloration characteristics include overall discoloration and localized discoloration; color spot characteristics include contaminated color spots and corroded color spots; color unevenness characteristics include those caused by manufacturing processes and those caused by damage repair. Figure 6 The diagram shown illustrates the coating peeling process.

[0112] In a preferred embodiment of the present invention, in step S32, the Labelme annotation tool is used to annotate the acquired defect image. The Labelme annotation interface is as follows: Figure 7 As shown, to ensure the sample quality of the dataset, the resulting data was imported into Labelme for manual review to guarantee the accuracy of the labels. The leaf labeling process is as follows: Figure 8 As shown.

[0113] In a specific implementation, as a preferred embodiment of the present invention, step S34 includes:

[0114] The leaf image data is defined as a function of two mathematical variables, represented by the horizontal and vertical coordinates of the image region, as follows:

[0115] Ω = {p = [x, y]} T , I(x,y):0≤x<W,0≤y<H}

[0116] Where W represents the width; H represents the height; p = [x, y] T Representing a pixel, I(x,y)=I(p)=[r,g,b] p The color value at image point p is represented by homogeneous coordinates p = [x, y, 1]. T Represents point p in the image;

[0117] Define the affine transformation in the image as follows:

[0118]

[0119] in, Let A represent the transformed second coordinates, and let A represent the affine transformation matrix. M controls the adjustment of image enhancement, and t controls the translation transformation.

[0120] In this embodiment, data augmentation techniques enrich the diversity of the wind turbine blade dataset, improving the learning and training efficiency and detection accuracy of the deep learning model. This effectively alleviates the class imbalance problem, enhancing the stability of subsequent model parameters when facing diverse defect types. The wind turbine blade defect image augmentation technique is as follows: Figure 9 As shown.

[0121] In a specific implementation, as a preferred embodiment of the present invention, in step S4, the CBAM attention mechanism module is added to the YOLOv7 backbone, so that when the model enters the backbone architecture network from the input end, it first filters out the regions of interest that the model is interested in. In this embodiment, as... Figure 10 As shown, by integrating the CBAM attention mechanism module with YOLOv7, corresponding weights are assigned to channel features and spatial features in the feature map to improve attention to visual objects and suppress attention to non-objects, thereby improving the overall accuracy of the object detection model.

[0122] In specific implementation, as a preferred embodiment of the present invention, such as Figure 11 , 12 As shown, the CBAM attention mechanism module includes a channel attention module and a spatial attention module, wherein:

[0123] The calculation formula for the channel attention module is as follows:

[0124]

[0125] For a feature map of size H×W×C, the channel attention module first receives the output features of the previous convolutional layer, i.e., the data of the feature map in the 1×1×C channel dimension, and performs a weighted operation on it. Then, it adaptively assigns weights to the input features through two convolutional operations: max pooling and average pooling, to extract global feature information. After the pooling operation, the feature map is converted into two one-dimensional vectors, removing spatial dimensional redundancy while retaining channel dimension information. Finally, the two one-dimensional vectors are input into a multilayer perceptron for processing. The output features are then fused through element-wise addition to generate a new feature vector. The fused feature vector is activated by the Sigmoid activation function to generate a channel-connected feature map. In the above formula, σ represents the Sigmoid function; W0 and W1 represent the weights of the fully connected layer and the output layer, respectively; MLP represents the multilayer perceptron, which implements non-linear function fitting of the output. and To obtain two C-dimensional pooled feature maps by performing average pooling and max pooling on the input feature layer respectively; M c (F) By obtaining the channel attention vector, the weight parameters of the multiple channels are adjusted appropriately;

[0126] The calculation formula for the spatial attention module is as follows:

[0127]

[0128] For a spatial feature map of size H×W×1, a spatial attention module is used to implement an adaptive weight allocation mechanism in the image to highlight key quality regions. The weighting coefficients generated by the channel attention mechanism are multiplied element-wise with the original input feature map. Then, the processed feature map is further processed through max pooling and average pooling to generate two independent feature maps, which are then stacked and convolved to reduce the feature dimension and output an H×W×1 feature map. Finally, a spatial attention feature map is generated through an activation function, and it is multiplied element-wise with the original input features again to obtain the final output feature.

[0129] In a specific implementation, as a preferred embodiment of the present invention, in step S5, during the training of the improved YOLOv7 defect detection model, a loss function is designed, including localization loss, classification loss, and target confidence loss, which are used to evaluate bounding box prediction, classification prediction, and confidence prediction, respectively. The process is as follows:

[0130] The localization loss function is designed as follows:

[0131] Loss box =(xx)* ) 2 +(yy * ) 2 +(ww * ) 2 +(hh * ) 2

[0132] Where x, y, w, and h represent the coordinates of the top-left corner of the prediction box and the length and width of the prediction box, respectively;

[0133] The classification loss function is designed as follows:

[0134]

[0135] Among them, y i The predicted class probability after the Sigmoid function. x i For category predictions, The true value (0 or 1) for the category;

[0136] The target confidence loss function is designed as follows:

[0137]

[0138] Where K represents the network feature output, divided into a K×K grid; M is the number of anchor boxes in each grid. For a target anchor box, For anchor boxes without target objects, λ noobj These are the weighting coefficients;

[0139] Based on the localization loss, classification loss, and target confidence loss designed in the design, the total loss function is designed as follows:

[0140] Loss = Loss box +Loss cla +Loss obj

[0141] Among them, Loss box This represents the localization loss function; Loss cla Represents the classification loss function; Loss obj This represents the target confidence loss function.

[0142] In this embodiment, a transfer learning strategy is employed for model training. Pre-trained backbone network weights on a large dataset are used for frozen training, reducing GPU memory usage while allowing for arbitrarily large batch sizes, thereby accelerating the convergence process of the defective model. The model training loss function curve is shown below. Figure 13As shown, after 50 rounds of training iterations, the network weights are frozen, entering a comprehensive parameter tuning phase, which significantly increases the memory usage. The network initialized using pre-trained weights shows a rapid decrease in loss in the early stages of training; after the weights are unfrozen in the 50th round, the loss curve fluctuates to some extent, and then gradually stabilizes through parameter tuning, eventually converging.

[0143] Table 2 shows the preliminary detection results of the improved YOLOv7 defect detection model on the training set for normal blades (OK) and abnormal surface defects (NG). The improved model was then analyzed using the current mainstream mAP-based target detection accuracy evaluation method. After 400 training iterations, the mAP value reached 96.36%.

[0144] Table 1 Training set results

[0145]

[0146] To verify the accuracy of the algorithm in detecting surface defects on wind turbine blades, the algorithm achieved an mAP value of 96.42% on the test set, as shown in Table 3.

[0147] Table 2 Test Set Results

[0148]

[0149] Experiments have shown that the improved YOLOv7 defect detection model proposed in this invention has good performance in detecting surface defects of wind turbine blades.

[0150] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An image processing method for real-time detection of defects in wind turbine blades, characterized in that, include: S1. Use a drone equipped with a high-speed camera to photograph the high-speed rotating wind turbine blades and obtain images of the blade surface; S2. Construct the U-net semantic segmentation model to segment the acquired blade surface image and extract the wind turbine blade contour feature image; S3. Based on the contour feature images of wind turbine blades, establish a dataset of wind turbine blade defects; S4. YOLOv7 is used as the defect detection model for wind turbine blades. At the same time, the CBAM attention mechanism module is introduced to optimize and improve the YOLOv7 defect detection model. S5. Based on the established wind turbine blade defect dataset, the improved YOLOv7 defect detection model is trained. The wind turbine blade contour feature image is input into the trained model, and the blade defect type and location information are output.

2. The image processing method for real-time detection of defects in wind turbine blades according to claim 1, characterized in that, Step S2 includes: S21. In the encoding stage, the U-net semantic segmentation model extracts different semantic and spatial features from the blade surface image layer by layer. S22. In the decoding stage, the different semantic and spatial features extracted from the blade surface image are continuously upsampled to reconstruct resolution details close to the original image, resulting in accurate image segmentation results. S23. The feature information between the encoding and decoding stages is connected by horizontal jumps to integrate features at different semantic levels and obtain the wind turbine blade outline feature image.

3. The image processing method for real-time detection of defects in wind turbine blades according to claim 2, characterized in that, The U-net semantic segmentation model introduces a cross-entropy loss function in image segmentation tasks. When processing each pixel location in an image, the output result is treated as a binary classification problem, that is, determining whether the pixel belongs to the target category or not. The cross-entropy loss function is as follows: Where N represents the total number of pixels; y i The label representing reality, y i The value is 0 or 1; Loss represents the output result. This represents the predicted probability.

4. The image processing method for real-time detection of wind turbine blade defects according to claim 1, characterized in that, Step S3 includes: S31. Classify the surface defects and damage characteristics of wind turbine blades according to geometric features, texture features, and color features; S32. Three types of wind turbine defects and damages are used as the detection objects: corrosion, blade gel coat peeling, and protective film damage. S33. Use the Labelme annotation tool to annotate the wind turbine blade defect data to obtain a wind turbine blade damage image dataset with category labels; S34. Based on image enhancement technology, the wind turbine blade damage image dataset with category labels is expanded to obtain a wind turbine blade defect dataset.

5. The image processing method for real-time detection of defects in wind turbine blades according to claim 4, characterized in that, In step S31: The geometric features include crack-like features, hole-like features, and deformation-like features. Crack-like features include linear cracks, bifurcated cracks, and network cracks; hole-like features include circular holes and irregular holes; and deformation-like features include bending deformation, torsional deformation, and local depressions. The texture features include surface roughness features, layering features, and texture disorder features. Surface roughness features include wear and corrosion; layering features include internal layering and surface coating layering; texture disorder features include repairing texture disorder and manufacturing defects. The color characteristics include discoloration characteristics, color spot characteristics, and color unevenness characteristics. Among them, discoloration characteristics include overall discoloration and local discoloration; color spot characteristics include pollution spots and corrosion spots; and color unevenness characteristics include those caused by manufacturing processes and those caused by damage repair.

6. The image processing method for real-time detection of defects in wind turbine blades according to claim 4, characterized in that, Step S34 includes: The leaf image data is defined as a function of two mathematical variables, represented by the horizontal and vertical coordinates of the image region, as follows: Ω={p=[x,y] T ,I(x,y):0≤x<W,0≤y<H} Where W represents the width; H represents the height; p = [x, y] T Representing a pixel, I(x,y)=I(p)=[r,g,b] p The color value at image point p is represented by homogeneous coordinates p = [x, y, 1]. T Represents point p in the image; Define the affine transformation in the image as follows: in, Let A represent the transformed second coordinates, and let A represent the affine transformation matrix. M controls the adjustment of image enhancement, and t controls the translation transformation.

7. The image processing method for real-time detection of defects in wind turbine blades according to claim 1, characterized in that, In step S4, the CBAM attention mechanism module is added to the YOLOv7 backbone, so that when the model enters the backbone architecture network from the input end, it first selects the region of interest of the model.

8. The image processing method for real-time detection of defects in wind turbine blades according to claim 7, characterized in that, The CBAM attention mechanism module includes a channel attention module and a spatial attention module, wherein: The calculation formula for the channel attention module is as follows: For a feature map of size H×W×C, the channel attention module first receives the output features of the previous convolutional layer, i.e., the data of the feature map in the 1×1×C channel dimension, and performs a weighted operation on it. Then, it adaptively assigns weights to the input features through two convolutional operations: max pooling and average pooling, to extract global feature information. After the pooling operation, the feature map is converted into two one-dimensional vectors, removing spatial dimensional redundancy while retaining channel dimension information. Finally, the two one-dimensional vectors are input into a multilayer perceptron for processing. The output features are then fused through element-wise addition to generate a new feature vector. The fused feature vector is activated by the Sigmoid activation function to generate a channel-connected feature map. In the above formula, σ represents the Sigmoid function; W0 and W1 represent the weights of the fully connected layer and the output layer, respectively; MLP represents the multilayer perceptron, which implements non-linear function fitting of the output. and To obtain two C-dimensional pooled feature maps by performing average pooling and max pooling on the input feature layer respectively; M c (F) By obtaining the channel attention vector, the weight parameters of the multiple channels are adjusted appropriately; The formula for calculating the spatial attention module is as follows: M s (F')=σ{f 7×7 (Avgpool(F');Maxpool(F'))} =σ{f 7×7 ([F' avg ;F' max ])} For a spatial feature map of size H×W×1, a spatial attention module is used to implement an adaptive weight allocation mechanism in the image to highlight key quality regions. The weighting coefficients generated by the channel attention mechanism are multiplied element-wise with the original input feature map. Then, the processed feature map is further processed through max pooling and average pooling to generate two independent feature maps, which are then stacked and convolved to reduce the feature dimension and output an H×W×1 feature map. Finally, a spatial attention feature map is generated through an activation function, and it is multiplied element-wise with the original input features again to obtain the final output feature.

9. The image processing method for real-time detection of defects in wind turbine blades according to claim 1, characterized in that, In step S5, during the training of the improved YOLOv7 defect detection model, loss functions are designed, including localization loss, classification loss, and target confidence loss, which are used to evaluate bounding box prediction, classification prediction, and confidence prediction, respectively. The process is as follows: The localization loss function is designed as follows: Loss box =(x-x * ) 2 +(y-y * ) 2 +(w-w * ) 2 +(h-h * ) 2 Where x, y, w, and h represent the coordinates of the top-left corner of the prediction box and the length and width of the prediction box, respectively; The classification loss function is designed as follows: Among them, y i The predicted class probability after the Sigmoid function. x i For category predictions, The true value of the category; The target confidence loss function is designed as follows: Where K represents the network feature output, divided into a K×K grid; M is the number of anchor boxes in each grid. For a target anchor box, For anchor boxes without target objects, λ noobj These are the weighting coefficients; Based on the localization loss, classification loss, and target confidence loss designed in the design, the total loss function is designed as follows: Loss=Loss box +Loss cla +Loss obj Among them, Loss box This represents the localization loss function; Loss cla Represents the classification loss function; Loss obj This represents the target confidence loss function.

Images