A method and system for detecting surface damage of fan blades

Abstract

The present invention provides a method and system for detecting surface damage of fan blades, belonging to the technical field of fan blade damage detection. The present invention combines the pyramid enhancement algorithm and the patch enhancement algorithm. The pyramid enhancement algorithm provides global multi-scale information, and the patch enhancement algorithm provides local detailed information, enabling the model to learn from information at different levels and improving the accuracy of fan blade damage detection. In the training stage of the model, the attention mechanism is used to focus on the features related to damage in the image, suppress the features unrelated to damage, generate enhanced features, map the enhanced features from Cartesian coordinates to polar coordinates to form polar coordinate features, highlight the expression of damage features in the radial and axial directions of the fan blade, and then fuse the polar coordinate features with the enhanced features to construct a feature map that combines fine-grained local information and coarse-grained global information, enabling the model to obtain the detection ability for targets of different scales.

Description

Technical Field

[0001] The present invention belongs to the technical field of wind turbine blade damage detection, and particularly relates to a method and system for detecting surface damage of wind turbine blades. Background Technique

[0002] Since wind turbines are usually installed in harsh environments such as the wild, during service, damages such as cracks, oil stains, and sand holes may occur on the surface of wind turbine blades. These damages will not only affect the power generation efficiency of the wind turbines, but also pose a serious threat to the safety of the equipment. For example, if the cracks on the surface of the wind turbine blade are not discovered and processed in time, they may continue to expand under the action of wind force, and eventually lead to blade fracture, triggering serious safety accidents. Therefore, it is of great significance to detect the damage of wind turbine blades.

[0003] In the related art, the detection of surface damage of wind turbine blades is mainly carried out by image detection, that is, using an unmanned aerial vehicle (UAV) equipped with a camera to capture images of the blades running in the air, and then using image recognition algorithms to detect the damaged parts. However, due to the randomness of the damage location, it brings great difficulties to image feature extraction, and easily leads to the problem of low detection accuracy. Moreover, for some minor damages or damages with high similarity to the background, there may be cases of missed detection or false detection.

[0004] Therefore, it is necessary to provide a method and system for detecting surface damage of wind turbine blades to solve the above problems. Summary of the Invention

[0005] The present invention provides a method and system for detecting surface damage of wind turbine blades. By combining the pyramid enhancement algorithm and the patch enhancement algorithm, the pyramid enhancement algorithm provides global multi-scale information, and the patch enhancement algorithm provides local detail information, enabling the model to learn from information at different levels and improving the accuracy of wind turbine blade damage detection; in the training stage of the model, the attention mechanism is used to focus on the features related to damage in the image, suppress the features unrelated to damage, generate enhanced features, map the enhanced features from Cartesian coordinates to polar coordinates to form polar coordinate features, highlighting the expression of damage features in the radial and axial directions of the wind turbine blade, and then fusing the polar coordinate features with the enhanced features to construct a feature map that combines fine-grained local information and coarse-grained global information, enabling the improved YOLOv8 model to obtain the detection ability for targets of different scales, and effectively solving at least one technical problem involved in the background technique.

[0006] In order to solve the above technical problems, the present invention is implemented as follows:

[0007] A method for detecting surface damage of wind turbine blades includes the following steps:

[0008] Step S1, obtain the original dataset, where the original dataset includes multiple original images of the surface of the fan blade, and the damage location and damage type are marked in each original image;

[0009] Step S2, use the pyramid enhancement algorithm to enhance each original image in the original dataset to form a pyramid image sequence, and then use overlapping sliding windows to extract patches from each layer in the pyramid image sequence to form a patch set, and reconstruct the training dataset with the pyramid image sequence and the patch set;

[0010] Step S3, build an improved YOLOv8 model based on the traditional YOLOv8 model. The neck network and detection head of the improved YOLOv8 model are consistent with the architecture in the traditional YOLOv8 model. The backbone network includes an initial convolutional layer, a C2F-FocalNextBlock module, a polar coordinate feature transformation module, and a feature fusion module. Feed the training dataset and its corresponding labels into the improved YOLOv8 model for training to obtain a trained improved YOLOv8 model; The training process includes the following steps: Feed the images in the training dataset into the initial convolutional layer to extract basic features, and then feed the extracted basic features into the C2F-FocalNextBlock module to focus on the features related to damage in the image based on the attention mechanism and suppress the features unrelated to damage to generate enhanced features; Then feed the enhanced features into the polar coordinate feature transformation module to map the enhanced features from the Cartesian coordinate system to the polar coordinate system to form polar coordinate features, highlighting the expression of damage features in the radial and axial directions of the fan blade; Finally, fuse the polar coordinate features output by the polar coordinate feature transformation module with the enhanced features output by the C2F-FocalNextBlock module to construct a fusion feature that combines fine-grained local information and coarse-grained global information; The neck network converts the fusion feature output by the backbone network into multi-scale features suitable for the detection task and passes them to the detection head for final bounding box regression and classification;

[0011] Step S4, for any fan blade surface damage detection process, first collect the image of the fan blade surface, and then extract the pyramid image sequence and the patch set. Use the pyramid image sequence and the patch set together as the input of the trained improved YOLOv8 model, and output the prediction results of the damage location and damage type in each image.

[0012] As a preferred improvement, the damage types include "dirt" and "damage".

[0013] As a preferred improvement, the generation process of the pyramid image sequence specifically includes the following steps:

[0014] Step S211, set the scaling ratio and the minimum size of the image ;

[0015] Step S212, according to the scaling ratio downsample the original image layer by layer to generate a lower-resolution next-layer image. The downsampling process is expressed as:

[0016]

[0017] In the formula, 、 respectively represent the images of the and the layers, ; represents the scaling operation, and the scaling process is expressed as:

[0018]

[0019]

[0020] In the formula, 、 respectively represent the height and width of the image of the layer; and respectively represent the height and width of the image of the layer;

[0021] Step S213, when or stop downsampling to obtain the pyramid image sequence , , represents the total number of layers of the pyramid.

[0022] As a preferred improvement, the generation process of the patch set specifically includes the following steps:

[0023] Step S221, set the size of the sliding window to , the horizontal step size and the vertical step size ;

[0024] Step S222, with the condition that two adjacent windows have an overlap rate of 10%, perform patch sampling along the horizontal and vertical directions of the image respectively to obtain the patch set .

[0025] As a preferred improvement, the initial convolutional layer adopts the CSPDarknet architecture; the C2F-FocalNextBlock module is constructed by introducing the focal block module in the CFNet network on the basis of the traditional C2F network and replacing the neck network of the C2F network. The processing process of the C2F-FocalNextBlock module is expressed as:

[0026]

[0027] In the formula, represents the processing process of the C2F-FocalNextBlock module.

[0028] As a preferred improvement, the focal block module includes a 7×7 depth convolutional layer, a window attention mechanism with a window size of 7×7, and a 1×1 convolutional layer connected in sequence. A LayerNorm layer and a GELU unit are connected after each 7×7 depth convolutional layer and window attention mechanism. The processing process of the focal block module is expressed as:

[0029]

[0030] In the formula, represents the input feature of the focal block module; represents the 7×7 depth separable convolution; represents the 1×1 convolution; represents the activation function; represents layer normalization; represents the 7×7 window attention mechanism, and the attention weight is calculated by the following formula:

[0031]

[0032]

[0033] In the formula, represents the input feature X of the attention weight; respectively represent the query, key, and value in the attention mechanism; is the dimension of the feature X ; represents matrix transpose; represents the activation function; represents the linear regression operation.

[0034] As a preferred improvement, the processing process of the polar coordinate feature transformation module includes the following steps:

[0035] Step S31, predict the center coordinates of the fan blade from the enhanced features The prediction process is expressed as: ,

[0036]

[0037] In the formula, respectively represent the horizontal and vertical coordinates of the center of the fan blade in the Cartesian coordinate system; represents Sigmoid the activation function; represents convolution; respectively represent the enhanced features the width and height of the feature map;

[0038] Step S32, map the enhanced features from the Cartesian coordinate system to the polar coordinate system, and generate polar coordinate features through bilinear interpolation, where:

[0039] The mapping process is expressed as:

[0040]

[0041] In the formula, respectively represent the horizontal and vertical coordinates of the point on the feature map in the Cartesian coordinate system; , respectively represent the point on the feature map the horizontal and vertical coordinates in the polar coordinate system, where, ; ; represents the enhanced feature the maximum radial distance of the feature map; ;

[0042] The polar coordinate feature generation process is expressed as:

[0043]

[0044] In the formula, represents the polar coordinate feature; represents the variable used to index the grid points of the feature map in the Cartesian coordinates; represents the interpolation weight, which is calculated by the bilinear interpolation formula;

[0045] Step S33, apply a radial-angular separable convolution to the polar coordinate features, inverse-transform the convolved polar coordinate features back to the Cartesian coordinate system, and fuse them with the original enhanced features The process is expressed as:

[0046]

[0047]

[0048]

[0049] In the formula, represents a 1D convolution along the radial direction; represents a 1D convolution along the angular direction; represents the polar coordinate features after convolution; represents the polar coordinate features that are inverse-transformed back to the Cartesian coordinate system; represents the inverse transformation of Cartesian to polar coordinates; represents a weight factor, , represents learnable parameters; represents the fused features.

[0050] As a preferred improvement, the fused features are fed into the neck network. The neck network fuses feature maps at different levels through a feature pyramid network and a path aggregation network, converts them into multi-scale features suitable for the detection task, and transmits them to the detection head for final bounding box regression and classification.

[0051] As a preferred improvement, the loss function for training is expressed as:

[0052]

[0053] In the formula, represents the total loss function; represents the bounding box loss, which is used to measure the accuracy of determining the target position; represents the classification loss, which is used to measure the accuracy of classifying the target category; represents the distribution focal loss, which is used to measure the degree to which the target is close to the target position; , , represent hyperparameters, which are used to balance the importance of different losses;

[0054] The training process updates the model parameters through an optimizer.

[0055] A system for implementing the above-mentioned wind turbine blade surface damage detection method, comprising:

[0056] An original dataset acquisition module, which is used to obtain an original dataset. The original dataset includes multiple original images of the wind turbine blade surface, and each original image is marked with the damage position and damage type;

[0057] A training dataset construction module, which is used to enhance each original image in the original dataset by using the pyramid enhancement algorithm to form a pyramid image sequence, and then use overlapping sliding windows to extract patches from each layer in the pyramid image sequence to form a patch set, and reconstruct the training dataset with the pyramid image sequence and the patch set;

[0058] An improved YOLOv8 model construction module, which is used to construct an improved YOLOv8 model based on the traditional YOLOv8 model. The neck network and the detection head of the improved YOLOv8 model are consistent with the architectures in the traditional YOLOv8 model. The backbone network includes an initial convolutional layer, a C2F-FocalNextBlock module, a polar coordinate feature transformation module, and a feature fusion module. The training dataset and its corresponding labels are fed into the improved YOLOv8 model for training to obtain a trained improved YOLOv8 model. The training process includes the following steps: feeding the images in the training dataset into the initial convolutional layer to extract basic features, then feeding the extracted basic features into the C2F-FocalNextBlock module to focus on the features related to damage in the image based on the attention mechanism and suppress the features unrelated to damage to generate enhanced features; then feeding the enhanced features into the polar coordinate feature transformation module to map the enhanced features from the Cartesian coordinate system to the polar coordinate system to form polar coordinate features, highlighting the expression of damage features in the radial and axial directions of the wind turbine blade; finally, fusing the polar coordinate features output by the polar coordinate feature transformation module with the enhanced features output by the C2F-FocalNextBlock module to construct a fusion feature that combines fine-grained local information and coarse-grained global information; the neck network converts the fusion feature output by the backbone network into multi-scale features suitable for the detection task and passes them to the detection head for final bounding box regression and classification;

[0059] A detection module, which is used for any wind turbine blade surface damage detection process. First, it collects images of the wind turbine blade surface, then extracts the pyramid image sequence and the patch set, and uses the pyramid image sequence and the patch set together as the input of the trained improved YOLOv8 model to output the prediction results of the damage location and damage type in each image.

[0060] The beneficial effects of the present invention are as follows:

[0061] (1) The present invention combines the pyramid enhancement algorithm and the patch enhancement algorithm. The pyramid enhancement algorithm is used to generate multi-scale images. The multi-scale images enable the model to learn its features at different resolutions, avoiding missed detections due to large scale differences. The patch enhancement algorithm divides the image into multiple small blocks, and each small block is used as an independent sample, which helps the model learn local features. For the subtle damages on the surface of the fan blade, the patch enhancement method can provide more learning opportunities and enhance its feature representation. The pyramid enhancement algorithm provides global multi-scale information, and the patch enhancement algorithm provides local detail information, enabling the model to learn from information at different levels and improving the accuracy of damage detection.

[0062] (2) Replace the neck network in the C2F module of the backbone network of the traditional YOLOv8 model with the focal block module in the CFnet network. The attention mechanism in the focal block module automatically focuses on the important parts related to damages in the image and suppresses the background information unrelated to damages, so that the model can learn damage features more effectively and improve the ability to identify damages.

[0063] (3) Use the polar coordinate feature transformation module to map the enhanced features extracted by the C2F-FocalNextBlock module from Cartesian coordinates to polar coordinates, which can express the rotational structure information of the fan blade in a more direct way, enabling the model to better capture the overall structural features of the fan blade, such as the circumferential distribution and radial extension of the fan blade, so as to more accurately identify the damage patterns related to rotation. Description of the Drawings

[0064] To more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings, where:

[0065] Figure 1 represents the network architecture diagram of the improved YOLOv8 model provided by the present invention;

[0066] Figure 2 represents the detection result of the "dirt" type of damage by the fan blade surface damage detection method provided by the present invention;

[0067] Figure 3 represents the detection result of the "damage" type of damage by the fan blade surface damage detection method provided by the present invention. Detailed Embodiments

[0068] Next, in combination with the embodiments of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0069] Please refer to Figure 1 , this embodiment provides a method for detecting surface damage of a fan blade, including the following steps:

[0070] Step S1, obtain an original data set, where the original data set includes multiple original images of the surface of a fan blade, and the damage position and damage type are marked in each original image.

[0071] The original data set adopts the fan blade detection image data set proposed by A. Shihavuddin in the prior art. This data includes 2,995 images with a size of 586×371 pixels. Each image uses the YOLO format to mark the damage position and damage type. The marked damage types include "dirt" and "damage", where: there are 581 "dirt" labels and 8,770 "damage" labels.

[0072] Step S2, use the pyramid enhancement algorithm to enhance each original image in the original data set to form a pyramid image sequence, and then use overlapping sliding windows to extract patches from each layer in the pyramid image sequence to form a patch set, and reconstruct the training data set with the pyramid image sequence and the patch set.

[0073] In the original data set, it includes images of the surface of the fan blade taken from different distances, and there are significant differences in the sizes of the same part of the fan blade in different images. For example, when taking a close-up shot, the details of the surface of the fan blade can be clearly shown, but only part of the structure of the fan blade may be shown in the image, and the overall structure of the fan blade cannot be fully shown; while when taking a long-distance shot, although the overall structure of the fan can be fully shown, the surface details will become blurred, and the proportion of the fan blade in the image is small.

[0074] To better learn this scale difference, the present invention combines the pyramid enhancement algorithm and the patch enhancement algorithm. The pyramid enhancement algorithm is used to generate multi-scale images. For example, for damages of different sizes (from small wear points to large cracks), the multi-scale images enable the model to learn their features at different resolutions, avoiding missed detections due to large scale differences. The patch enhancement algorithm divides the image into multiple small blocks, and each small block is used as an independent sample, which helps to learn local features. For the subtle damages on the local surface of the fan blade, the patch enhancement method can provide more learning opportunities and enhance their feature representation. The pyramid enhancement algorithm provides global multi-scale information, and the patch enhancement algorithm provides local detail information, enabling the model to learn from information at different levels and improving the accuracy of fan blade damage detection.

[0075] The generation process of the pyramid image sequence specifically includes the following steps:

[0076] Step S211, set the scaling ratio and the minimum size of the image ;

[0077] Step S212, perform downsampling on the original image layer by layer according to the scaling ratio to generate the next lower-resolution image. The downsampling process is expressed as:

[0078]

[0079] In the formula, , respectively represent the and the layer images, ; represents the scaling operation, and the scaling process is expressed as:

[0080]

[0081]

[0082] In the formula, , respectively represent the height and width of the layer image; and respectively represent the height and width of the layer image;

[0083] Step S213, when or , stop downsampling to obtain the pyramid image sequence , , Indicates the total number of layers of the pyramid.

[0084] The generation process of the patch set specifically includes the following steps:

[0085] Step S221, set the size of the sliding window to , the horizontal step size of the sliding window and the vertical step size ;

[0086] Step S222, with the condition that two adjacent windows have an overlap rate of 10%, perform patch sampling along the horizontal and vertical directions of the image respectively to obtain the patch set .

[0087] Step S3, build an improved YOLOv8 model based on the traditional YOLOv8 model. The neck network and detection head of the improved YOLOv8 model are consistent with the architecture in the traditional YOLOv8 model. The backbone network includes an initial convolutional layer, C2F-FocalNextBlock modules, a polar coordinate feature transformation module, and a feature fusion module. Send the training dataset and its corresponding labels into the improved YOLOv8 model for training to obtain the trained improved YOLOv8 model. The training process includes the following steps: Send the images in the training dataset into the initial convolutional layer to extract basic features, and then send the extracted basic features into the C2F-FocalNextBlock modules. Based on the attention mechanism, focus on the features related to damage in the image and suppress the features unrelated to damage to generate enhanced features. Then send the enhanced features into the polar coordinate feature transformation module to map the enhanced features from the Cartesian coordinate system to the polar coordinate system to form polar coordinate features, highlighting the expression of damage features in the radial and axial directions of the wind turbine blade. Finally, fuse the polar coordinate features output by the polar coordinate feature transformation module with the enhanced features output by the C2F-FocalNextBlock modules to construct a fusion feature that combines fine-grained local information and coarse-grained global information. The neck network converts the fusion feature output by the backbone network into multi-scale features suitable for the detection task and passes them to the detection head for final bounding box regression and classification.

[0088] The initial convolutional layer adopts the conventional CSPDarknet architecture in the art, and its process of processing the input image also belongs to the conventional technology in the art, which will not be elaborated in this embodiment.

[0089] To increase the aerodynamic performance, the surface of the wind turbine blade usually has complex textures, which appear as lines and patterns with different directions and intensities in the image. And the shapes of different parts of the wind turbine blade are unique, forming specific geometric contours in the image, and the changes in its edges and curved surfaces are diverse, bringing challenges to feature extraction and object detection.

[0090] To solve this problem, the present invention introduces the focalblock module in the CFNet network on the basis of the traditional C2F network, constructs the C2F-FocalNextBlock module by replacing the neck network of the C2F network with the focal block module. The focal block module can help the network automatically focus on the regions that are more important for damage detection, such as key damage features like cracks and wear, while suppressing the features of some unimportant backgrounds or normal regions based on the attention mechanism.

[0091] The basic features extracted by the initial convolutional layer are fed into the C2F-FocalNextBlock module to output enhanced features , and the processing process is expressed as:

[0092]

[0093] In the formula, represents the processing process of the C2F-FocalNextBlock module.

[0094] The focal block module includes a 7×7 depth convolutional layer, a window attention mechanism with a window size of 7×7, and a 1×1 convolutional layer connected in sequence. A LayerNorm layer and a GELU unit are connected after each 7×7 depth convolutional layer and window attention mechanism.

[0095] After the input features enter the focal block module, they first go through an atrous depth convolution operation to expand the receptive field of the convolutional kernel without increasing the number of parameters, enabling neurons to obtain information in a larger range. Then, through skip connections, the original input features are fused with the convolved features to retain the important information in the original features and prevent information loss. Subsequently, a 1×1 convolutional operation is performed to adjust the number of channels to adapt to the requirements of the subsequent network structure and reduce the computational amount at the same time. Nonlinearity is introduced through the GELU activation function to enhance the model's expressive ability. The LayerNorm layer after each 7×7 depth convolutional layer and window attention mechanism operation normalizes the features, accelerating the model's convergence and improving stability.

[0096] The processing process of the focal block module is expressed as:

[0097]

[0098] In the formula, represents the input features of the focal block module; represents the 7×7 depthwise separable convolution; represents the 1×1 convolution; represents Activation function; Layer normalization; Indicates a 7×7 window attention mechanism, and the attention weights are calculated by the following formula:

[0099]

[0100]

[0101] In the formula, represents the input feature X of the attention weights; respectively represent the query, key, and value in the attention mechanism; is the feature X dimension; represents matrix transpose; represents activation function; represents a linear regression operation.

[0102] Wind turbine blades usually have a rotationally symmetric geometric structure. Traditional Cartesian coordinates have the disadvantage of being less intuitive when describing such rotational features, while the polar coordinate system locates points by distance and angle and is naturally suitable for describing objects with rotational characteristics. Therefore, the present invention uses a polar coordinate feature transformation module to map the enhanced features output by the C2F-FocalNextBlock module from the Cartesian coordinate system to the polar coordinate system, which can express the rotational structure information of the wind turbine blade in a more direct way, enabling the model to better capture the overall structural features of the wind turbine blade, such as the circumferential distribution and radial extension of the wind turbine blade, so as to more accurately identify damage patterns related to rotation.

[0103] In the polar coordinate system, the radial direction (from the center to the edge) and circumferential direction (the circumferential direction around the center) features of the wind turbine blade can be clearly separated and expressed. For wind turbine blades, damage may develop along the radial direction (such as cracks from the blade root to the tip) or the circumferential direction (such as wear on the blade circumference). The polar coordinate feature transformation module can specifically extract features in these two directions, helping the model to more keenly detect damage features in different directions, and then effectively fuse the extracted local damage details with the global wind turbine structure information. Locally, it can capture damage features such as microcracks and corrosion spots; globally, it can grasp the overall shape and layout of the wind turbine blade, providing more comprehensive information for accurate damage judgment. This fusion of local-global features helps to improve the damage recognition accuracy and localization accuracy of the model.

[0104] In addition, in the Cartesian coordinate system, the feature extraction of wind blade images may contain some redundant information irrelevant to damage detection, and is also easily interfered by background noise. In the polar coordinate system, the feature extraction of wind blade images can focus on key features related to blade rotation and structure, reduce unnecessary information interference, make the model more focused on learning and identifying damage features, and improve the accuracy and robustness of damage detection.

[0105] The angle and distance information in the polar coordinate system can provide a more accurate coordinate description for the location of damage. For wind turbine blade damage detection, it is crucial to know the exact location of the damage, which helps with subsequent repair and maintenance work. The polar coordinate feature module can use this precise coordinate information to more accurately locate the damage and improve the reliability of detection; various complex damage patterns may occur in wind turbines during actual operation, such as multi-angle cracks, irregular corrosion areas, etc. Polar coordinate features can better adapt to these complex damage forms. Through the analysis and learning of polar coordinate features, the model can more effectively identify and distinguish different types of damage and improve the detection capabilities of complex damage patterns.

[0106] The processing of the polar coordinate feature transformation module includes the following steps:

[0107] Step S31, from the enhanced features Predict wind turbine blade center coordinates , the prediction process is expressed as:

[0108]

[0109] In the formula, Respectively represent the horizontal and vertical coordinates of the center of the fan blade in the Cartesian coordinate system; express Sigmoid Activation function; express convolution; Represents enhanced features The feature map width and height of

[0110] Step S32: Enhance the feature Mapping from the Cartesian coordinate system to the polar coordinate system, the polar coordinate features are generated by bilinear interpolation, where:

[0111] The mapping process is expressed as:

[0112]

[0113] In the formula, Respectively represent the horizontal and vertical coordinates of the points on the feature map in the Cartesian coordinate system; , respectively represent points on the feature map the horizontal and vertical coordinates in the polar coordinate system, where ; ; represents the enhanced feature the maximum radial distance of the feature map ;

[0114] The polar coordinate feature generation process is expressed as:

[0115]

[0116] In the formula, represents the polar coordinate feature; represents the variable used to index the grid points of the feature map in Cartesian coordinates, through which the feature values of each grid point can be obtained; represents the interpolation weight, which is calculated by the bilinear interpolation formula;

[0117] Step S33, apply a radial-angular separable convolution to the polar coordinate feature, inverse-transform the convolved polar coordinate feature back to the Cartesian coordinate system, and fuse it with the original enhanced feature The process is expressed as:

[0118]

[0119]

[0120]

[0121] In the formula, represents the 1D convolution along the radial direction; represents the 1D convolution along the angular direction; represents the convolved polar coordinate feature; represents the polar coordinate feature inverse-transformed back to the Cartesian coordinate system; represents the inverse transformation of Cartesian to polar coordinates; represents the weight factor, , represents the learnable parameter; represents the fused feature.

[0122] After converting the Cartesian coordinate image to a polar coordinate image, the radial-angular separable convolution better matches the structure of the polar coordinate space, can make full use of the distribution law of features in the polar coordinate space, can extract features in the radial and angular directions respectively, and better capture the feature information in these different directions.

[0123] Fused feature It is sent to the neck network, which fuses feature maps of different levels through the Feature Pyramid Network (FPN) and the Path Aggregation Network (PAN), converts them into multi-scale features suitable for the detection task, and transmits them to the detection head for final bounding box regression and classification, outputting the prediction results for the damage location and damage type; finally, through post-processing methods such as Non-Maximum Suppression (NMS), multiple prediction results output by the detection head are screened and merged, and redundant bounding boxes with too high overlap are removed according to factors such as the class probability and overlap degree of the bounding boxes to obtain the final object detection results.

[0124] The loss function for training is expressed as:

[0125]

[0126] In the formula, represents the total loss function; represents the bounding box loss, which is used to measure the accuracy of determining the target position; represents the classification loss, which is used to measure the accuracy of classifying the target category; represents the distribution focal loss, which is used to measure the degree to which the target is close to the target position; 、 、 represent hyperparameters, which are used to balance the importance of different losses.

[0127] The training process updates the model parameters through an optimizer, and the optimizer is specifically selected as .

[0128] Step S4, for any wind turbine blade surface damage detection process, first collect the images of the wind turbine blade surface, then extract the pyramid image sequence and the patch set, and use the pyramid image sequence and the patch set together as the input of the trained improved YOLOv8 model, and output the prediction results for the damage location and damage type in each image.

[0129] The collection of the wind turbine blade surface images is realized by means of a drone. The drone is equipped with a camera to take the images of the wind turbine blade surface running in the air. After the image collection is completed, preprocessing is also required, including grayscale conversion, denoising, and illumination equalization and other processing to improve the image quality.

[0130] Please refer to Figure 2 and Figure 3 , Figure 2 represents the detection result of the "dirt" type of damage by the wind turbine blade surface damage detection method provided by the present invention; Figure 3It shows the detection results of the "damage" type of damage by the fan blade surface damage detection method provided by the present invention. Under both damage types, accurate prediction is achieved, indicating the effectiveness of the fan blade surface damage detection method provided by the present invention.

[0131] This embodiment also provides a system for executing the above-mentioned fan blade surface damage detection method, including:

[0132] An original dataset acquisition module, configured to acquire an original dataset, where the original dataset includes multiple original images of the fan blade surface, and the damage position and damage type are marked in each original image;

[0133] A training dataset construction module, configured to enhance each original image in the original dataset by using a pyramid enhancement algorithm to form a pyramid image sequence, and then use overlapping sliding windows to extract patches from each layer in the pyramid image sequence to form a patch set, and reconstruct the training dataset with the pyramid image sequence and the patch set;

[0134] An improved YOLOv8 model construction module, configured to construct an improved YOLOv8 model based on the traditional YOLOv8 model. The neck network and detection head of the improved YOLOv8 model are consistent with the architecture in the traditional YOLOv8 model. The backbone network includes an initial convolutional layer, a C2F-FocalNextBlock module, a polar coordinate feature transformation module, and a feature fusion module. The training dataset and its corresponding labels are sent into the improved YOLOv8 model for training to obtain a trained improved YOLOv8 model; the training process includes the following steps: sending the images in the training dataset into the initial convolutional layer to extract basic features, and then sending the extracted basic features into the C2F-FocalNextBlock module to focus on the features related to damage in the image based on the attention mechanism and suppress the features unrelated to damage to generate enhanced features; then sending the enhanced features into the polar coordinate feature transformation module to map the enhanced features from the Cartesian coordinate system to the polar coordinate system to form polar coordinate features, highlighting the expression of damage features in the radial and axial directions of the fan blade; finally, fusing the polar coordinate features output by the polar coordinate feature transformation module with the enhanced features output by the C2F-FocalNextBlock module to construct a fusion feature that combines fine-grained local information and coarse-grained global information; the neck network converts the fusion feature output by the backbone network into multi-scale features suitable for the detection task and transmits them to the detection head for final bounding box regression and classification;

[0135] The detection module is used for any detection process of the surface damage of the fan blade. First, it collects the images of the fan blade surface, then extracts the pyramid image sequence and the patch set, and uses the pyramid image sequence and the patch set together as the input of the trained improved YOLOv8 model to output the prediction results of the damage positions and damage types in each image.

[0136] The embodiments of the present invention have been described above in conjunction with the accompanying drawings. However, the present invention is not limited to the above specific embodiments. The above specific embodiments are merely illustrative and not restrictive. Under the inspiration of the present invention, those of ordinary skill in the art can also make many forms without departing from the purpose of the present invention and the scope protected by the claims, and all of them belong to the protection scope of the present invention.

Claims

1. A method for detecting surface damage of a wind turbine blade, characterized in that: The steps include: Step S1, obtaining an original data set, wherein the original data set includes a plurality of original images of the surface of a wind turbine blade, each original image being marked with a damage location and a damage type; Step S2, using a pyramid enhancement algorithm to enhance each original image in the original data set to form a pyramid image sequence, then using overlapping sliding windows to extract patches from each layer in the pyramid image sequence to form a patch set, and reconstructing the training data set with the pyramid image sequence and the patch set; Step S3, constructing an improved YOLOv8 model based on the traditional YOLOv8 model, the neck network and detection head of the improved YOLOv8 model are consistent with the architecture in the traditional YOLOv8 model, the backbone network includes an initial convolutional layer, a C2F-FocalNextBlock module, a polar coordinate feature transformation module and a feature fusion module, and sending the training data set and its corresponding labels to the improved YOLOv8 model for training to obtain a trained improved YOLOv8 model; the training process includes the following steps: sending the images in the training data set to the initial convolutional layer to extract basic features, and then sending the extracted basic features to the C2F-FocalNextBlock module, Based on the attention mechanism, the features related to damage in the image are focused on, and the features unrelated to damage are suppressed to generate enhanced features. Then, the enhanced features are sent to the polar coordinate feature transformation module, and the enhanced features are mapped from the Cartesian coordinate system to the polar coordinate system to form polar coordinate features, highlighting the expression of damage features in the radial and axial directions of the wind turbine blades. Finally, the polar coordinate features output by the polar coordinate feature transformation module are fused with the enhanced features output by the C2F-FocalNextBlock module to construct fused features that combine fine-grained local information and coarse-grained global information. The neck network converts the fused features output by the backbone network into multi-scale features suitable for the detection task, and passes them to the detection head for the final bounding box regression and classification. Step S4, for any wind blade surface damage detection process, first collect the image of the wind blade surface, then extract the pyramid image sequence and patch set, use the pyramid image sequence and patch set as the input of the trained improved YOLOv8 model, and output the prediction results of the damage location and damage type in each image.

2. The method for detecting surface damage of a wind turbine blade according to claim 1, characterized in that: Damage types include Dirt and Damage.

3. The method for detecting surface damage of a wind turbine blade according to claim 1, characterized in that: The generation process of the pyramid image sequence specifically includes the following steps: Step S211, setting the zoom ratio and the minimum image size ; Step S212: according to the scaling ratio For the original image Downsampling is performed layer by layer to generate the next layer of images with lower resolution. The downsampling process is expressed as: In the formula, , Respectively represent and Layer images, ; Represents the scaling operation, and the scaling process is expressed as: In the formula, , Respectively represent The height and width of the layer image; and Respectively represent The height and width of the layer image; Step S213, when or When , stop downsampling and get the pyramid image sequence , , Indicates the total number of pyramid layers.

4. The method for detecting surface damage of a wind turbine blade according to claim 3, characterized in that: The process of generating a patch set specifically includes the following steps: Step S221, set the size of the sliding window to , the horizontal step size of the sliding window and vertical step length ; Step S222: Under the condition that two adjacent windows have a 10% overlap rate, patch sampling is performed along the horizontal and vertical directions of the image to obtain a patch set .

5. The method for detecting surface damage of a wind turbine blade according to claim 1, characterized in that: The initial convolutional layer adopts the CSPDarknet architecture; The C2F-FocalNextBlock module introduces the focal block module in the CFNet network on the basis of the traditional C2F network, and replaces the neck network of the C2F network with the focal block module. The processing process of the C2F-FocalNextBlock module is expressed as follows: In the formula, Represents the processing of the C2F-FocalNextBlock module.

6. The method for detecting surface damage of a wind turbine blade according to claim 5, characterized in that: The focal block module consists of a 7×7 deep convolutional layer, a window attention mechanism with a window size of 7×7, and a 1×1 convolutional layer. Each 7×7 deep convolutional layer and the window attention mechanism are connected to a LayerNorm layer and a GELU unit. The processing process of the focalblock module is expressed as follows: In the formula, Represents the input features of the focal block module; Represents 7×7 depth-wise separable convolution; Represents 1×1 convolution; express Activation function; Representation layer normalization; represents a 7×7 window attention mechanism, and the attention weight is calculated by the following formula: In the formula, Represents input features X The attention weight of Represent the query, key, and value in the attention mechanism respectively; Features X Dimensions; Represents matrix transpose; express Activation function; Represents a linear regression operation.

7. The method for detecting surface damage of a wind turbine blade according to claim 6, characterized in that: The processing of the polar coordinate feature transformation module includes the following steps: Step S31, from the enhanced features Predict wind turbine blade center coordinates , the prediction process is expressed as: In the formula, Respectively represent the horizontal and vertical coordinates of the center of the fan blade in the Cartesian coordinate system; express Sigmoid Activation function; express convolution; Represents enhanced features The feature map width and height of Step S32: Enhance the feature Mapping from the Cartesian coordinate system to the polar coordinate system, the polar coordinate features are generated by bilinear interpolation, where: The mapping process is expressed as: In the formula, Respectively represent the horizontal and vertical coordinates of the points on the feature map in the Cartesian coordinate system; , Represents the points on the feature map The horizontal and vertical coordinates in the polar coordinate system, where ; ; Represents enhanced features The maximum radial distance of the characteristic map, ; The polar coordinate feature generation process is expressed as: In the formula, Represents polar coordinate features; Represents the variable used to index the feature map grid points in Cartesian coordinates; Represents the interpolation weight, which is calculated by the bilinear interpolation formula; Step S33, applying radial-angular separable convolution to the polar coordinate features, inversely transforming the convolved polar coordinate features back to the Cartesian coordinate system, and combining them with the original enhanced features The fusion process is expressed as: In the formula, Represents 1D convolution along the radial direction; represents a 1D convolution along the angular direction; Represents the polar coordinate features after convolution; Polar coordinate features representing the inverse transformation back to Cartesian coordinates; Represents the inverse transformation from Cartesian to polar coordinates; represents the weight factor, , represents a learnable parameter; Represents fusion features.

8. The method for detecting surface damage of a wind turbine blade according to claim 7, characterized in that: Fusion Features It is sent to the neck network, which fuses feature maps of different levels through the feature pyramid network and path aggregation network, converts them into multi-scale features suitable for the detection task, and passes them to the detection head for final bounding box regression and classification.

9. The method for detecting surface damage of a wind turbine blade according to claim 1, characterized in that: The loss function of training is expressed as: In the formula, Represents the total loss function; represents the bounding box loss, which is used to measure the accuracy of determining the target location; Represents the classification loss, which is used to measure the accuracy of classifying the target category; represents the distribution focal loss, which is used to measure how close the target is to the target location; , , represents a hyperparameter used to balance the importance of different losses; The training process is through Optimizer to update model parameters.

10. A system for executing the wind turbine blade surface damage detection method according to any one of claims 1 to 9, characterized in that: include: The original data set acquisition module is used to obtain the original data set, wherein the original data set includes a plurality of original images of the surface of the wind turbine blade, each original image being marked with a damage location and a damage type; A training data set construction module is used to enhance each original image in the original data set using a pyramid enhancement algorithm to form a pyramid image sequence, and then use overlapping sliding windows to extract patches from each layer in the pyramid image sequence to form a patch set, and reconstruct the training data set with the pyramid image sequence and the patch set; The improved YOLOv8 model construction module is used to build an improved YOLOv8 model based on the traditional YOLOv8 model. The neck network and detection head of the improved YOLOv8 model are consistent with the architecture of the traditional YOLOv8 model. The backbone network includes an initial convolution layer, a C2F-FocalNextBlock module, a polar coordinate feature transformation module, and a feature fusion module. The training data set and its corresponding labels are sent to the improved YOLOv8 model for training to obtain a trained improved YOLOv8 model. The training process includes the following steps: sending the images in the training data set to the initial convolution layer to extract basic features, and then sending the extracted basic features to the C2F-FocalNextBl The FocalNextBlock module focuses on the features related to damage in the image based on the attention mechanism, suppresses the features not related to damage, and generates enhanced features; then the enhanced features are sent to the polar coordinate feature transformation module, and the enhanced features are mapped from the Cartesian coordinate system to the polar coordinate system to form polar coordinate features, highlighting the expression of damage features in the radial and axial directions of the wind turbine blades; finally, the polar coordinate features output by the polar coordinate feature transformation module are fused with the enhanced features output by the C2F-FocalNextBlock module to construct a fusion feature that combines fine-grained local information and coarse-grained global information; the neck network converts the fusion features output by the backbone network into multi-scale features suitable for the detection task, and passes them to the detection head for the final bounding box regression and classification; The detection module is used for any wind turbine blade surface damage detection process. First, the image of the wind turbine blade surface is collected, and then the pyramid image sequence and patch set are extracted. The pyramid image sequence and patch set are used as the input of the trained improved YOLOv8 model, and the prediction results of the damage location and damage type in each image are output.

Images