Fan blade surface damage detection method and system

Abstract

The invention provides a fan blade surface damage detection method and system, and belongs to the technical field of fan blade damage detection. The mode of combining the pyramid enhancement algorithm and the patch enhancement algorithm is adopted, the pyramid enhancement algorithm provides global multi-scale information, the patch enhancement algorithm provides local detail information, the model can learn from different levels of information, and the accuracy of fan blade damage detection is improved; in the training stage of the model, an attention mechanism is adopted to pay attention to features related to damage in an image, features irrelevant to damage are inhibited, enhanced features are generated, the enhanced features are mapped to a polar coordinate system from Cartesian coordinates to form polar coordinate features, expressions of the damage features in the radial direction and the axial direction of the fan blade are highlighted, and the model is obtained. And then the polar coordinate features and the enhanced features are fused, and a feature map fusing fine-grained local information and coarse-grained global information is constructed, so that the model obtains the detection capability for targets with different scales.

Description

Technical Field

[0001] The present invention belongs to the technical field of fan blade damage detection, and in particular relates to a fan blade surface damage detection method and system. Background Art

[0002] Since wind turbines are usually installed in harsh environments such as the wild, cracks, oil stains, and sand holes may occur on the surface of the wind turbine blades during service. These damages not only affect the efficiency of wind turbine power generation, but may also pose a serious threat to the safety of the equipment. For example, if cracks on the surface of wind turbine blades are not discovered and handled in time, they may continue to expand under the action of wind, eventually causing the blades to break and causing serious safety accidents. Therefore, it is of great significance to detect damage to wind turbine blades.

[0003] In related technologies, wind turbine blade surface damage detection is mainly carried out through image detection, that is, using a drone equipped with a camera to take images of blades running in the air, and then using image recognition algorithms to detect damaged parts. However, due to the randomness of the damage location, it is very difficult to extract image features, which can easily lead to low detection accuracy. In addition, for some minor damage or damage with a high similarity to the background, missed detection or false detection may occur.

[0004] Therefore, it is necessary to provide a method and system for detecting surface damage of wind turbine blades to solve the above problems. Summary of the invention

[0005] The present invention provides a method and system for detecting surface damage of a wind turbine blade. The method adopts a combination of a pyramid enhancement algorithm and a patch enhancement algorithm. The pyramid enhancement algorithm provides global multi-scale information, and the patch enhancement algorithm provides local detail information, which can enable a model to learn from information at different levels and improve the accuracy of damage detection on the wind turbine blade. In the training stage of the model, an attention mechanism is adopted to focus on damage-related features in an image, suppress features unrelated to the damage, generate enhanced features, map the enhanced features from Cartesian coordinates to a polar coordinate system to form polar coordinate features, highlight the expression of damage features in the radial and axial directions of the wind turbine blades, and then fuse the polar coordinate features with the enhanced features to construct a feature map that fuses fine-grained local information and coarse-grained global information, so that an improved YOLOv8 model can obtain the ability to detect targets of different scales, and can effectively solve at least one technical problem involved in the background technology.

[0006] In order to solve the above-mentioned technical problems, the present invention is achieved as follows: A method for detecting surface damage of a fan blade comprises the following steps: Step S1, obtaining an original data set, wherein the original data set includes a plurality of original images of the surface of a wind turbine blade, each original image being marked with a damage location and a damage type; Step S2, using a pyramid enhancement algorithm to enhance each original image in the original data set to form a pyramid image sequence, then using overlapping sliding windows to extract patches from each layer in the pyramid image sequence to form a patch set, and reconstructing the training data set with the pyramid image sequence and the patch set; Step S3, constructing an improved YOLOv8 model based on the traditional YOLOv8 model, the neck network and detection head of the improved YOLOv8 model are consistent with the architecture in the traditional YOLOv8 model, the backbone network includes an initial convolutional layer, a C2F-FocalNextBlock module, a polar coordinate feature transformation module and a feature fusion module, and sending the training data set and its corresponding labels to the improved YOLOv8 model for training to obtain a trained improved YOLOv8 model; the training process includes the following steps: sending the images in the training data set to the initial convolutional layer to extract basic features, and then sending the extracted basic features to the C2F-FocalNextBlock module, Based on the attention mechanism, the features related to damage in the image are focused on, and the features unrelated to damage are suppressed to generate enhanced features. Then, the enhanced features are sent to the polar coordinate feature transformation module, and the enhanced features are mapped from the Cartesian coordinate system to the polar coordinate system to form polar coordinate features, highlighting the expression of damage features in the radial and axial directions of the wind turbine blades. Finally, the polar coordinate features output by the polar coordinate feature transformation module are fused with the enhanced features output by the C2F-FocalNextBlock module to construct fused features that combine fine-grained local information and coarse-grained global information. The neck network converts the fused features output by the backbone network into multi-scale features suitable for the detection task, and passes them to the detection head for the final bounding box regression and classification. Step S4, for any wind blade surface damage detection process, first collect the image of the wind blade surface, then extract the pyramid image sequence and patch set, use the pyramid image sequence and patch set as the input of the trained improved YOLOv8 model, and output the prediction results of the damage location and damage type in each image.

[0007] As a preferred improvement, the damage types include "dirt" and "damage".

[0008] As a preferred improvement, the generation process of the pyramid image sequence specifically includes the following steps: Step S211, setting the zoom ratio and the minimum image size ; Step S212: according to the scaling ratio For the original image Downsampling is performed layer by layer to generate the next layer of images with lower resolution. The downsampling process is expressed as: In the formula, , Respectively represent and Layer images, ; Represents the scaling operation, and the scaling process is expressed as: In the formula, , Respectively represent The height and width of the layer image; and Respectively represent The height and width of the layer image; Step S213, when or When , stop downsampling and get the pyramid image sequence , , Indicates the total number of pyramid layers.

[0009] As a preferred improvement, the process of generating a patch set specifically includes the following steps: Step S221, set the size of the sliding window to , the horizontal step size of the sliding window and vertical step length ; Step S222: Under the condition that two adjacent windows have a 10% overlap rate, patch sampling is performed along the horizontal and vertical directions of the image to obtain a patch set .

[0010] As a preferred improvement, the initial convolutional layer adopts the CSPDarknet architecture; the C2F-FocalNextBlock module introduces the focal block module in the CFNet network on the basis of the traditional C2F network, and replaces the neck network of the C2F network with the focal block module. The processing process of the C2F-FocalNextBlock module is expressed as follows: In the formula, Represents the processing of the C2F-FocalNextBlock module.

[0011] As a preferred improvement, the focal block module includes a 7×7 deep convolution layer, a window attention mechanism with a window size of 7×7, and a 1×1 convolution layer connected in sequence. Each 7×7 deep convolution layer and the window attention mechanism are connected to a LayerNorm layer and a GELU unit. The processing process of the focal block module is expressed as follows: In the formula, Represents the input features of the focal block module; Represents 7×7 depth-wise separable convolution; Represents 1×1 convolution; express Activation function; Representation layer normalization; represents a 7×7 window attention mechanism, and the attention weight is calculated by the following formula: In the formula, Represents input features X The attention weight of Represent the query, key, and value in the attention mechanism respectively; Features X Dimensions; Represents matrix transpose; express Activation function; Represents a linear regression operation.

[0012] As a preferred improvement, the processing process of the polar coordinate feature transformation module includes the following steps: Step S31, from the enhanced features Predict wind turbine blade center coordinates , the prediction process is expressed as: In the formula, Respectively represent the horizontal and vertical coordinates of the center of the fan blade in the Cartesian coordinate system; express Sigmoid Activation function; express convolution; Represents enhanced features The feature map width and height of Step S32: Enhance the feature Mapping from the Cartesian coordinate system to the polar coordinate system, the polar coordinate features are generated by bilinear interpolation, where: The mapping process is expressed as: In the formula, Respectively represent the horizontal and vertical coordinates of the points on the feature map in the Cartesian coordinate system; , Represents the points on the feature map The horizontal and vertical coordinates in the polar coordinate system, where ; ; Represents enhanced features The maximum radial distance of the characteristic map, ; The polar coordinate feature generation process is expressed as: In the formula, Represents polar coordinate features; Represents the variable used to index the feature map grid points in Cartesian coordinates; represents the interpolation weight, which is calculated by the bilinear interpolation formula; Step S33, applying radial-angular separable convolution to the polar coordinate features, inversely transforming the convolved polar coordinate features back to the Cartesian coordinate system, and combining them with the original enhanced features The fusion process is expressed as: In the formula, Represents 1D convolution along the radial direction; represents a 1D convolution along the angular direction; Represents the polar coordinate features after convolution; Polar coordinate features representing the inverse transformation back to Cartesian coordinates; Represents the inverse transformation from Cartesian to polar coordinates; represents the weight factor, , represents a learnable parameter; Represents fusion features.

[0013] As a preferred improvement, the fusion feature It is sent to the neck network, which fuses feature maps of different levels through the feature pyramid network and path aggregation network, converts them into multi-scale features suitable for the detection task, and passes them to the detection head for final bounding box regression and classification.

[0014] As a preferred improvement, the loss function of training is expressed as: In the formula, Represents the total loss function; represents the bounding box loss, which is used to measure the accuracy of determining the target location; Represents the classification loss, which is used to measure the accuracy of classifying the target category; represents the distribution focal loss, which is used to measure how close the target is to the target location; , , represents a hyperparameter used to balance the importance of different losses; The training process is through Optimizer to update model parameters.

[0015] A system for executing the above-mentioned wind turbine blade surface damage detection method comprises: The original data set acquisition module is used to obtain the original data set, wherein the original data set includes a plurality of original images of the surface of the wind turbine blade, each original image being marked with a damage location and a damage type; A training data set construction module is used to enhance each original image in the original data set using a pyramid enhancement algorithm to form a pyramid image sequence, and then use overlapping sliding windows to extract patches from each layer in the pyramid image sequence to form a patch set, and reconstruct the training data set with the pyramid image sequence and the patch set; The improved YOLOv8 model construction module is used to build an improved YOLOv8 model based on the traditional YOLOv8 model. The neck network and detection head of the improved YOLOv8 model are consistent with the architecture of the traditional YOLOv8 model. The backbone network includes an initial convolution layer, a C2F-FocalNextBlock module, a polar coordinate feature transformation module, and a feature fusion module. The training data set and its corresponding labels are sent to the improved YOLOv8 model for training to obtain a trained improved YOLOv8 model. The training process includes the following steps: sending the images in the training data set to the initial convolution layer to extract basic features, and then sending the extracted basic features to the C2F-FocalNextBl The FocalNextBlock module focuses on the features related to damage in the image based on the attention mechanism, suppresses the features not related to damage, and generates enhanced features; then the enhanced features are sent to the polar coordinate feature transformation module, and the enhanced features are mapped from the Cartesian coordinate system to the polar coordinate system to form polar coordinate features, highlighting the expression of damage features in the radial and axial directions of the wind turbine blades; finally, the polar coordinate features output by the polar coordinate feature transformation module are fused with the enhanced features output by the C2F-FocalNextBlock module to construct a fusion feature that combines fine-grained local information and coarse-grained global information; the neck network converts the fusion features output by the backbone network into multi-scale features suitable for the detection task, and passes them to the detection head for the final bounding box regression and classification; The detection module is used for any wind turbine blade surface damage detection process. First, the image of the wind turbine blade surface is collected, and then the pyramid image sequence and patch set are extracted. The pyramid image sequence and patch set are used as the input of the trained improved YOLOv8 model, and the prediction results of the damage location and damage type in each image are output.

[0016] The beneficial effects of the present invention are: (1) The present invention adopts a combination of the pyramid enhancement algorithm and the patch enhancement algorithm, and uses the pyramid enhancement algorithm to generate multi-scale images. The multi-scale images allow the model to learn its features at different resolutions to avoid missed detections due to large scale differences; the patch enhancement algorithm divides the image into multiple small blocks, each of which serves as an independent sample, which helps the model learn local features. For local minor damage on the surface of the wind turbine blade, the patch enhancement method can provide more learning opportunities and enhance its feature representation. The pyramid enhancement algorithm provides global multi-scale information, and the patch enhancement algorithm provides local detail information, which allows the model to learn from information at different levels and improve the accuracy of damage detection; (2) The neck network in the C2F module in the traditional YOLOv8 model backbone network is replaced by the focal block module in the CFnet network. The attention mechanism in the focal block module is used to automatically focus on the important parts of the image related to the injury and suppress background information that is not related to the injury, so that the model can learn the injury features more effectively and improve the ability to recognize the injury. (3) The enhanced features extracted by the C2F-FocalNextBlock module are mapped from Cartesian coordinates to polar coordinates using the polar coordinate feature transformation module. This can express the rotational structural information of the wind blades in a more direct way, enabling the model to better capture the overall structural characteristics of the wind blades, such as the circumferential distribution and radial extension of the wind blades, thereby more accurately identifying rotation-related damage patterns. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work, among which: Figure 1 A network architecture diagram showing the improved YOLOv8 model provided by the present invention; Figure 2 Indicates the detection result of the fan blade surface damage detection method provided by the present invention for the "dirt" type of damage; Figure 3 It represents the detection result of the wind turbine blade surface damage detection method provided by the present invention for the "damage" type of damage. DETAILED DESCRIPTION

[0018] The following will be combined with the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0019] See also Figure 1 This embodiment provides a method for detecting surface damage of a wind turbine blade, comprising the following steps: Step S1, obtaining an original data set, wherein the original data set includes a plurality of original images of the surface of a wind turbine blade, each original image being marked with a damage location and a damage type.

[0020] The original data set adopts the fan blade detection image data set proposed by A. Shihavuddin in the prior art, which includes 2995 images with a size of 586×371 pixels. Each image uses the YOLO format to annotate the damage location and damage type. The annotated damage types include "dirt" and "damage", among which: there are 581 "dirt" labels and 8770 "damage" labels.

[0021] Step S2, using a pyramid enhancement algorithm to enhance each original image in the original data set to form a pyramid image sequence, and then using overlapping sliding windows to extract patches from each layer in the pyramid image sequence to form a patch set, and reconstructing the training data set with the pyramid image sequence and the patch set.

[0022] The original data set includes images of the surface of wind turbine blades taken from different distances. The sizes of the same parts of wind turbine blades in different images vary greatly. For example, when shooting at a close distance, the details of the surface of the wind turbine blades can be clearly displayed, but the image may only show part of the structure of the wind turbine blades and cannot fully display the overall structure of the wind turbine blades; when shooting at a long distance, although the overall structure of the wind turbine can be fully displayed, the surface details will become blurred, and the proportion of the wind turbine blades in the image is small.

[0023] In order to better learn this scale difference, the present invention adopts a combination of pyramid enhancement algorithm and patch enhancement algorithm, and uses the pyramid enhancement algorithm to generate multi-scale images. For example, for damage of different sizes (from small wear points to large cracks), multi-scale images allow the model to learn its features at different resolutions to avoid missed detection due to large scale differences; while the patch enhancement algorithm divides the image into multiple small blocks, each small block as an independent sample, which helps to learn local features. For local subtle damage on the surface of the fan blade, the patch enhancement method can provide more learning opportunities and enhance its feature representation. The pyramid enhancement algorithm provides global multi-scale information, and the patch enhancement algorithm provides local detail information, which allows the model to learn from information at different levels and improve the accuracy of fan blade damage detection.

[0024] The generation process of the pyramid image sequence specifically includes the following steps: Step S211, setting the zoom ratio and the minimum image size ; Step S212: according to the scaling ratio For the original image Downsampling is performed layer by layer to generate the next layer of images with lower resolution. The downsampling process is expressed as: In the formula, , Respectively represent and Layer images, ; Represents the scaling operation, and the scaling process is expressed as: In the formula, , Respectively represent The height and width of the layer image; and Respectively represent The height and width of the layer image; Step S213, when or When , stop downsampling and get the pyramid image sequence , , Indicates the total number of pyramid layers.

[0025] The process of generating a patch set specifically includes the following steps: Step S221, set the size of the sliding window to , the horizontal step size of the sliding window and vertical step length ; Step S222: Under the condition that two adjacent windows have a 10% overlap rate, patch sampling is performed along the horizontal and vertical directions of the image to obtain a patch set .

[0026] Step S3, constructing an improved YOLOv8 model based on the traditional YOLOv8 model, the neck network and detection head of the improved YOLOv8 model are consistent with the architecture in the traditional YOLOv8 model, the backbone network includes an initial convolutional layer, a C2F-FocalNextBlock module, a polar coordinate feature transformation module and a feature fusion module, and sending the training data set and its corresponding labels to the improved YOLOv8 model for training to obtain a trained improved YOLOv8 model; the training process includes the following steps: sending the images in the training data set to the initial convolutional layer to extract basic features, and then sending the extracted basic features to the C2F-FocalNextBlock module, Based on the attention mechanism, the features related to damage in the image are focused on, the features unrelated to damage are suppressed, and enhanced features are generated; then the enhanced features are sent to the polar coordinate feature transformation module, and the enhanced features are mapped from the Cartesian coordinate system to the polar coordinate system to form polar coordinate features, highlighting the expression of damage features in the radial and axial directions of the wind turbine blades; finally, the polar coordinate features output by the polar coordinate feature transformation module are fused with the enhanced features output by the C2F-FocalNextBlock module to construct a fused feature that combines fine-grained local information and coarse-grained global information; the neck network converts the fused features output by the backbone network into multi-scale features suitable for the detection task, and passes them to the detection head for the final bounding box regression and classification.

[0027] The initial convolutional layer adopts the conventional CSPDarknet architecture in the art, and its processing process of the input image also belongs to the conventional technology in the art, which will not be described in detail in this embodiment.

[0028] In order to improve aerodynamic performance, the surface of wind turbine blades usually has complex textures, which appear as lines and patterns of different directions and intensities in the image. In addition, different parts of wind turbine blades have unique shapes, forming specific geometric contours in the image, and their edges and surfaces vary widely, which brings challenges to feature extraction and target detection.

[0029] In order to solve this problem, the present invention introduces the focalblock module in the CFNet network based on the traditional C2F network, and constructs the C2F-FocalNextBlock module by replacing the neck network of the C2F network with the focal block module. The focal block module can help the network automatically focus on those areas that are more important for damage detection, such as key damage features such as cracks and wear, based on the attention mechanism, while suppressing some irrelevant background or normal area features.

[0030] Basic features extracted by the initial convolutional layer Sent to the C2F-FocalNextBlock module, outputting enhanced features , the processing process is expressed as: In the formula, Represents the processing of the C2F-FocalNextBlock module.

[0031] The focal block module consists of a 7×7 deep convolutional layer, a window attention mechanism with a window size of 7×7, and a 1×1 convolutional layer connected in sequence. Each 7×7 deep convolutional layer and the window attention mechanism are connected to a LayerNorm layer and a GELU unit.

[0032] After the input features enter the focal block module, they first undergo an expanded deep convolution operation to expand the receptive field of the convolution kernel without increasing the number of parameters, allowing neurons to obtain a wider range of information. Next, the original input features are fused with the convolutional features through a jump connection to retain important information in the original features and prevent information loss. Subsequently, a 1×1 convolution operation is performed to adjust the number of channels to adapt to the subsequent network structure requirements while reducing the amount of calculation. Nonlinearity is then introduced through the GELU activation function to enhance the model's expressiveness. The LayerNorm layer after each 7×7 deep convolution layer and window attention mechanism operation normalizes the features to accelerate model convergence and improve stability.

[0033] The processing of the focal block module is expressed as: In the formula, Represents the input features of the focal block module; Represents 7×7 depth-wise separable convolution; Represents 1×1 convolution; express Activation function; Representation layer normalization; represents a 7×7 window attention mechanism, and the attention weight is calculated by the following formula: In the formula, Represents input features X The attention weight of Represent the query, key, and value in the attention mechanism respectively; Features X Dimensions; Represents matrix transpose; express Activation function; Represents a linear regression operation.

[0034] Wind turbine blades usually have a rotationally symmetric geometric structure. Traditional Cartesian coordinates are not intuitive enough when describing such rotational features. The polar coordinate system uses distance and angle to locate points, which is naturally suitable for describing objects with rotational characteristics. Therefore, the present invention uses a polar coordinate feature transformation module to map the enhanced features output by the C2F-FocalNextBlock module from the Cartesian coordinate system to the polar coordinate system, which can express the rotational structural information of the wind turbine blades in a more direct way, so that the model can better capture the overall structural features of the wind turbine blades, such as the circumferential distribution and radial extension of the wind turbine blades, so as to more accurately identify the damage mode related to rotation.

[0035] In the polar coordinate system, the radial direction (from the center to the edge) and circumferential direction (circumferential direction around the center) features of the wind blades can be clearly separated and expressed. For wind blades, damage may develop along the radial direction (such as cracks from the root to the tip of the blade) or the circumferential direction (such as wear on the circumference of the blade). The polar coordinate feature transformation module can specifically extract features in these two directions, which helps the model to more keenly perceive damage features in different directions, and then effectively fuse the extracted local damage details with the global wind turbine structure information. Locally, it can capture damage features such as tiny cracks and corrosion spots; globally, it can grasp the overall shape and layout of the wind blades, providing more comprehensive information for accurate judgment of damage. This fusion of local and global features helps to improve the model's recognition accuracy and positioning accuracy for damage.

[0036] In addition, in the Cartesian coordinate system, the feature extraction of wind blade images may contain some redundant information irrelevant to damage detection, and is also easily interfered by background noise. In the polar coordinate system, the feature extraction of wind blade images can focus on key features related to blade rotation and structure, reduce unnecessary information interference, make the model more focused on learning and identifying damage features, and improve the accuracy and robustness of damage detection.

[0037] The angle and distance information in the polar coordinate system can provide a more accurate coordinate description for the location of damage. For wind turbine blade damage detection, it is crucial to know the exact location of the damage, which helps with subsequent repair and maintenance work. The polar coordinate feature module can use this precise coordinate information to more accurately locate the damage and improve the reliability of detection; various complex damage patterns may occur in wind turbines during actual operation, such as multi-angle cracks, irregular corrosion areas, etc. Polar coordinate features can better adapt to these complex damage forms. Through the analysis and learning of polar coordinate features, the model can more effectively identify and distinguish different types of damage and improve the detection capabilities of complex damage patterns.

[0038] The processing of the polar coordinate feature transformation module includes the following steps: Step S31, from the enhanced features Predict wind turbine blade center coordinates , the prediction process is expressed as: In the formula, Respectively represent the horizontal and vertical coordinates of the center of the fan blade in the Cartesian coordinate system; express Sigmoid Activation function; express convolution; Represents enhanced features The feature map width and height of Step S32: Enhance the feature Mapping from the Cartesian coordinate system to the polar coordinate system, the polar coordinate features are generated by bilinear interpolation, where: The mapping process is expressed as: In the formula, Respectively represent the horizontal and vertical coordinates of the points on the feature map in the Cartesian coordinate system; , Represents the points on the feature map The horizontal and vertical coordinates in the polar coordinate system, where ; ; Represents enhanced features The maximum radial distance of the characteristic map, ; The polar coordinate feature generation process is expressed as: In the formula, Represents polar coordinate features; Represents variables used to index the grid points of the feature map in Cartesian coordinates, through which the eigenvalues ​​of each grid point can be obtained; represents the interpolation weight, which is calculated by the bilinear interpolation formula; Step S33, applying radial-angular separable convolution to the polar coordinate features, inversely transforming the convolved polar coordinate features back to the Cartesian coordinate system, and combining them with the original enhanced features The fusion process is expressed as: In the formula, Represents 1D convolution along the radial direction; represents a 1D convolution along the angular direction; Represents the polar coordinate features after convolution; Polar coordinate features representing the inverse transformation back to Cartesian coordinates; Represents the inverse transformation from Cartesian to polar coordinates; represents the weight factor, , represents a learnable parameter; Represents fusion features.

[0039] After converting the Cartesian coordinate image into a polar coordinate image, the radial and angular separable convolutions are more closely matched with the structure of the polar coordinate space, and can make full use of the distribution law of features in the polar coordinate space, extract features in the radial and angular directions respectively, and better capture the feature information in these different directions.

[0040] Fusion Features It is sent to the neck network, which fuses feature maps of different levels through the feature pyramid network (FPN) and the path aggregation network (PAN), converts them into multi-scale features suitable for the detection task, and passes them to the detection head for final bounding box regression and classification, and outputs the prediction results of the damage location and damage type; finally, through post-processing methods such as non-maximum suppression (NMS), the multiple prediction results output by the detection head are screened and merged, and according to factors such as the category probability and overlap degree of the bounding box, the redundant bounding boxes with too high overlap are removed to obtain the final target detection result.

[0041] The loss function of training is expressed as: In the formula, Represents the total loss function; represents the bounding box loss, which is used to measure the accuracy of determining the target location; Represents the classification loss, which is used to measure the accuracy of classifying the target category; represents the distribution focal loss, which is used to measure how close the target is to the target location; , , represents a hyperparameter used to balance the importance of different losses.

[0042] The training process updates the model parameters through the optimizer, and the specific choice of optimizer is .

[0043] Step S4, for any wind blade surface damage detection process, first collect the image of the wind blade surface, then extract the pyramid image sequence and patch set, use the pyramid image sequence and patch set as the input of the trained improved YOLOv8 model, and output the prediction results of the damage location and damage type in each image.

[0044] The acquisition of wind turbine blade surface images is achieved with the help of drones, which use drones equipped with cameras to capture images of wind turbine blade surfaces in the air. After image acquisition is completed, pre-processing is required, including grayscale, denoising, and illumination equalization, to improve image quality.

[0045] See also Figure 2 and Figure 3 , Figure 2 Indicates the detection result of the fan blade surface damage detection method provided by the present invention for the "dirt" type of damage; Figure 3 The figure shows the detection results of the wind turbine blade surface damage detection method provided by the present invention for the "damage" type of damage. Accurate prediction is achieved under both types of damage, indicating the effectiveness of the wind turbine blade surface damage detection method provided by the present invention.

[0046] This embodiment also provides a system for executing the above-mentioned wind turbine blade surface damage detection method, comprising: The original data set acquisition module is used to obtain the original data set, wherein the original data set includes a plurality of original images of the surface of the wind turbine blade, each original image being marked with a damage location and a damage type; A training data set construction module is used to enhance each original image in the original data set using a pyramid enhancement algorithm to form a pyramid image sequence, and then use overlapping sliding windows to extract patches from each layer in the pyramid image sequence to form a patch set, and reconstruct the training data set with the pyramid image sequence and the patch set; The improved YOLOv8 model construction module is used to build an improved YOLOv8 model based on the traditional YOLOv8 model. The neck network and detection head of the improved YOLOv8 model are consistent with the architecture of the traditional YOLOv8 model. The backbone network includes an initial convolution layer, a C2F-FocalNextBlock module, a polar coordinate feature transformation module, and a feature fusion module. The training data set and its corresponding labels are sent to the improved YOLOv8 model for training to obtain a trained improved YOLOv8 model. The training process includes the following steps: sending the images in the training data set to the initial convolution layer to extract basic features, and then sending the extracted basic features to the C2F-FocalNextBl The FocalNextBlock module focuses on the features related to damage in the image based on the attention mechanism, suppresses the features not related to damage, and generates enhanced features; then the enhanced features are sent to the polar coordinate feature transformation module, and the enhanced features are mapped from the Cartesian coordinate system to the polar coordinate system to form polar coordinate features, highlighting the expression of damage features in the radial and axial directions of the wind turbine blades; finally, the polar coordinate features output by the polar coordinate feature transformation module are fused with the enhanced features output by the C2F-FocalNextBlock module to construct a fusion feature that combines fine-grained local information and coarse-grained global information; the neck network converts the fusion features output by the backbone network into multi-scale features suitable for the detection task, and passes them to the detection head for the final bounding box regression and classification; The detection module is used for any wind turbine blade surface damage detection process. First, the image of the wind turbine blade surface is collected, and then the pyramid image sequence and patch set are extracted. The pyramid image sequence and patch set are used as the input of the trained improved YOLOv8 model, and the prediction results of the damage location and damage type in each image are output.

[0047] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation modes, which are merely illustrative rather than restrictive. Under the guidance of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the present invention and the claims, all of which are within the protection of the present invention.

Claims

1. A method for detecting surface damage of a wind turbine blade, characterized in that: The steps include: Step S1, obtaining an original data set, wherein the original data set includes a plurality of original images of the surface of a wind turbine blade, each original image being marked with a damage location and a damage type; Step S2, using a pyramid enhancement algorithm to enhance each original image in the original data set to form a pyramid image sequence, then using overlapping sliding windows to extract patches from each layer in the pyramid image sequence to form a patch set, and reconstructing the training data set with the pyramid image sequence and the patch set; Step S3, constructing an improved YOLOv8 model based on the traditional YOLOv8 model, the neck network and detection head of the improved YOLOv8 model are consistent with the architecture in the traditional YOLOv8 model, the backbone network includes an initial convolutional layer, a C2F-FocalNextBlock module, a polar coordinate feature transformation module and a feature fusion module, and sending the training data set and its corresponding labels to the improved YOLOv8 model for training to obtain a trained improved YOLOv8 model; the training process includes the following steps: sending the images in the training data set to the initial convolutional layer to extract basic features, and then sending the extracted basic features to the C2F-FocalNextBlock module, Based on the attention mechanism, the features related to damage in the image are focused on, and the features unrelated to damage are suppressed to generate enhanced features. Then, the enhanced features are sent to the polar coordinate feature transformation module, and the enhanced features are mapped from the Cartesian coordinate system to the polar coordinate system to form polar coordinate features, highlighting the expression of damage features in the radial and axial directions of the wind turbine blades. Finally, the polar coordinate features output by the polar coordinate feature transformation module are fused with the enhanced features output by the C2F-FocalNextBlock module to construct fused features that combine fine-grained local information and coarse-grained global information. The neck network converts the fused features output by the backbone network into multi-scale features suitable for the detection task, and passes them to the detection head for the final bounding box regression and classification. Step S4, for any wind blade surface damage detection process, first collect the image of the wind blade surface, then extract the pyramid image sequence and patch set, use the pyramid image sequence and patch set as the input of the trained improved YOLOv8 model, and output the prediction results of the damage location and damage type in each image.

2. The method for detecting surface damage of a wind turbine blade according to claim 1, characterized in that: Damage types include Dirt and Damage.

3. The method for detecting surface damage of a wind turbine blade according to claim 1, characterized in that: The generation process of the pyramid image sequence specifically includes the following steps: Step S211, setting the zoom ratio and the minimum image size ; Step S212: according to the scaling ratio For the original image Downsampling is performed layer by layer to generate the next layer of images with lower resolution. The downsampling process is expressed as: In the formula, , Respectively represent and Layer images, ; Represents the scaling operation, and the scaling process is expressed as: In the formula, , Respectively represent The height and width of the layer image; and Respectively represent The height and width of the layer image; Step S213, when or When , stop downsampling and get the pyramid image sequence , , Indicates the total number of pyramid layers.

4. The method for detecting surface damage of a wind turbine blade according to claim 3, characterized in that: The process of generating a patch set specifically includes the following steps: Step S221, set the size of the sliding window to , the horizontal step size of the sliding window and vertical step length ; Step S222: Under the condition that two adjacent windows have a 10% overlap rate, patch sampling is performed along the horizontal and vertical directions of the image to obtain a patch set .

5. The method for detecting surface damage of a wind turbine blade according to claim 1, characterized in that: The initial convolutional layer adopts the CSPDarknet architecture; The C2F-FocalNextBlock module introduces the focal block module in the CFNet network on the basis of the traditional C2F network, and replaces the neck network of the C2F network with the focal block module. The processing process of the C2F-FocalNextBlock module is expressed as follows: In the formula, Represents the processing of the C2F-FocalNextBlock module.

6. The method for detecting surface damage of a wind turbine blade according to claim 5, characterized in that: The focal block module consists of a 7×7 deep convolutional layer, a window attention mechanism with a window size of 7×7, and a 1×1 convolutional layer. Each 7×7 deep convolutional layer and the window attention mechanism are connected to a LayerNorm layer and a GELU unit. The processing process of the focalblock module is expressed as follows: In the formula, Represents the input features of the focal block module; Represents 7×7 depth-wise separable convolution; Represents 1×1 convolution; express Activation function; Representation layer normalization; represents a 7×7 window attention mechanism, and the attention weight is calculated by the following formula: In the formula, Represents input features X The attention weight of Represent the query, key, and value in the attention mechanism respectively; Features X Dimensions; Represents matrix transpose; express Activation function; Represents a linear regression operation.

7. The method for detecting surface damage of a wind turbine blade according to claim 6, characterized in that: The processing of the polar coordinate feature transformation module includes the following steps: Step S31, from the enhanced features Predict wind turbine blade center coordinates , the prediction process is expressed as: In the formula, Respectively represent the horizontal and vertical coordinates of the center of the fan blade in the Cartesian coordinate system; express Sigmoid Activation function; express convolution; Represents enhanced features The feature map width and height of Step S32: Enhance the feature Mapping from the Cartesian coordinate system to the polar coordinate system, the polar coordinate features are generated by bilinear interpolation, where: The mapping process is expressed as: In the formula, Respectively represent the horizontal and vertical coordinates of the points on the feature map in the Cartesian coordinate system; , Represents the points on the feature map The horizontal and vertical coordinates in the polar coordinate system, where ; ; Represents enhanced features The maximum radial distance of the characteristic map, ; The polar coordinate feature generation process is expressed as: In the formula, Represents polar coordinate features; Represents the variable used to index the feature map grid points in Cartesian coordinates; Represents the interpolation weight, which is calculated by the bilinear interpolation formula; Step S33, applying radial-angular separable convolution to the polar coordinate features, inversely transforming the convolved polar coordinate features back to the Cartesian coordinate system, and combining them with the original enhanced features The fusion process is expressed as: In the formula, Represents 1D convolution along the radial direction; represents a 1D convolution along the angular direction; Represents the polar coordinate features after convolution; Polar coordinate features representing the inverse transformation back to Cartesian coordinates; Represents the inverse transformation from Cartesian to polar coordinates; represents the weight factor, , represents a learnable parameter; Represents fusion features.

8. The method for detecting surface damage of a wind turbine blade according to claim 7, characterized in that: Fusion Features It is sent to the neck network, which fuses feature maps of different levels through the feature pyramid network and path aggregation network, converts them into multi-scale features suitable for the detection task, and passes them to the detection head for final bounding box regression and classification.

9. The method for detecting surface damage of a wind turbine blade according to claim 1, characterized in that: The loss function of training is expressed as: In the formula, Represents the total loss function; represents the bounding box loss, which is used to measure the accuracy of determining the target location; Represents the classification loss, which is used to measure the accuracy of classifying the target category; represents the distribution focal loss, which is used to measure how close the target is to the target location; , , represents a hyperparameter used to balance the importance of different losses; The training process is through Optimizer to update model parameters.

10. A system for executing the wind turbine blade surface damage detection method according to any one of claims 1 to 9, characterized in that: include: The original data set acquisition module is used to obtain the original data set, wherein the original data set includes a plurality of original images of the surface of the wind turbine blade, each original image being marked with a damage location and a damage type; A training data set construction module is used to enhance each original image in the original data set using a pyramid enhancement algorithm to form a pyramid image sequence, and then use overlapping sliding windows to extract patches from each layer in the pyramid image sequence to form a patch set, and reconstruct the training data set with the pyramid image sequence and the patch set; The improved YOLOv8 model construction module is used to build an improved YOLOv8 model based on the traditional YOLOv8 model. The neck network and detection head of the improved YOLOv8 model are consistent with the architecture of the traditional YOLOv8 model. The backbone network includes an initial convolution layer, a C2F-FocalNextBlock module, a polar coordinate feature transformation module, and a feature fusion module. The training data set and its corresponding labels are sent to the improved YOLOv8 model for training to obtain a trained improved YOLOv8 model. The training process includes the following steps: sending the images in the training data set to the initial convolution layer to extract basic features, and then sending the extracted basic features to the C2F-FocalNextBl The FocalNextBlock module focuses on the features related to damage in the image based on the attention mechanism, suppresses the features not related to damage, and generates enhanced features; then the enhanced features are sent to the polar coordinate feature transformation module, and the enhanced features are mapped from the Cartesian coordinate system to the polar coordinate system to form polar coordinate features, highlighting the expression of damage features in the radial and axial directions of the wind turbine blades; finally, the polar coordinate features output by the polar coordinate feature transformation module are fused with the enhanced features output by the C2F-FocalNextBlock module to construct a fusion feature that combines fine-grained local information and coarse-grained global information; the neck network converts the fusion features output by the backbone network into multi-scale features suitable for the detection task, and passes them to the detection head for the final bounding box regression and classification; The detection module is used for any wind turbine blade surface damage detection process. First, the image of the wind turbine blade surface is collected, and then the pyramid image sequence and patch set are extracted. The pyramid image sequence and patch set are used as the input of the trained improved YOLOv8 model, and the prediction results of the damage location and damage type in each image are output.

Images