Wind turbine blade surface damage detection method based on multi-attribute perception attention
By introducing a multi-attribute perceived attention mechanism in wind power blade damage detection, the feature extraction and bounding box generation capabilities of the detection model are improved, and the problem of low accuracy of damage detection in complex backgrounds is solved, and more efficient damage detection is achieved.
Patent Information
- Application Number
- CN202310631464.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-31
- Publication Date
- 2025-05-16
- Estimated Expiration
- 2043-05-31
AI Technical Summary
Wind power blade damage detection has low accuracy in complex backgrounds, and existing machine vision methods are difficult to effectively identify the damaged parts of the blade.
The surface damage detection method of wind power blades based on multi-attribute perceived attention is adopted. By establishing a multi-category damage image database, embedding channel-spatial attention module and designing a multi-attribute perceived loss function, the feature extraction capability and bounding box generation accuracy of the detection model are improved.
The accuracy and efficiency of wind power blade damage detection is significantly improved in complex backgrounds, and is especially suitable for detection tasks with complex backgrounds and light damage.
Smart Images
Figure CN116596909B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of wind turbine blade surface damage detection, and in particular to a wind turbine blade surface damage detection method based on multi-attribute perception attention. Background Art
[0002] Wind turbine blades are an important component of wind turbines, accounting for about 20% of the total cost of the unit, and directly affecting the ability of wind turbines to capture wind energy. Wind farms are usually installed in remote areas such as mountains, deserts and offshore areas, which makes the blades directly exposed to sand, salt spray and lightning. Therefore, the blades are one of the most vulnerable parts of the unit.
[0003] With the development of computing platform computing power, machine vision methods based on deep learning have played an important role in damage detection in many fields. At present, the main method of wind turbine blade damage detection is still manual, specifically, wind turbine operation and maintenance personnel use high-altitude hanging baskets or ground-based high-power telescopes to detect the health of blades, but this method is inefficient and the accuracy is easily affected by workers' experience. According to relevant research, the detection method based on machine vision only requires drones equipped with image sensors, which can significantly reduce turbine downtime from 1.5 hours (using climbers and telescopes) to an average of 20 minutes. Therefore, the method based on machine vision has become an important development direction for wind turbine blade damage detection.
[0004] Machine vision-based methods have achieved some results in many fields, but there are still many constraints in wind turbine blade damage detection. Since wind farms are usually built in the wild with complex backgrounds and wind turbine blades are huge, the damaged parts are relatively difficult to detect in the entire visual perception surface, which seriously affects the accuracy of wind turbine blade damage detection. Summary of the invention
[0005] In order to solve the above technical problems, the present invention provides a wind turbine blade surface damage detection method based on multi-attribute perception attention which has a simple algorithm and is accurate and efficient.
[0006] The technical solution of the present invention to solve the above problem is: a method for detecting surface damage of wind turbine blades based on multi-attribute perception attention, comprising the following steps:
[0007] 1) Establish a database of multiple types of wind turbine blade damage images: collect different types of wind turbine blade damage image data, and use image processing methods to expand the number and enhance the quality of wind turbine blade images;
[0008] 2) Establishing a channel-spatial attention enhanced feature extraction backbone network: By sequentially embedding the channel attention and spatial attention modules in the feature extraction network, a damage feature enhanced feature extraction network is established;
[0009] 3) Establishing a multi-attribute perception loss function for the damage bounding box: In view of the complex information of the bounding box, a multi-attribute loss function for the bounding box is established. By perceiving multiple attributes of the bounding box, the difference between the predicted box and the true box is described, thereby guiding the model to generate an accurate predicted box.
[0010] 4) Construct a wind turbine blade damage detection model: Construct a wind turbine blade damage detection model and use a database of multiple types of wind turbine blade damage images to train the wind turbine blade damage detection model. Through the damage feature enhanced feature extraction network, the wind turbine blade damage detection model will focus on the damage features, and then generate a damage bounding box through a multi-attribute perception loss function. Finally, the damage type and damage location of the blade are obtained.
[0011] In the above-mentioned wind turbine blade surface damage detection method based on multi-attribute perception attention, in the step 1), the established wind turbine blade damage image database contains three types of damage images, namely, sand holes, skin detachment, and cracks, and undamaged blade images, wherein the damage images are expanded in number and the changes in damage under different environments are simulated through morphological transformation and image adjustment. The process of morphological transformation and image adjustment includes: inversion, rotation, brightness adjustment, color adjustment, contrast adjustment, and sharpness adjustment; the purpose of inverting and rotating the image is to simulate the acquisition of damage image information at different angles; by adjusting the brightness and chromaticity of the image, the characteristics of the damaged image under different lighting conditions are obtained; the adjustment of contrast and sharpness is used to enhance the damage characteristics of the image and highlight the foreground information.
[0012] In the above-mentioned wind turbine blade surface damage detection method based on multi-attribute perception attention, the step 2) specifically includes:
[0013] Step 2-1): Establish a convolutional neural network feature extraction network backbone;
[0014] Step 2-2): Establish a channel attention module;
[0015] Step 2-3): Establish a spatial attention module;
[0016] Step 2-4): Insert the channel attention module and the spatial attention module sequentially after the feature extraction block of the feature extraction network backbone.
[0017] In the above-mentioned wind turbine blade surface damage detection method based on multi-attribute perception attention, in the step 2-2), the channel attention module adjusts the channel weights of the feature map extracted by the feature extraction block, and assigns higher weights to channels containing more damage features, thereby achieving the purpose of enhancing damage features; specifically, the two-dimensional space of the feature map is compressed, and the feature map size is compressed from (C, H, W) to (C, 1, 1), where C represents Channel, which means the channel, H represents Height, which means the height, and W represents Width, which means the width, thereby achieving a summary of the channel-containing features; the specific compression methods used are average pooling and maximum pooling, and average pooling is used to compress the feature map. Average pooling takes the average of the features in each channel to retain its overall features; maximum pooling retains the features with the largest feature value in each channel, discards the others, and retains its significant features. These two feature compression methods are processed in parallel, and the size of the original features (C, H, W) is compressed to (C, 1, 1) respectively; then, the two compressed features are sent to the shared multi-layer perceptron respectively to establish the connection between the channels, so as to obtain new channel weights; finally, the two new channel weights are added and mapped to (0, 1) through the sigmoid function, and then the mapped weights are weighted to the original feature map to achieve channel enhancement of the damage features; the whole process is expressed as follows:
[0018] M c (F)=σ(MLP(AvgPool(F))+MLP(MaxPool(F)))
[0019] Where: F represents the feature map, AvgPool and MaxPool represent average pooling and maximum pooling respectively, MLP represents multi-layer perceptron, and σ represents Sigmoid function.
[0020] In the above-mentioned wind turbine blade surface damage detection method based on multi-attribute perception attention, in the step 2-3), the spatial attention module perceives the spatial information in the feature map, increases the weight of the spatial position that may contain the damage feature, and thus realizes the spatial enhancement of the damage feature; specifically, the channel information of the feature map is compressed, and the feature map size is compressed from (C, H, W) to (1, H, W), so as to realize the generalization of the channel inclusion feature; the specific compression method used is the average operation and the maximum operation, and the average value and the maximum value of the channel features at the same position are retained; then, the two spatial weights are stacked into (2, H, W), and then the convolution operation is used to compress and establish spatial connections to form a spatial weight of (1, H, W); finally, the obtained spatial weight is mapped to (0, 1) through the Sigmoid function, and the original feature map is weighted, so as to realize the spatial enhancement of the damage feature; the whole process is expressed by the following formula:
[0021] Mc (F)=σ(Conv(Avg(F))+Conv(Max(F)))
[0022] Where: F represents the feature map, Avg and Max represent the average operation and the maximum operation respectively, Conv represents the convolution operation, and σ represents the Sigmoid function.
[0023] In the above-mentioned wind turbine blade surface damage detection method based on multi-attribute perception attention, in step 2-4), the feature extraction trunk is divided into several feature extraction blocks according to the output size, and the attention modules in steps 2-2) and 2-3) are embedded in turn after each feature extraction block, and finally an enhanced feature extraction network is formed.
[0024] In the above-mentioned wind turbine blade surface damage detection method based on multi-attribute perception attention, in the step 3), the predicted frame and the true frame are connected by establishing a multi-attribute perception loss function of the damage boundary frame, and the difference between the predicted frame and the true frame is measured by measuring the intersection and union ratio, center point distance, aspect ratio and side length difference between the predicted frame and the true frame, so as to guide the predicted frame to approach the true frame; its loss function is defined as follows:
[0025]
[0026] Among them, IoU represents the ratio of the intersection and union of the predicted box and the real box, ρ 2 (b,b gt ) represents the predicted box b and the real box b gt The square of the center point distance, c is the diagonal square of the minimum circumscribed matrix of the predicted box and the true box, Represents the ratio of the width to the length of the real box, Represents the ratio of the width to the length of the prediction box, Represents the ratio of the difference between the length of the predicted box and the true box to the length of the minimum circumscribed matrix, It represents the ratio of the difference between the width of the predicted box and the true box to the width of the minimum circumscribed matrix.
[0027] In the above-mentioned wind turbine blade surface damage detection method based on multi-attribute perception and attention, in the step 4), the constructed wind turbine blade damage detection model includes a feature extraction layer, an attention module, a feature fusion layer and a damage bounding box generation layer; first, several convolutional neural network layers are stacked, and then the attention module is embedded in it after a specific layer according to steps 2-4), so as to extract the features contained in the input image layer by layer; among the extracted image features, according to the network depth, the image features are output at 1 / 3 depth, 2 / 3 depth and deepest depth, and the different depth features are fused using sampling, splicing and convolution methods, and then a feature tensor is output; in the generation of the damage bounding box, the feature tensor output in the previous stage is decoded using a convolutional neural network, and the decoded information is used to generate a preliminary prediction box according to the preset anchor box, and then the multi-attribute perception loss function is used to make the prediction box converge to the target box, and finally the detected damage boundary box and type are output.
[0028] The beneficial effects of the present invention are:
[0029] 1. The present invention proposes a wind turbine blade surface damage detection method based on multi-attribute perception attention. Compared with other methods, it can have a stronger detection capability for blade damage in a real wind farm with a complex environmental background, and has excellent performance in calibrating the blade damage boundary box. It is particularly suitable for wind turbine blade damage detection tasks with complex backgrounds and relatively mild damage.
[0030] 2. According to the damage characteristics of wind turbine blades, the present invention proposes a design scheme of a channel space attention module integrated into the feature extraction network. This design can improve the sensitivity of the detection network to the damage characteristics of wind turbine blades, and can distinguish the damage characteristics from the background characteristics, thereby achieving the purpose of enhancement.
[0031] 3. The present invention designs a multi-attribute perception loss function for bounding box generation, which focuses the damage detection model's attention on multiple attributes of the bounding box, jointly measures the difference between the predicted box and the true box, and guides the predicted box to converge to the true box, thereby generating a more accurate bounding box. BRIEF DESCRIPTION OF THE DRAWINGS
[0032] Figure 1 It is the overall flow chart of the present invention.
[0033] Figure 2 Schematic diagram of the feature extraction network enhanced by channel spatial attention.
[0034] Figure 3 Schematic diagram of multi-attribute perception loss function.
[0035] Figure 4 Generate a schematic diagram for the wind turbine blade damage bounding box.
[0036] Figure 5 This is the structural diagram of the wind turbine blade damage detection model.
[0037] Figure 6 Comparison of the accuracy of the attention-enhanced feature extraction network and the unenhanced network.
[0038] Figure 7 It is the actual damage detection result diagram and visualization diagram of the present invention.
[0039] Figure 8 This is a comparison chart of the multi-attribute perception loss function. DETAILED DESCRIPTION
[0040] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.
[0041] like Figure 1 As shown, a wind turbine blade surface damage detection method based on multi-attribute perception attention includes the following steps:
[0042] 1) Establish a database of multiple types of wind turbine blade damage images: collect different types of wind turbine blade damage image data, and use image processing methods to expand the quantity and enhance the quality of wind turbine blade images.
[0043] In step 1), the established wind turbine blade damage image database includes three types of damage images, namely, sand holes, skin shedding, and cracks, and images of undamaged blades, wherein the damage images are expanded in number and the changes in damage under different environments are simulated through morphological transformation and image adjustment. The process of morphological transformation and image adjustment includes: inversion, rotation, brightness adjustment, color adjustment, contrast adjustment, and sharpness adjustment; the purpose of inverting and rotating images is to simulate the acquisition of damage image information at different angles; by adjusting the brightness and chromaticity of the image, the characteristics of the damaged image under different lighting conditions are obtained; the adjustment of contrast and sharpness is used to enhance the damage characteristics of the image and highlight the foreground information.
[0044] 2) Establish a channel-spatial attention enhanced feature extraction backbone network: By sequentially embedding channel attention and spatial attention modules in the feature extraction network, a damage feature enhanced feature extraction network is established.
[0045] like Figure 1 As shown in FIG. 1 , the channel-spatial attention enhanced feature extraction network performs weighted processing on the input feature map through the channel attention module and the spatial attention module in sequence, which specifically includes the following steps:
[0046] Step 2-1): Establish a convolutional neural network feature extraction network backbone.
[0047] Step 2-2): Establish a channel attention module.
[0048] In step 2-2), the channel attention module adjusts the channel weights of the feature map extracted by the feature extraction block, and assigns higher weights to channels containing more damage features, thereby achieving the purpose of enhancing damage features; specifically, the two-dimensional space of the feature map is compressed, and the feature map size is compressed from (C, H, W) to (C, 1, 1), where C represents Channel, H represents Height, and W represents Width, thereby achieving a summary of the features contained in the channel; the specific compression methods used are average pooling and maximum pooling. Average pooling averages the features in each channel to retain its overall features. , that is, the feature (C, H, W) is compressed to (C, 1, 1); the maximum pooling retains the feature with the largest feature value in each channel, discards the others, and retains its significant features. These two feature compression methods are processed in parallel, and the size of the original feature (C, H, W) is compressed to (C, 1, 1) respectively; then, the two compressed features are sent to the shared multi-layer perceptron respectively to establish the connection between the channels, so as to obtain new channel weights; finally, the two new channel weights are added and mapped to (0, 1) through the sigmoid function, and then the mapped weights are weighted to the original feature map to achieve channel enhancement of the damage feature; the whole process is expressed as follows:
[0049] M c (F)=σ(MLP(AvgPool(F))+MLP(MaxPool(F)))
[0050] Where: F represents the feature map, AvgPool and MaxPool represent average pooling and maximum pooling respectively, MLP represents multi-layer perceptron, and σ represents Sigmoid function.
[0051] Step 2-3): Establish a spatial attention module.
[0052] In step 2-3), the spatial attention module perceives the spatial information in the feature map and increases the weight of the spatial position that may contain the damage feature, thereby realizing the spatial enhancement of the damage feature; specifically, the channel information of the feature map is compressed, and the feature map size is compressed from (C, H, W) to (1, H, W), thereby realizing the generalization of the channel-containing features; the specific compression method used is the average operation and the maximum operation, and the average and maximum values of the channel features at the same position are retained; then, the two spatial weights are stacked into (2, H, W), and then the convolution operation is used to compress and establish spatial connections to form a spatial weight of (1, H, W); finally, the obtained spatial weight is mapped to (0, 1) through the Sigmoid function, and the original feature map is weighted to realize the spatial enhancement of the damage feature; the whole process is expressed as follows:
[0053] M c (F)=σ(Conv(Avg(F))+Conv(Max(F)))
[0054] Where: F represents the feature map, Avg and Max represent the average operation and the maximum operation respectively, Conv represents the convolution operation, and σ represents the Sigmoid function.
[0055] Step 2-4): Insert the channel attention module and the spatial attention module sequentially after the feature extraction block of the feature extraction network backbone.
[0056] In step 2-4), the feature extraction backbone is divided into several feature extraction blocks according to the output size, and the attention modules in steps 2-2) and 2-3) are embedded in turn after each feature extraction block to finally form an enhanced feature extraction network.
[0057] 3) Establish a multi-attribute perception loss function for the damaged bounding box: In view of the complex information of the bounding box, a multi-attribute loss function for the bounding box is established. By perceiving multiple attributes of the bounding box, the difference between the predicted box and the true box is described, thereby guiding the model to generate an accurate predicted box.
[0058] In step 3), the predicted box and the true box are connected by establishing a multi-attribute perception loss function of the damaged bounding box. The difference between the predicted box and the true box is perceived by multi-attributes collaboratively by measuring the intersection-over-union ratio, center point distance, aspect ratio, and side length difference between the predicted box and the true box, thereby guiding the predicted box to approach the true box. Figure 3 As shown in the figure, a schematic diagram of the multi-attribute perception loss function and the actual meaning of each term in the expression are presented. The loss function is defined as follows:
[0059]
[0060] Among them, IoU represents the ratio of the intersection and union of the predicted box and the real box, ρ 2 (b,b gt ) represents the predicted box b and the real box b gt The square of the center point distance, c is the diagonal square of the minimum circumscribed matrix of the predicted box and the true box, Represents the ratio of the width to the length of the real box, Represents the ratio of the width to the length of the prediction box, Represents the ratio of the difference between the length of the predicted box and the true box to the length of the minimum circumscribed matrix, It represents the ratio of the difference between the width of the predicted box and the true box to the width of the minimum circumscribed matrix.
[0061] like Figure 4The figure shows a schematic diagram of wind turbine blade damage bounding box generation. The bounding box generation part of the damage detection model generates a candidate box based on the features extracted by the feature extraction network. The multi-attribute perception loss function adjusts the candidate box parameters according to the difference between the target box and the candidate box, so that its position and side length are close to the target box, so that an accurate prediction box can be generated.
[0062] 4) Constructing a wind turbine blade damage detection model
[0063] like Figure 5 As shown in the figure, the damage detection model constructs an attention-enhanced feature extraction network by embedding the attention module into the deep convolutional neural network, thereby extracting features from the input leaf image. In the figure, Focus is an operation of sampling and slicing the image, Conv represents a standard convolutional layer and a standardized operation, C3 represents three convolutional layers stacked, SPP represents a spatial pyramid pooling method, and Attention represents the attention module described in step 2); the shallow features, middle features, and deep features extracted from the enhanced feature extraction network are fused in the feature fusion part. During the fusion process, splicing, convolution, sampling and other methods are used to fuse features of different depths to generate a feature tensor. Concat in the figure represents splicing, UpSample represents upsampling, and DownSample represents downsampling; the generated feature tensor first generates a prediction box according to the features and the preset box during the bounding box generation process, and the prediction box is then used according to the multi-attribute loss function described in step 3) to converge to the target box, thereby obtaining a damage bounding box.
[0064] Finally, in order to verify its effectiveness, the present invention is demonstrated by using rigorous experiments. The experimental platform is as follows: Intel i5-10600KF CPU, NVIDIA GeForce RTX2060 8G GPU, 16G memory, and Pytorch1.8.1 and Cuda 10.2.89 are used as the software environment.
[0065] In order to verify the effectiveness of the proposed channel space enhanced feature extraction network, the present invention uses an unenhanced network model and an enhanced network model for comparison, and all experiments are tested in the same environment. Its measurement index is mAP@0.5. Before introducing the calculation of mAP, there are several concepts that need to be understood. IoU (Intersection over Union) indicates the degree of overlap between the predicted bounding box "A" and the target bounding box "B". The larger its value, the closer the predicted box is to the target box. The calculation formula of IoU is as follows.
[0066]
[0067] In general, when IoU is greater than a threshold, the result is considered True. The present invention sets this threshold to 0.5. In other words, when the model believes that there is a damaged target at a location, it will generate a bounding box "A", and if the overlapping area ratio of the real box "B" and "A" exceeds 0.5, it is considered to be TP (True Positives). The definitions of FP (False Positives) and FN (False Negatives) are similar. In this way, the two indicators of Precision and recall can be defined. The specific calculation method is shown in the following formula.
[0068]
[0069]
[0070] The AP (Average Precision) of a category of targets can be determined by setting a two-dimensional coordinate graph with Recall as the horizontal axis and Precision as the vertical axis, and plotting the PR curve and the coordinate axes to obtain the area of the closed block. mAP is the average AP of all categories that the model can detect.
[0071] like Figure 6 As shown in Figure 3, the network with attention enhancement has a significant improvement over the network without attention enhancement.
[0072] Figure 7 The detection results of the attention enhancement network in the present invention and its visualization images are shown. Figure 7 The three rows of images in the figure are the original image, the output detection results, and the feature visualization of the last layer in the backbone. From the detection results, we can see that the network using attention enhancement not only detects the slight cracks contained in the image, but also gives a higher confidence. In the visualization, the depth of the color represents the weight of the network, that is, the degree of focus. Figure 7 It can be seen that the attention-enhanced network gives a higher weight to the minor cracks below, thereby detecting minor damage. This means that the attention-enhanced feature extraction network of the present invention can guide the network to pay attention to unimportant damage, thereby improving the detection rate of the network.
[0073] In order to verify the effectiveness of the multi-attribute-aware damage function in this invention, this experiment uses the most advanced loss function to conduct experimental comparisons in the wind blade dataset. In this part, all methods use the same network structure and parameters, the only difference is the loss function for bounding box generation.
[0074] Figure 8The detection results of some damaged wind turbine blade images are given. In the detection results of blade images with sand holes, the method using the multi-attribute-aware damage function loss function has the highest confidence. In addition, its bounding box does not contain too much background and almost completely overlaps with the true box. The methods using other loss functions include part of the healthy area in the bounding box. In the crack damage detection results, the bounding box generated by the multi-attribute-aware damage function method significantly reduces the difference with the true box and obtains the highest confidence. The detection results of skin detachment are similar to those of the other two types of damage, among which the method using the multi-attribute-aware damage function achieves the best results. These experimental results confirm the effectiveness of the multi-attribute-aware damage function.
[0075] In summary, the present invention aims at the problem of poor matching between the damage bounding box and the real box in blade damage detection, starting from the two aspects of damage feature extraction and bounding box difference measurement, and proposes a wind turbine blade surface damage detection algorithm with multi-attribute perception and attention. This method embeds the channel space attention module into the backbone of the feature extraction network, enhances the channel and space of the damage features, and improves the feature extraction ability of the backbone network; then a multi-attribute perception loss function is designed for bounding box generation. This loss function places the attention of the bounding box generation network on multiple attributes of the bounding box at the same time, and more comprehensively perceives the direct differences of the bounding box, thereby generating a damage bounding box with a higher matching degree with the real box. Compared with the prior art, the present invention has greatly improved the existing damage detection method and can be widely applied to real wind farms with complex backgrounds.
Claims
1. A wind turbine blade surface damage detection method based on multi-attribute perception attention, characterized in that: The following steps are involved: 1) Establish a database of multiple types of wind turbine blade damage images: collect different types of wind turbine blade damage image data, and use image processing methods to expand the number and enhance the quality of wind turbine blade images; 2) Establish a feature extraction backbone network with channel-spatial attention enhancement; The step 2) specifically includes: Step 2-1): Establish a convolutional neural network feature extraction network backbone; Step 2-2): Establish a channel attention module; Step 2-3): Establish a spatial attention module; Step 2-4): insert the channel attention module and the spatial attention module into the feature extraction block of the feature extraction network backbone in sequence; 3) Establishing a multi-attribute perception loss function for the damage bounding box: In view of the complex information of the bounding box, a multi-attribute loss function for the bounding box is established. By perceiving multiple attributes of the bounding box, the difference between the predicted box and the true box is described, thereby guiding the model to generate an accurate predicted box. In the step 3), the predicted frame and the true frame are connected by establishing a multi-attribute perception loss function of the damaged bounding box. The difference between the predicted frame and the true frame is perceived by multi-attributes collaboratively by measuring the intersection-over-union ratio, center point distance, aspect ratio, and side length difference between the predicted frame and the true frame, thereby guiding the predicted frame to approach the true frame. The loss function is defined as follows: Among them, IoU represents the ratio of the intersection and union of the predicted box and the real box, ρ 2 (b,b gt ) represents the predicted box b and the real box b gt The square of the center point distance, c is the diagonal square of the minimum circumscribed matrix of the predicted box and the real box, Represents the ratio of the width to the length of the real box, Represents the ratio of the width to the length of the prediction box, Represents the ratio of the difference between the length of the predicted box and the true box to the length of the minimum circumscribed matrix, Represents the ratio of the difference between the width of the predicted box and the true box to the width of the minimum circumscribed matrix; 4) Constructing a wind turbine blade damage detection model: Constructing a wind turbine blade damage detection model, using a multi-category wind turbine blade damage image database to train the wind turbine blade damage detection model, using a damage feature enhanced feature extraction network to direct the wind turbine blade damage detection model's attention to the damage features, and then generating a damage bounding box through a multi-attribute perception loss function, and finally obtaining the blade damage type and damage location; The constructed wind turbine blade damage detection model includes a feature extraction layer, an attention module, a feature fusion layer and a damage bounding box generation layer.
2. The wind turbine blade surface damage detection method based on multi-attribute perception attention according to claim 1 is characterized by: In the step 1), the established wind turbine blade damage image database includes three types of damage images, namely, sand holes, skin shedding, and cracks, and images of undamaged blades, wherein the damage images are expanded in number and the changes in damage under different environments are simulated through morphological transformation and image adjustment. The process of morphological transformation and image adjustment includes: inversion, rotation, brightness adjustment, color adjustment, contrast adjustment, and sharpness adjustment; the purpose of inverting and rotating images is to simulate the acquisition of damage image information at different angles; by adjusting the brightness and chromaticity of the image, the characteristics of the damaged image under different lighting conditions are obtained; the adjustment of contrast and sharpness is used to enhance the damage characteristics of the image and highlight the foreground information.
3. The wind turbine blade surface damage detection method based on multi-attribute perception and attention according to claim 1 is characterized by: In the step 2-2), the channel attention module adjusts the channel weights of the feature map extracted by the feature extraction block, and assigns higher weights to channels containing more damage features, thereby achieving the purpose of enhancing damage features; specifically, the two-dimensional space of the feature map is compressed, and the feature map size is compressed from (C, H, W) to (C, 1, 1), where C represents Channel, H represents Height, and W represents Width, thereby achieving a summary of the features contained in the channel; the specific compression methods used are average pooling and maximum pooling, and average pooling takes the features in each channel as the maximum pooling. The average operation retains its overall characteristics; the maximum pooling retains the features with the largest eigenvalues in each channel, discards the others, and retains its significant features. These two feature compression methods are processed in parallel, and the size of the original features (C, H, W) is compressed to (C, 1, 1) respectively; then, the two compressed features are sent to the shared multi-layer perceptron respectively to establish the connection between the channels, so as to obtain new channel weights; finally, the two new channel weights are added and mapped to (0, 1) through the sigmoid function, and then the mapped weights are weighted to the original feature map to achieve channel enhancement of the damage features; the whole process is expressed as follows: M c (F)=σ(MLP(AvgPool(F))+MLP(MaxPool(F))) Where: F represents the feature map, AvgPool and MaxPool represent average pooling and maximum pooling respectively, MLP represents multi-layer perceptron, and σ represents Sigmoid function.
4. The wind turbine blade surface damage detection method based on multi-attribute perception and attention according to claim 1 is characterized by: In the step 2-3), the spatial attention module increases the weight of the spatial position that may contain the damage feature by perceiving the spatial information in the feature map, thereby realizing spatial enhancement of the damage feature; specifically, the channel information of the feature map is compressed, and the feature map size is compressed from (C, H, W) to (1, H, W), thereby realizing the generalization of the channel-containing features; the specific compression method used is the average operation and the maximum operation, and the average value and the maximum value of the channel features at the same position are retained; Subsequently, the two spatial weights are stacked into (2, H, W), and then the convolution operation is used to compress and establish spatial connections to form a spatial weight of (1, H, W); finally, the obtained spatial weight is mapped to (0, 1) through the Sigmoid function, and the original feature map is weighted to achieve spatial enhancement of damage features; The whole process is expressed as follows: M c (F)=σ(Conv(Avg(F))+Conv(Max(F))) Where: F represents the feature map, Avg and Max represent the average operation and the maximum operation respectively, Conv represents the convolution operation, and σ represents the Sigmoid function.
5. The wind turbine blade surface damage detection method with multi-attribute perception and attention according to claim 1 is characterized by: In the step 2-4), the feature extraction backbone is divided into several feature extraction blocks according to the output size, and the attention modules in steps 2-2) and 2-3) are embedded in turn after each feature extraction block, and finally an enhanced feature extraction network is formed.
6. The wind turbine blade surface damage detection method based on multi-attribute perception and attention according to claim 1 is characterized by: In the step 4), several convolutional neural network layers are first stacked, and then the attention module is embedded in the specific layer according to steps 2-4), so as to extract the features contained in the input image layer by layer; among the extracted image features, according to the network depth, the image features are output at 1 / 3 depth, 2 / 3 depth and deepest depth, and the different depth features are fused using sampling, splicing and convolution methods, and then a feature tensor is output; in the generation of the damage bounding box, the feature tensor output in the previous stage is decoded using a convolutional neural network, and the decoded information is used to generate a preliminary prediction box according to the preset anchor box, and then the multi-attribute perception loss function is used to make the prediction box converge to the target box, and finally the detected damage bounding box and type are output.