A wind turbine blade clearance detection method based on lightweight segmentation network

Abstract

The present invention relates to the technical field of wind turbine blade safety monitoring, specifically a wind turbine blade clearance detection method based on a lightweight segmentation network, comprising: S1, collecting multiple pictures containing wind turbine blades and constructing a training set; S2, constructing an improved target detection network; S3, selecting a wind turbine blade picture from the training set and inputting it into the improved target detection network to obtain a detection result; S4, calculating the loss between the detection result and the true result, looping from S3 to S4 until the total loss function converges, and updating the weight of the improved target detection network; S5, testing the trained target detection network to obtain a test result, and calculating the clearance value through the test result. The present invention incorporates the CBAM attention mechanism into the third and fourth stages of the backbone network, which effectively enhances the backbone network's ability to perceive features.

Description

Technical Field

[0001] The present invention relates to the technical field of wind turbine blade safety monitoring, and in particular to a wind turbine blade clearance detection method based on a lightweight segmentation network. Background Art

[0002] As time goes by, the performance and life of the unit are decreasing year by year, resulting in frequent accidents such as wind turbine tower sweeping, tower collapse, and blade breakage. Therefore, researchers have proposed that blade clearance be used to measure whether the wind turbine is operating safely. Blade clearance refers to the vertical geometric distance between the wind turbine blade and the tower surface during operation, and the minimum clearance is the vertical distance between the blade and the tower surface when the blade runs to the lowest point. Changes in the wind turbine clearance distance may lead to a smaller wind sweeping area, affecting the power generation efficiency, or even a tower sweeping accident, causing serious safety hazards. At present, solving the wind turbine clearance distance estimation and avoiding tower sweeping problems in complex and changing environments is one of the important directions for the development of wind power generation technology.

[0003] He et al. proposed a portable blade tip clearance measurement system based on a high-speed camera. This method calculates the blade tip clearance based on image data. However, due to the limited camera range, it can only capture the blade conditions in a fixed area. The acquired data cannot guarantee its integrity, and it is difficult to fully evaluate the clearance. However, the difficulty of wind turbine clearance calculation based on image processing is that it is difficult to accurately obtain the outline of the blade in complex environments such as light changes, heavy fog and rain, and night. In view of this, Zhang proposed a wind turbine tower and blade tip clearance measurement and control method based on millimeter-wave radar sensors. Through eight millimeter-wave radar sensors installed on the surface of the tower, the change in the distance between the tower and the blade during operation is continuously measured, and the minimum clearance distance between the blade and the tower is estimated. It can adapt to bad weather and poor light conditions, but the cost is expensive.

[0004] As the installation of wind turbines is gradually developing towards offshore and mountainous areas, the outdoor environment is complex and changeable. The image data analysis of wind turbine blades is greatly affected by heavy rain, fog and snow. Domestic and foreign researchers have used the advantages of deep learning's robustness and strong anti-interference ability to overcome the disadvantages of image clearance detection that is easily affected by special environments. Wu et al. proposed a recognition method for automatic recognition of coded marks on wind turbine rotor blades based on an improved YOLOv8 model. By introducing spatial and channel reconstruction convolution (Sc Conv) and Wise-Io U (WIo U) loss functions, and developing a new model YOLOv8-ScConv-WIoU model, this method improves the accuracy and robustness of coded mark recognition, but it cannot estimate the wind turbine clearance distance in real time.

[0005] Generally speaking, as long as the wind speed reaches 3 meters per second, the blade tip speed can reach more than 280 kilometers per hour. Therefore, the speed of the detection network needs to be further improved, but at the same time, due to the limited computing power of wind turbine equipment, the commonly used detection algorithms are difficult to play the advantages of high precision in wind turbine equipment. Summary of the invention

[0006] The present invention provides a fan blade clearance detection method based on a lightweight segmentation network to solve the technical problems mentioned in the background technology.

[0007] To achieve the above object, the technical solution of the present invention is achieved as follows:

[0008] The present invention provides a fan blade clearance detection method based on a lightweight segmentation network, comprising the following steps:

[0009] S1, collect multiple wind turbine blade images and build a training set;

[0010] S2. On the basis of the original YOLOv8 network model, an improved YOLOv8-ENNet target detection network is constructed. The improved YOLOv8-ENNet target detection network includes a backbone network connected in sequence, an improved bottleneck layer Neck embedded with an ECA module, and a prediction layer Head. The backbone network uses the EfficientNetV2 backbone network that introduces the CBAM attention mechanism.

[0011] S3, select a wind turbine blade image from the training set and input it into the improved YOLOv8-ENNet target detection network, and finally obtain an image with a bounding box and the target category and confidence level marked around the bounding box; that is, the detection result;

[0012] S4, construct a total loss function, calculate the loss between the detection result and the true result, loop S3 to S4 until the total loss function converges, and use the back propagation algorithm to update the weights of the improved YOLOv8-ENNet target detection network to obtain the trained YOLOv8-ENNet target detection network;

[0013] S5. Test the trained YOLOv8-ENNet target detection network to obtain test results, and calculate the clearance value through the test results.

[0014] Furthermore, the S1 specifically includes the following steps:

[0015] S11. Install a camera on the wind turbine tower, with the camera being mounted between the blade and the tower and facing the tip of the blade;

[0016] S12. Use a camera to shoot blades, blade tips and towers to obtain a video containing towers, blades and blade tips, then extract frames and clean the video to obtain multiple wind turbine blade images, where the wind turbine blade images contain towers, blade tips and / or blades, and then use the multiple wind turbine blade images to construct a training set.

[0017] Furthermore, the EfficientNetV2 backbone network in S2 contains 8 stages in total; the 8 stages are connected in sequence;

[0018] Among them, the 0th stage is a convolution layer; compared with the EfficientNetV1 network, the first to third stages replace the original MBConv module with the Fused-MBConv module to reduce the computational complexity; the fourth to sixth stages are two connected MBConv modules; the seventh stage is a sequentially connected convolution layer, Pooling&FC module and spatial pyramid pooling fast module SPPF; the third and fourth stages are embedded with the CBAM attention mechanism to reduce the impact of redundant features on subsequent convolution operations; the MBConv modules in the fourth to sixth stages are embedded with the SE module; thereby enhancing the ability of feature representation;

[0019] The ECA module is embedded in the cross-stage feature fusion module C2f in the bottleneck layer Neck, and is embedded in multiple Bottleneck modules in the cross-stage feature fusion module C2f;

[0020] The prediction layer Head includes three detection heads, large, medium and small, to perform multi-scale detection.

[0021] Furthermore, the ECA module weights the feature map in the channel dimension through global average pooling, convolutional layers and activation functions to enhance important channel features.

[0022] Furthermore, the working process in the ECA module is as follows:

[0023] Step a1: First, the global average pooling in the ECA module will be input into the feature map of the ECA module Convert to a one-dimensional vector , capture the global spatial information and obtain the global feature representation of each channel;

[0024] Step a2: Input the global feature representation of each channel into the convolutional layer. The convolutional layer uses a one-dimensional convolution kernel channel descriptor of size k to perform 1D convolution operation to capture important feature information through local cross-channel interaction.

[0025] Step a3: Pass the output of the convolutional layer through the Sigmoid activation function. The output value is compressed to between 0 and 1, and the weight of each channel is generated. The weight of each channel can appropriately adjust the feature map input to the ECA module. Information in

[0026] Step a4: Multiply each channel of the original feature map by the corresponding channel attention weight to generate an enhanced feature map, i.e., a reweighted feature map.

[0027] Furthermore, the one-dimensional vector in step a1 The details are as follows:

[0028] ;

[0029] Among them, C is the total number of channels, H is the height of the feature map, and W is the width of the feature map; is the feature map input to the ECA module No. channels; Indicates the current channel number; They refer to the feature maps The height and width of

[0030] The calculation formula of the 1D convolution operation in step a2 is:

[0031] ;

[0032] Among them, k is the convolution kernel size;

[0033] The calculation formula of the Sigmoid function in step a3 is:

[0034] ;

[0035] in, It is The attention weight of each channel; Represents the Sigmoid function symbol;

[0036] The calculation formula of the re-weighted feature map is:

[0037] ;

[0038] in, is the reweighted channels.

[0039] Furthermore, the S3 specifically includes the following steps:

[0040] S31. Select a fan blade image from the training set and input it into the improved YOLOv8-ENNet target detection network. The EfficientNetV2 backbone network extracts features from the fan blade image and outputs several feature maps of different scales.

[0041] S32, inputting several feature maps of different scales into the improved bottleneck layer Neck and performing feature fusion and optimization, and outputting fused feature maps having the same number as the feature maps of different scales;

[0042] S33. Input the fused feature maps into the prediction layer Head. The prediction layer Head maps the fused feature maps into a preset target detection frame to obtain a picture with a bounding box and the target category and confidence level marked around the bounding box, that is, the detection result.

[0043] Furthermore, the total loss function in S4 is formed by adding the position loss function, the confidence loss function and the classification loss function according to the set coefficients;

[0044] The position loss function is used to calculate the position loss between the bounding box obtained in S3 and the true bounding box; the confidence loss function is used to calculate the loss between the confidence obtained in S3 and the true confidence; the classification loss function is used to calculate the loss between the target category obtained in S3 and the true category.

[0045] Furthermore, the S5 specifically includes the following steps:

[0046] S51, select a wind turbine blade image and input it into the trained YOLOv8-ENNet target detection network to obtain the test result;

[0047] S52, measuring the tower edge and the blade tip position according to the test result to obtain the pixel straight-line distance between the two;

[0048] S53, then using the tower edge and the blade tip position to obtain the pixel distance between the two, and performing detection and calculation by a fan blade tip and fan blade fusion detection method or a blade tip segmentation detection method to obtain a final clearance value; wherein the blade tip segmentation detection method calculates the clearance value by the following formula, and the specific formula is as follows:

[0049] d c =nd 2；

[0050] Among them, d c represents the clearance value, n represents the pixel value between the blade tip and the tower; d2 represents the calibrated pixel distance.

[0051] Furthermore, the selection rules of the two detection methods of the fan blade tip and fan blade fusion detection method and the blade tip segmentation detection method are as follows:

[0052] When the blades in the fan blade image are not blocked, the fan blade tip and fan blade fusion detection method is used for detection and calculation; otherwise, the blade tip segmentation detection method is used for detection and calculation to ensure the stability of blade tip detection.

[0053] Beneficial effects of the present invention:

[0054] 1. The backbone network in the present invention is based on the EfficientNetV2 backbone network. The composite scaling method takes into account the increase in fixed computing resources, captures high-resolution images, and obtains features of more pixels. The CBAM attention mechanism is added to the EfficientNetV2 backbone network to fuse multi-scale features, and the ECA module is embedded in the Bottleneck module in the bottleneck layer Neck, which enhances the model's ability to extract multi-scale features.

[0055] 2. The present invention incorporates the CBAM attention mechanism into the third and fourth stages of the EfficientNetV2 backbone network, effectively enhancing the backbone network's ability to perceive features.

[0056] 3. The present invention improves the wind turbine clearance measurement method. According to different application scenarios, two different detection methods, namely, the wind turbine blade tip and wind turbine blade fusion detection method and the blade tip segmentation detection method, can be used to dynamically detect the target, thereby ensuring the stability of the blade tip detection and improving the robustness of the blade tip detection, thereby improving the accuracy of estimating the wind turbine clearance distance.

[0057] 4. The present invention can monitor the wind turbine clearance distance in real time. Once the wind turbine clearance distance exceeds the set threshold, timely intervention in the unit control can prevent the blades from hitting the tower and causing catastrophic failure, thereby ensuring the safe operation of the blades and the tower. And by intervening to control the swept wind area, it can be maintained at the maximum area and float around, which can improve the wind turbine power generation efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 It is a structural block diagram of the improved YOLOv8-ENNet target detection network in the present invention;

[0059] Figure 2 Schematic diagram of the structure of the CBAM attention mechanism in the present invention;

[0060] Figure 3 This is the principle diagram of clearance measurement based on image ranging;

[0061] Figure 4 It is a structural block diagram of the cross-stage feature fusion module C2f in the present invention. DETAILED DESCRIPTION

[0062] In order to facilitate the understanding of the present invention, the present invention will be described more fully below with reference to the relevant drawings. Preferred embodiments of the present invention are provided in the drawings. However, the present invention can be implemented in many other different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present invention more thorough and comprehensive.

[0063] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0064] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which the present invention belongs. The terms used herein in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more of the related listed items.

[0065] Reference Figure 1 The embodiment of the present application provides a wind turbine blade clearance detection method based on a lightweight segmentation network, comprising the following steps:

[0066] S1, collect multiple wind turbine blade images and build a training set;

[0067] S2. On the basis of the original YOLOv8 network model, an improved YOLOv8-ENNet target detection network is constructed. The improved YOLOv8-ENNet target detection network includes a backbone network, an improved bottleneck layer Neck embedded with an ECA module, and a prediction layer Head connected in sequence. The backbone network uses the EfficientNetV2 backbone network that introduces the CBAM attention mechanism. The EfficientNetV2 backbone network reduces the number of parameters while increasing the training speed, that is, the training is efficient while maintaining the parameter efficiency. Compared with the EfficientNet network, the expansion ratio of the MBConv module in the EfficientNetV2 backbone network structure is reduced, which reduces the amount of calculation and video memory usage.

[0068] S3, select a wind turbine blade image from the training set and input it into the improved YOLOv8-ENNet target detection network, and finally obtain an image with a bounding box and the target category and confidence level marked around the bounding box; that is, the detection result;

[0069] S4, construct a total loss function, calculate the loss between the detection result and the true result, loop S3 to S4 until the total loss function converges, and use the back propagation algorithm to update the weights of the improved YOLOv8-ENNet target detection network to obtain the trained YOLOv8-ENNet target detection network;

[0070] S5. Test the trained YOLOv8-ENNet target detection network to obtain test results, and calculate the clearance value through the test results.

[0071] In some embodiments, the S1 specifically includes the following steps:

[0072] S11. Install a camera on the wind turbine tower. The camera is installed between the blade and the tower and faces the tip of the blade. The camera is installed at the following position: Figure 3 As shown;

[0073] S12. Use a camera to shoot blades, blade tips and towers to obtain a video containing towers, blades and blade tips, then extract frames and clean the video to obtain multiple wind turbine blade images, where the wind turbine blade images contain towers, blade tips and / or blades, and then use the multiple wind turbine blade images to construct a training set.

[0074] In some embodiments, reference Figure 1 , the EfficientNetV2 backbone network in S2 contains 8 stages in total; the 8 stages are connected in sequence;

[0075] Among them, the 0th stage is a convolution layer; compared with the EfficientNetV1 network, the first to third stages replace the original MBConv module with the Fused-MBConv module, which fuses the standard convolution and the depth-separable convolution. This fusion reduces the computational complexity; the fourth to sixth stages are two connected MBConv modules; the seventh stage is a sequentially connected convolution layer, Pooling&FC module and spatial pyramid pooling fast module SPPF; among them, the third and fourth stages embed the CBAM attention mechanism to further enhance the attention of the YOLOv8-ENNet target detection network to key important features and global features, while reducing the influence of redundant features on subsequent convolution operations; the MBConv modules in the fourth to sixth stages are embedded with SE modules; by adaptively readjusting the channel weights of the feature map, the ability of feature representation is enhanced, and the effect is significant when processing complex image features;

[0076] The SE module generates the attention weight of each channel through global average pooling and fully connected layers to better capture global information;

[0077] The ECA module is embedded in the cross-stage feature fusion module C2f in the bottleneck layer Neck, and is embedded in multiple Bottleneck modules in the cross-stage feature fusion module C2f; this design improves the feature selection and feature fusion capabilities between channels, and enhances the detection accuracy and robustness of the YOLOv8-ENNet target detection network in complex environments. The ECA module is a lightweight and efficient channel attention mechanism that can significantly reduce the computational cost and parameter amount while maintaining the performance advantages of the SE module, and is an upgraded version of the SE module.

[0078] The prediction layer Head includes three detection heads, large, medium and small, to perform multi-scale detection.

[0079] In some embodiments, the working process of the cross-stage feature fusion module C2f is as follows:

[0080] First, the feature map input to the cross-stage feature fusion module C2f is convolved by 1×1 to change the number of channels of the feature map. Then, the feature map is segmented into multiple parts through feature segmentation, and multiple feature subsets are created accordingly. After entering the bottleneck layer, each segmented feature subset is processed through two consecutive convolution layers to extract features and increase the receptive field. After the convolution operation, it enters the ECA module, which weights the feature map in the channel dimension through global average pooling, convolution layer and activation function to enhance important channel features. The schematic diagram of the bottleneck layer embedded in the ECA module is shown in the attached figure. Figure 4 shown.

[0081] In some embodiments, the working process in the ECA module is as follows:

[0082] Step a1: First, the global average pooling in the ECA module will be input into the feature map of the ECA module Convert to a one-dimensional vector , capture the global spatial information and obtain the global feature representation of each channel; one-dimensional vector The details are as follows:

[0083] ;

[0084] Among them, C is the total number of channels, H is the height of the feature map, and W is the width of the feature map; is the feature map input to the ECA module No. channels; Indicates the current channel number; They refer to the feature maps The height and width of

[0085] Step a2: Input the global feature representation of each channel into the convolution layer. The convolution layer uses a one-dimensional convolution kernel channel descriptor of size k to perform 1D convolution operation, capturing important feature information through local cross-channel interaction. This step avoids the fully connected layer, reducing the number of parameters and computational cost. The calculation formula for the 1D convolution operation is:

[0086] ;

[0087] Among them, k is the convolution kernel size;

[0088] Step a3: Pass the output of the convolutional layer through the Sigmoid activation function. The output value is compressed to between 0 and 1, and the weight of each channel is generated. The weight of each channel can appropriately adjust the feature map input to the ECA module. The information in; the calculation formula of the Sigmoid function is:

[0089] ;

[0090] in, It is The attention weight of each channel; Represents the Sigmoid function symbol;

[0091] Step a4: multiply each channel of the original feature map by the corresponding channel attention weight to generate an enhanced feature map, i.e., a reweighted feature map; by adjusting the weight of each channel, the feature information of important channels is enhanced, the model representation capability is improved, and uncritical features are ignored. The calculation formula of the reweighted feature map is:

[0092] ;

[0093] in, is the reweighted channels.

[0094] In some embodiments, S3 specifically includes the following steps:

[0095] S31. Select a fan blade image from the training set and input it into the improved YOLOv8-ENNet target detection network. The EfficientNetV2 backbone network extracts features from the fan blade image and outputs several feature maps of different scales.

[0096] Specifically, a fan blade image is selected from the training set and input into the improved YOLOv8-ENNet target detection network. The fan blade image is extracted from the 0th to 2nd stage of the EfficientNetV2 backbone network to obtain the feature map. ; Since the third and fourth stages in the EfficientNetV2 backbone network are responsible for extracting deeper and more abstract high-dimensional features. In the third stage of the present invention, the Fused-MBConv module processes the feature map input to this stage. Apply a normal 3×3 convolution to get the feature map , extracts local spatial features through standard convolution operations, while considering the interaction between input channels, which can not only extract local information, but also expand the receptive field, thereby sensitively capturing the spatial structure information in the feature map. With the support of the CBAM attention mechanism, the feature map can be extracted. When extracting feature information, the importance of different channels and different spatial positions is fully considered, thereby improving the effect of feature extraction and reducing the impact of redundant features on subsequent convolution operations. The structure of the CBAM attention mechanism is as follows: Figure 2 shown.

[0097] Then, the feature map Then a layer of 1×1 convolution is performed to adjust the number of channels of the feature map while keeping the spatial resolution unchanged, fusing the feature information from different channels to generate a new, more compact feature map. , and finally through the skip connection, the input feature map Directly with the feature map after convolution Add to form the output feature map , and passed to the next network stage for further processing;

[0098] In the fourth stage, the feature map passed to the previous stage Perform depth-wise separable convolution, perform a 3×3 convolution operation on each channel separately, keep the receptive field size, and apply a 1×1 convolution to the feature map Channel expansion is performed to increase the number of channels of the feature map and obtain the feature map .

[0099] In the SE module, the feature map is pooled by global average pooling. Compressed into a global feature vector, then re-weighted by a pair of fully connected layers to re-weight the channel importance, and finally re-scaled the feature map by a channel-level product operation The CBAM attention mechanism combines channel attention and spatial attention, overcoming the defect of insufficient expression of spatial information caused by the SE module focusing on the relationship between feature channels and ignoring the importance of spatial position. By automatically learning and selecting important feature channels and spatial positions, the feature representation ability is improved, and more attention is paid to feature maps. The importance of different spatial positions in the image is enhanced, which enhances the robustness to noise and interference. After the SE module, the CBAM attention mechanism further introduces the spatial attention mechanism and the channel attention mechanism to further strengthen the features of important channels and focus on the feature map. The importance of different spatial positions in the feature map is further adjusted through 1x1 convolution The number of channels and feature representation ensures that after a series of attention mechanisms and convolution processing, the feature map With the required channel dimension and feature representation. Finally, through the jump connection, the input feature map Directly with the feature map after convolution Add them together to get the final output feature map of this stage , and passed to the next stage for more advanced feature extraction.

[0100] S32, inputting several feature maps of different scales into the improved bottleneck layer Neck and performing feature fusion and optimization, and outputting fused feature maps having the same number as the feature maps of different scales;

[0101] S33. Input the fused feature maps into the prediction layer Head. The prediction layer Head maps the fused feature maps into a preset target detection frame to obtain a picture with a bounding box and the target category and confidence level marked around the bounding box, that is, the detection result.

[0102] In some embodiments, the total loss function in S4 is formed by adding the position loss function, the confidence loss function and the classification loss function according to the set coefficients;

[0103] The position loss function is used to calculate the position loss between the bounding box obtained in S3 and the true bounding box; the confidence loss function is used to calculate the loss between the confidence obtained in S3 and the true confidence; the classification loss function is used to calculate the loss between the target category obtained in S3 and the true category.

[0104] In some embodiments, S5 specifically includes the following steps:

[0105] S51, select a wind turbine blade image and input it into the trained YOLOv8-ENNet target detection network to obtain the test result;

[0106] S52, measuring the tower edge and the blade tip position according to the test result to obtain the pixel distance between the two;

[0107] S53, then using the tower edge and the blade tip position to obtain the pixel distance between the two, and performing detection and calculation by a fan blade tip and fan blade fusion detection method or a blade tip segmentation detection method to obtain a final clearance value; wherein the blade tip segmentation detection method calculates the clearance value by the following formula, and the specific formula is as follows:

[0108] d c =nd 2；

[0109] Among them, d c represents the clearance value, n represents the pixel value between the blade tip and the tower; d2 represents the calibrated pixel distance.

[0110] In some embodiments, the selection rules of the two detection methods of the fan blade tip and fan blade fusion detection method and the blade tip segmentation detection method are:

[0111] When the blades in the fan blade image are not blocked, the fan blade tip and fan blade fusion detection method is used for detection and calculation; otherwise, the blade tip segmentation detection method is used for detection and calculation to ensure the stability of blade tip detection.

[0112] The following is a comparison and verification of the improved YOLOv8-ENNet target detection network provided by the present invention and other networks;

[0113] A real-time detection experiment was conducted on the wind turbine operation video. Through experimental comparison, the average confidence of the proposed model (i.e. the improved YOLOv8-ENNet target detection network) is higher than that of the original network model, and it has a better effect on processing tiny details in complex environments, and the missed detection and false detection are also better than the original model.

[0114] In order to better evaluate the detection performance of different experimental groups, the present invention will further compare the data under various indicators of the model and quantitatively analyze the detection effect of the experimental group, see experimental data comparison table 1.

[0115] Table 1 Experimental data comparison data:

[0116] ;

[0117] The experimental data show that the basic evaluation indexes of the improved YOLOv8-ENNet target detection network are better than the original YOLOv8 network model in average effect. Through quantitative data analysis, the improved YOLOv8-ENNet target detection network has a good balance between the length of calculation time and the quality of the effect. The Precision, Recall, F1, mAP, mAp@0.5, and GFLOPs of the improved YOLOv8-ENNet target detection network are 0.963, 0.963, 0.963, 0.974, 0.747, and 3.600, respectively. The Precision Precision is increased by 15.19%, the Recall Recall is slightly decreased by 2.01%, and the average precision mAp@0.5 is increased by 16.90%. Because the original network model has weak feature attention to tiny details such as leaf tips, the average precision mAp@0.5 of the leaf tip type is only 0.370. Compared with the same category indicators of the improved model, the percentage difference between the two is 38.%.

[0118] Experimental data verify that the introduction of the CBAM attention mechanism in the EfficientNetV2 backbone network and the embedding of the ECA module in the bottleneck layer can effectively enhance the perception of local information features and the attention to detail features, thereby improving the performance of wind turbine blade tip detection and segmentation. In summary, through experimental qualitative and quantitative analysis, the improved YOLOv8-ENNet target detection network in the present invention is better than the original YOLOv8 network model in processing small target details in complex and changeable outdoor environments.

[0119] To further enhance the model's attention to key features and global features, the CBAM attention mechanism is integrated into the third and fourth stages of the EfficientNetV2 backbone network. The ECA module is embedded in the C2f module in the bottleneck layer Neck to improve the feature fusion capability between channels. The following three experiments are used to verify the performance of the embedding module:

[0120] Experiment 1 uses the EfficientNetV2 backbone network without the introduction of the CBAM attention mechanism. Experiment 2 introduces the CBAM attention mechanism into the EfficientNetV2 backbone network based on Experiment 1. Experiment 3 embeds the ECA module in the bottleneck layer Neck based on Experiment 2.

[0121] Combined with the quantitative comparison and qualitative comparison analysis of the experiments in Table 2 below, the integration of the CBAM attention mechanism significantly improves the performance of the model, among which the precision Precision is improved by 3.45% compared with Experiment 1, and the average precision mAp@0.5 is improved by 9.89% compared with Experiment 1, but the qualitative analysis shows that the difference between the two is very small. In Experiment 3, by embedding the ECA module, the average precision mAp@0.5 calculated by the network model using the intersection-over-union ratio threshold of 0.5 reached 0.747, an increase of 3.46% compared with Experiment 2, and an increase of 13.69% compared with Experiment 1. Its harmonic mean F1 of precision and recall is also higher than Experiment 1 and Experiment 2, and its average confidence is significantly higher than Experiment 1 and Experiment 2.

[0122] Table 2 Quantitative analysis data of attention mechanism comparison;

[0123] ;

[0124] As can be seen from Table 2, the improved YOLOv8-ENNet target detection network has improved the average detection accuracy (mAp@0.5) by 38.96% compared with the original YOLOv8 network model, the number of model parameters has been reduced, and the FPS has also reached the standard of real-time detection.

[0125] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be covered within the protection scope of the present invention. In addition, the technical solutions between the various embodiments of the present invention can be combined with each other, but it must be based on the ability of ordinary technicians in the field to implement. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A wind turbine blade clearance detection method based on a lightweight segmentation network, characterized in that: The steps include: S1, collect multiple wind turbine blade images and build a training set; S2. On the basis of the original YOLOv8 network model, an improved YOLOv8-ENNet target detection network is constructed. The improved YOLOv8-ENNet target detection network includes a backbone network connected in sequence, an improved bottleneck layer Neck embedded with an ECA module, and a prediction layer Head. The backbone network uses the EfficientNetV2 backbone network that introduces the CBAM attention mechanism. S3, select a wind turbine blade image from the training set and input it into the improved YOLOv8-ENNet target detection network, and finally obtain an image with a bounding box and the target category and confidence level marked around the bounding box; that is, the detection result; S4, construct a total loss function, calculate the loss between the detection result and the true result, loop S3 to S4 until the total loss function converges, and use the back propagation algorithm to update the weights of the improved YOLOv8-ENNet target detection network to obtain the trained YOLOv8-ENNet target detection network; S5. Test the trained YOLOv8-ENNet target detection network to obtain test results, and calculate the clearance value through the test results; The EfficientNetV2 backbone network in S2 contains 8 stages in total; the 8 stages are connected in sequence; Among them, the 0th stage is a convolution layer; compared with the EfficientNetV1 network, the first to third stages replace the original MBConv module with the Fused-MBConv module to reduce the computational complexity; the fourth to sixth stages are two connected MBConv modules; the seventh stage is a sequentially connected convolution layer, Pooling&FC module and spatial pyramid pooling fast module SPPF; the third and fourth stages are embedded with the CBAM attention mechanism to reduce the impact of redundant features on subsequent convolution operations; the MBConv modules in the fourth to sixth stages are embedded with the SE module; thereby enhancing the ability of feature representation; The ECA module is embedded in the cross-stage feature fusion module C2f in the bottleneck layer Neck, and is embedded in multiple Bottleneck modules in the cross-stage feature fusion module C2f; The prediction layer Head includes three detection heads, large, medium and small, to perform multi-scale detection.

2. The fan blade clearance detection method according to claim 1, characterized in that: The S1 specifically includes the following steps: S11. Install a camera on the wind turbine tower, with the camera being mounted between the blade and the tower and facing the tip of the blade; S12. Use a camera to shoot blades, blade tips and towers to obtain a video containing towers, blades and blade tips, then extract frames and clean the video to obtain multiple wind turbine blade images, where the wind turbine blade images contain towers, blade tips and / or blades, and then use the multiple wind turbine blade images to construct a training set.

3. The fan blade clearance detection method according to claim 1, characterized in that: The ECA module weights the feature map in the channel dimension through global average pooling, convolutional layers and activation functions to enhance important channel features.

4. The fan blade clearance detection method according to claim 3, characterized in that: The working process in the ECA module is as follows: Step a1: First, the global average pooling in the ECA module will be input into the feature map of the ECA module Convert to a one-dimensional vector , capture the global spatial information and obtain the global feature representation of each channel; Step a2: Input the global feature representation of each channel into the convolutional layer. The convolutional layer uses a one-dimensional convolution kernel channel descriptor of size k to perform 1D convolution operation to capture important feature information through local cross-channel interaction. Step a3: Pass the output of the convolutional layer through the Sigmoid activation function. The output value is compressed to between 0 and 1, and the weight of each channel is generated. The weight of each channel can appropriately adjust the feature map input to the ECA module. Information in Step a4: Multiply each channel of the original feature map by the corresponding channel attention weight to generate an enhanced feature map, i.e., a reweighted feature map.

5. The fan blade clearance detection method according to claim 4, characterized in that: The one-dimensional vector in step a1 The details are as follows: ； Among them, C is the total number of channels, H is the height of the feature map, and W is the width of the feature map; is the feature map input to the ECA module No. channels; Indicates the current channel number; , They refer to the feature maps The height and width of The calculation formula of the 1D convolution operation in step a2 is: ； Among them, k is the convolution kernel size; Represents the output result after 1D convolution operation; The calculation formula of the Sigmoid function in step a3 is: ； in, It is The attention weight of each channel; Represents the symbol of Sigmoid function; The calculation formula of the re-weighted feature map is: ； in, is the reweighted channels.

6. The fan blade clearance detection method according to claim 1, characterized in that: The S3 specifically includes the following steps: S31. Select a fan blade image from the training set and input it into the improved YOLOv8-ENNet target detection network. The EfficientNetV2 backbone network extracts features from the fan blade image and outputs several feature maps of different scales. S32, inputting several feature maps of different scales into the improved bottleneck layer Neck and performing feature fusion and optimization, and outputting fused feature maps having the same number as the feature maps of different scales; S33. Input the fused feature maps into the prediction layer Head. The prediction layer Head maps the fused feature maps into a preset target detection frame to obtain a picture with a bounding box and the target category and confidence level marked around the bounding box, that is, the detection result.

7. The fan blade clearance detection method according to claim 1, characterized in that: The total loss function in S4 is formed by adding the position loss function, the confidence loss function and the classification loss function according to the set coefficients; The position loss function is used to calculate the position loss between the bounding box obtained in S3 and the true bounding box; the confidence loss function is used to calculate the loss between the confidence obtained in S3 and the true confidence; the classification loss function is used to calculate the loss between the target category obtained in S3 and the true category.

8. The fan blade clearance detection method according to claim 1, characterized in that: The S5 specifically includes the following steps: S51, select a wind turbine blade image and input it into the trained YOLOv8-ENNet target detection network to obtain the test result; S52, measuring the tower edge and the blade tip position according to the test result to obtain the pixel straight-line distance between the two; S53, then using the tower edge and the blade tip position to obtain the pixel distance between the two, and performing detection and calculation by a fan blade tip and fan blade fusion detection method or a blade tip segmentation detection method to obtain a final clearance value; wherein the blade tip segmentation detection method calculates the clearance value by the following formula, and the specific formula is as follows: ； in, Indicates the net value, Indicates the pixel value between the blade tip and the tower; Indicates the calibrated pixel distance.

9. The fan blade clearance detection method according to claim 8, characterized in that: The selection rules of the two detection methods of the fan blade tip and fan blade fusion detection method and the blade tip segmentation detection method are as follows: When the blades in the fan blade image are not blocked, the fan blade tip and fan blade fusion detection method is used for detection and calculation; otherwise, the blade tip segmentation detection method is used for detection and calculation to ensure the stability of blade tip detection.

Images