Offshore wind turbine blade defect detection method

Abstract

The invention provides an offshore wind turbine blade defect detection method. The method comprises the following steps: carrying out labeling processing on an obtained offshore wind turbine blade operation image set to obtain a training sample; an offshore wind turbine blade defect identification model based on the YOLO V8 target identification model is constructed; training an offshore wind turbine blade defect identification model by using the training sample; and inputting a real-time offshore wind turbine blade operation image into the trained offshore wind turbine blade defect identification model to obtain a detection result. According to the method, the instantaneity, the detection precision, the detail identification capability and the adaptability of offshore wind turbine blade defect identification can be effectively improved.

Description

Technical Field

[0001] The present application belongs to the technical field of offshore wind power generation, and specifically, relates to a method for detecting defects in offshore wind turbine blades. Background Art

[0002] Offshore wind power generation is an important part of renewable energy. However, the harsh offshore environment and frequent lightning strikes make offshore wind turbine blades prone to failure, resulting in high maintenance costs. Timely and accurate detection of abnormal conditions of wind turbines is crucial to ensure their safe operation and extend their service life. Offshore wind turbine blade anomalies include blade coating damage, scratches, cracks, and black spots caused by lightning strikes. Traditional detection methods mainly rely on regular maintenance and sensor-based data monitoring, but these methods have many problems, such as the need for manual feature design, poor generalization ability, sensitivity to noise and interference, difficulty in handling complex backgrounds and occlusions, inability to achieve end-to-end learning, high reliance on professional knowledge, difficulty in dealing with the diversity of faults, and difficulty in meeting real-time requirements. However, these methods have limited real-time performance and accuracy. At the same time, large-scale benchmark test data for wind turbine anomaly detection is still not comprehensive enough.

[0003] In recent years, computer vision and machine learning technologies have been widely used in wind turbine anomaly detection to improve detection efficiency and accuracy. By using computer vision technology, blade anomalies, such as damage, scratches or cracks, can be automatically identified and alarms can be issued in time to reduce maintenance time and costs. However, these methods still have room for improvement in terms of real-time performance and accuracy. Summary of the invention

[0004] The technical problem solved by the present application is: how to provide an offshore wind turbine blade defect detection method that can improve real-time performance, detection accuracy, detail recognition capability and adaptability.

[0005] The present application provides a method for detecting defects in offshore wind turbine blades, the method comprising:

[0006] The acquired video data of offshore wind turbine blade operation is labeled to obtain training samples;

[0007] An offshore wind turbine blade defect recognition model based on the YOLO V8 target recognition model was constructed;

[0008] Using the training samples to train the offshore wind turbine blade defect recognition model;

[0009] The real-time operating image of the offshore wind turbine blade is input into the trained offshore wind turbine blade defect recognition model to obtain the detection results.

[0010] Optionally, the method of labeling the acquired offshore wind turbine blade operation video data to obtain training samples includes:

[0011] 5 frames of offshore wind turbine blade operation images are captured from the offshore wind turbine blade operation video in 1 second;

[0012] Perform data enhancement processing on each offshore wind turbine blade operation image, so that each offshore wind turbine blade operation image is expanded into multiple images;

[0013] The image annotation tool is used to annotate the offshore wind turbine blade operation image to obtain a training sample.

[0014] Optionally, the labels of the offshore wind turbine blade operation image after data annotation include: blade lightning strike point, blade crack, blade scratch, and blade coating damage.

[0015] Optionally, the data enhancement processing includes at least one of the following operations: rotation, scaling, cropping and color transformation.

[0016] Optionally, the method for constructing an offshore wind turbine blade defect recognition model based on the YOLO V8 target recognition model includes:

[0017] Replace all modules except the SPPF module in the backbone of the YOLO V8 target recognition model with the accelerated reasoning network architecture FasterNet;

[0018] The c2f convolution module in the head of the YOLO V8 target recognition model is replaced with the precision-enhanced convolution block CrossDySnakeConv to obtain the offshore wind turbine blade defect recognition model.

[0019] Optionally, the precision improvement convolution block CrossDySnakeConv includes:

[0020] The initial convolution layer is used to convert the number of input channels into the intermediate number of feature channels and perform preliminary extraction of input features;

[0021] The multi-level feature extraction Bottleneck_DySnakeConv layer is used to adaptively adjust the convolution kernel according to the input features to capture complex features;

[0022] The concatenation layer is used to concatenate multiple feature vectors to obtain a feature tensor;

[0023] The output convolution layer is used to output the feature tensor.

[0024] The present application provides a method for detecting defects in offshore wind turbine blades, which has the following technical effects:

[0025] 1. Efficient real-time detection: By accelerating the inference network architecture FasterNet, the model inference speed is improved, enabling the model to handle the detection tasks of offshore wind turbine blades in real time, providing rapid response and timely feedback; it realizes continuous monitoring of offshore wind turbines and real-time fault warning, significantly improving the safety and reliability of offshore wind turbine operation.

[0026] 2. High-precision recognition: Through the precision-enhancing convolution block CrossDySnakeConv, the model has higher detection accuracy under complex backgrounds and multi-scale targets; it can accurately identify and locate tiny defects on the blades (such as scratches, cracks, lightning black spots, and coating damage), reduce missed detections and false detections, and improve detection quality.

[0027] 3. Strong robustness and adaptability: The model has strong robustness and adaptability, and can work stably in complex and changeable marine environments; it can adapt to different weather and lighting conditions to ensure accurate detection in various environments, thereby improving the reliability and wide application of the system.

[0028] 4. Multi-scale target processing capability: The model can effectively handle targets of different scales, whether it is large-scale blade damage or tiny cracks and scratches, it can accurately identify and locate them; provide a comprehensive assessment of the health status of offshore wind turbine blades, help to promptly detect and handle different types of defects, and extend the service life of offshore wind turbines. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 is a flow chart of a method for detecting defects in offshore wind turbine blades according to one or more embodiments;

[0030] Figure 2 A system framework diagram of an offshore wind turbine blade defect recognition model according to one or more embodiments;

[0031] Figure 3 A schematic diagram of an accelerated reasoning network architecture FasterNet according to one or more embodiments;

[0032] Figure 4 A schematic diagram of a precision enhancement convolution block CrossDySnakeConv according to one or more embodiments;

[0033] Figure 5 A comparison chart of the accuracy of the ablation experiment performed according to one or more embodiments with other models. DETAILED DESCRIPTION

[0034] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0035] Before describing the various embodiments of the present application in detail, the technical concept of the present application is first briefly described: the traditional offshore wind turbine blade defect detection method is insufficient in terms of real-time performance, detection accuracy, detail recognition capability and adaptability. To this end, the offshore wind turbine blade defect detection method provided by the present application constructs an offshore wind turbine blade defect recognition model based on the YOLO V8 target recognition model, uses the annotated offshore wind turbine blade operation images for training, and uses the trained offshore wind turbine blade defect recognition model to detect the real-time offshore wind turbine blade operation images. This method can effectively improve the real-time performance, detection accuracy, detail recognition capability and adaptability of defect recognition. The specific principles of the offshore wind turbine blade defect detection method of the present application are described in combination with more embodiments below.

[0036] Specifically, Figure 1 and Figure 2 As shown, the offshore wind turbine blade defect detection method provided in the first embodiment includes the following steps:

[0037] Step S10: labeling the acquired offshore wind turbine blade operation image set to obtain training samples;

[0038] Step S20: constructing an offshore wind turbine blade defect recognition model based on the YOLO V8 target recognition model;

[0039] Step S30: using the training samples to train an offshore wind turbine blade defect recognition model;

[0040] Step S40: inputting the real-time offshore wind turbine blade operation image into the trained offshore wind turbine blade defect recognition model to obtain a detection result.

[0041] Specifically, the method for labeling the acquired offshore wind turbine blade operation image set to obtain training samples includes a process.

[0042] First, image frames are captured at a frequency of 5 frames per second from the offshore wind turbine blade operation video. Specifically, multiple offshore wind turbine operation videos are shot using drones and processed using Python scripts. The script uses the OpenCV library, uses the cv2.VideoCapture function to read video files, obtains the video frame rate through the cv2.CAP_PROP_FPS property, and uses the cv2.imwrite function to save the captured image frames to the specified directory. Subsequently, after manual screening, photos with obvious features, accurate labels and metadata are retained, and blurred, overexposed or underexposed photos are removed. Offshore wind turbine blade operation images of various scenes, scales, and operating states are captured, and the offshore wind turbine blade operation images are divided into 70% for training, 15% for validation, and 15% for test.

[0043] Next, data enhancement processing is performed on each offshore wind turbine blade operation image, so that each offshore wind turbine blade operation image is expanded into multiple images. Specifically, this paper uses the torchvision.transforms module in the PyTorch platform for data enhancement. The specific operations used include: using the transforms.RandomRotation function to randomly rotate the image; using the transforms.RandomResizedCrop function to randomly crop the image and scale it to a specified size; using the transforms.RandomCrop function to randomly crop the image; and using the transforms.ColorJitter function to randomly transform the brightness, contrast, saturation and hue of the image. This is an important technology to improve the performance and robustness of the model, effectively expand the data set, increase the diversity of the data, and thus improve the generalization ability of the model and its performance in practical applications. Exemplarily, data enhancement is performed by at least one of the following operations: rotation, scaling, cropping and color transformation, so as to increase the amount of training data, as well as the performance and generalization ability of the model.

[0044] Finally, the image annotation tool was used to annotate the images of offshore wind turbine blade operation to obtain training samples. In the training set and validation set, the image annotation tool was used to annotate the images for four abnormal conditions: blade lightning strike point, blade crack, blade scratch, and blade coating damage, and the corresponding labels were generated.

[0045] In one or more embodiments, the method for constructing an offshore wind turbine blade defect recognition model based on the YOLO V8 target recognition model includes two parts:

[0046] First, replace all modules except the SPPF module in the backbone of the YOLO V8 target recognition model with the accelerated reasoning network architecture FasterNet. Figure 3 As shown in the figure, by integrating the accelerated reasoning network architecture FasterNet, the backbone part in the YOLOV8 target recognition network is replaced with the accelerated reasoning network architecture FasterNet. By optimizing the computing efficiency and network structure, the model maintains real-time and high efficiency when processing images.

[0047] Second, the c2f convolution module in the head of the YOLO V8 target recognition model is replaced with the precision-enhanced convolution block CrossDySnakeConv to obtain an offshore wind turbine blade defect recognition model. Combining multi-level feature extraction and feature vector concatenation, it can more comprehensively capture the spatial and local variation characteristics of the input data. Figure 4 As shown, the precision-enhanced convolution block CrossDySnakeConv includes:

[0048] The initial convolution layer cv1 is used to convert the number of input channels into the number of intermediate feature channels and perform preliminary extraction of input features. The initial convolution layer cv1 converts the number of input channels into the number of intermediate feature channels, and performs preliminary extraction of input features through convolution operations and activation functions. Suppose the input tensor is x,

[0049]

[0050] In the above formula, N is the batch size, C1 is the number of input channels, H×W is the height and width of the feature map, and the expression of the feature tensor Y1 is:

[0051] Y i =SiLU(Conv(x;W1))

[0052] In the above formula, W1 is the weight of the initial convolutional layer, Conv represents the convolution operation, and SiLU is the activation function.

[0053] The multi-level feature extraction Bottleneck_DySnakeConv layer is used to adaptively adjust the convolution kernel according to the different input features to capture complex features. Each Bottleneck_DySnakeConv module uses a dynamic convolution operation to adaptively adjust the convolution kernel according to the different input features to capture complex features. Input the feature Y obtained above i , where i represents the feature of the i-th layer, the dynamic convolution operation expression is:

[0054] Y i+1 =DumamicConv(Y i ; W i+1)

[0055]

[0056] In the above formula, * represents the convolution operation, j represents the index of the dynamic convolution kernel, and W i+1 is the weight of the i+1th layer of dynamic convolution. DynamicConv is a dynamic convolution operation including BatchNorm and SiLU. BatchNorm normalizes the data and makes the activation function SiLU perform better during training.

[0057] The concatenation layer concat is used to concatenate multiple feature vectors to obtain feature tensors. The feature vectors Y1, Y2, ..., Y n Through the feature concatenation operation, they are merged into a larger feature tensor Z. This operation enables features at different levels to interact and merge. The expression of feature concatenation Z is as follows:

[0058] Z=Concat(Y1,Y2,...,Y n )

[0059] Concatenate features from different levels.

[0060] The output convolution layer is used to output the feature tensor. The combined feature tensor Z is input into the final convolution operation to generate the final output feature tensor Y out , this operation further processes and transforms the combined features through an additional convolutional layer, Y out The expression is as follows:

[0061] Y out =SiLU(Conv(Z;W out ))

[0062] The precision-enhancing convolutional block CrossDySnakeConv can efficiently capture complex input features and improve the model's expressiveness and flexibility by combining multi-level feature extraction and dynamic convolution techniques. Through feature segmentation, layer-by-layer processing, and final fusion, this module effectively improves the precision of the above-mentioned offshore wind turbine blade defect recognition model.

[0063] Exemplarily, when training is performed in step S30, the relevant parameters are set as shown in Table 1.

[0064]

[0065]

[0066] Table 1

[0067] In order to further verify the accuracy of the offshore wind turbine blade defect recognition model improved by this embodiment, an ablation experiment is performed using the YOLOV8 model as a benchmark model. The experimental environment is as described in Table 2:

[0068] Ultralytics YOLOv8.2.16 Python 3.9.19 PyTorch 2.0.0+cu118 CUDA:0 Graphics RTX 3050

[0069] Table 2

[0070] Reference Figure 5 , the experimental results are shown in Table 3:

[0071]

[0072] Table 3

[0073] It can be seen from Table 3 that the detection accuracy, recall rate, and inference speed of the offshore wind turbine blade defect recognition model of this embodiment are 53.41%, 46.39%, and 13.0 ms, respectively. It can be concluded that the offshore wind turbine blade defect recognition model proposed in this embodiment improves the accuracy while the model inference speed is much higher than the model YOLO V8+CrossDySnakeConv, achieving a double improvement in accuracy and speed, with obvious excellent results.

[0074] The offshore wind turbine blade defect detection method provided in this embodiment fills the gap in the offshore wind turbine operation abnormality sample set, and designs a precision-enhanced convolution block CrossDySnakeConv based on snake-shaped dynamic convolution and cross convolution, realizes the cross-fusion of multi-level feature vectors, and can more comprehensively capture the spatial and local change characteristics of the input data. Based on the YOLO V8 algorithm, the designed precision-enhanced convolution block CrossDySnakeConv is introduced, and the backbone part is replaced by the accelerated inference network architecture FasterNet to form the CrossDySnak-FasterNet-YOLO V8 offshore wind turbine blade multi-target defect detection model, which improves the detection accuracy, real-time detection capability, and robustness of the model.

[0075] The specific implementation methods of the present application are described in detail above. Although some embodiments have been shown and described, those skilled in the art should understand that these embodiments can be modified and improved without departing from the principles and spirit of the present application whose scope is defined by the claims and their equivalents. These modifications and improvements should also be within the scope of protection of the present application.

Claims

1. A method for detecting defects in offshore wind turbine blades, characterized in that: The method comprises: The acquired offshore wind turbine blade operation image set is annotated to obtain training samples; An offshore wind turbine blade defect recognition model based on the YOLO V8 target recognition model was constructed; Using the training samples to train the offshore wind turbine blade defect recognition model; The real-time operating image of the offshore wind turbine blade is input into the trained offshore wind turbine blade defect recognition model to obtain the detection results.

2. The offshore wind turbine blade defect detection method according to claim 1, characterized in that: The method of labeling the acquired offshore wind turbine blade image set to obtain training samples includes: From the offshore wind turbine blade operation video, 5 frames of offshore wind turbine blade operation images are captured at 1 second intervals; Perform data enhancement processing on each offshore wind turbine blade operation image, so that each offshore wind turbine blade operation image is expanded into multiple images; The image annotation tool is used to annotate the offshore wind turbine blade operation image to obtain a training sample.

3. The offshore wind turbine blade defect detection method according to claim 2, characterized in that: The labels of the offshore wind turbine blade operation image after data annotation include: blade lightning strike point, blade crack, blade scratch, and blade coating damage.

4. The offshore wind turbine blade defect detection method according to claim 2, characterized in that: The data enhancement processing includes at least one of the following operations: rotation, scaling, cropping and color transformation.

5. The offshore wind turbine blade defect detection method according to claim 2, characterized in that: The method for constructing an offshore wind turbine blade defect recognition model based on the YOLO V8 target recognition model includes: Replace all modules except the SPPF module in the backbone of the YOLO V8 target recognition model with the accelerated reasoning network architecture FasterNet; The c2f convolution module in the head of the YOLO V8 target recognition model is replaced with the precision-enhanced convolution block CrossDySnakeConv to obtain the offshore wind turbine blade defect recognition model.

6. The offshore wind turbine blade defect detection method according to claim 5, characterized in that: The precision-enhanced convolution block CrossDySnakeConv includes: The initial convolution layer is used to convert the number of input channels into the intermediate number of feature channels and perform preliminary extraction of input features; The multi-level feature extraction Bottleneck_DySnakeConv layer is used to adaptively adjust the convolution kernel according to the input features to capture complex features; The concatenation layer is used to concatenate multiple feature vectors to obtain a feature tensor; The output convolution layer is used to output the feature tensor.

Images