A method for detecting blade tip defects in offshore wind turbines based on Mobile JAM networks

Abstract

This invention provides a method for detecting blade tip defects in offshore wind turbines based on a Mobile JAM network, belonging to the fields of deep learning and computer vision. The method includes: creating a standardized dataset directory structure; visually annotating the original offshore wind turbine blade images while removing special characters from image file names; dividing the annotated data into training, validation, and test sets according to a preset ratio; capturing positive sample images of blade tip defects and extracting defect region patches; using data augmentation techniques to randomly transform the defect region patches and paste them onto the background image; building a Mobile JAM detection model architecture; optimizing the network training; real-time monitoring and adjustment during training; and performing blade tip defect detection on new input images. This method can achieve accurate detection of blade tip defects in offshore wind turbines even with few target sample categories, and has good application prospects in areas such as fine-grained detection of blade tip defects.

Description

Technical Field

[0001] This invention relates to the field of wind turbine blade tip defect detection, and more particularly to a method for detecting defects at the blade tips of offshore wind turbines based on a Mobile JAM network. This method is primarily used for defect detection at the blade tips of offshore wind turbines. Background Technology

[0002] As a crucial clean energy device, the integrity of the blade tip region is paramount for the safe operation of offshore wind turbines. Traditional blade tip defect detection relies primarily on manual inspection, which suffers from low efficiency, high cost, and significant risks. While existing deep learning-based object detection methods have achieved some degree of automation, they still face challenges in practical applications: defective positive samples are far fewer than defect-free negative samples, making model training difficult; defective regions occupy a small proportion of the overall image, making feature extraction challenging; and high-precision models often require substantial computation, making real-time detection difficult. Existing deep learning methods such as Faster R-CNN and YOLO perform well in general object detection, but their accuracy is limited in specific small-object detection tasks like offshore wind turbine blade tip defects, and they also demand high computational resources. Therefore, a blade tip defect detection method specifically designed for offshore wind turbine blades is needed. Summary of the Invention

[0003] The purpose of this invention is to provide a method for detecting blade tip defects in offshore wind turbines based on Mobile JAM networks, addressing the technical problems of sample imbalance, small target detection, and model complexity in existing wind turbine blade detection methods. This method can achieve accurate detection of blade tip defects in offshore wind turbines.

[0004] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0005] A method for detecting blade tip defects in offshore wind turbines based on a Mobile JAM network, the method comprising the following steps:

[0006] Step 1: Create a standardized dataset directory structure, including subdirectories for training, validation, and test sets;

[0007] Step 2: Use interactive annotation tools to visually annotate the original offshore wind turbine blade images, generate label files, and clean up special characters in the image file names to ensure file system compatibility;

[0008] Step 3: Divide the labeled data into training set, validation set and test set according to the preset ratio, and automatically organize them into the corresponding directories;

[0009] Step 4: Capture positive sample images of leaf tip defects and extract defect area patches, then uniformly scale them to a standard size;

[0010] Step 5: Using data augmentation techniques, randomly transform the defective area patch and paste it onto the background image;

[0011] Step 6: Build a Mobile JAM detection model architecture based on MobileViT and attention mechanisms;

[0012] Step 7: Optimize network training by combining cosine annealing scheduling and early stopping mechanism, monitor loss and accuracy in real time during training, and visualize training curves;

[0013] Step 8: Perform test set inference on the trained model and output the predicted class and confidence score;

[0014] Step 9: Generate a detailed prediction results report, including file name, prediction category, and confidence statistics;

[0015] Step 10: Perform leaf tip defect detection on the new input image.

[0016] Furthermore, in step 1, three subdirectories, train, val, and test, are created under the data root directory. Under each subdirectory, two category directories, has_defect and no_defect, are created to form a complete six-level directory structure.

[0017] Furthermore, in step 2, the image is displayed using an OpenCV visualization interface, allowing users to annotate it using keyboard keys: the y key marks it as defective, the n key marks it as defect-free, the s key skips the current image, and the q key exits and saves the annotation results. Regular expressions are used to match and replace non-alphanumeric characters in the filename with underscores, while protecting the integrity of the image file extension.

[0018] Furthermore, in step 4, the defective area is cropped from the center of the defective image, the patch is uniformly adjusted to 128×128 pixels, and saved as a PNG format.

[0019] Furthermore, in step 5, the patch is randomly rotated from -30° to +30°, randomly scaled by 0.6 to 1.4 times, and its brightness and contrast are randomly adjusted. Then, the transformed patch is randomly pasted into a reasonable position on the background image. The composite image is checked, and images with obvious problems are removed.

[0020] Furthermore, the specific process of step 6 is as follows:

[0021] Step 6.1: Feature Extraction Perform two-branch convolution operations;

[0022] Step 6.2: After convolution of the two branches, sum the features and then perform softmax smoothing.

[0023] ;

[0024] In the formula, For one branch output, It has two branch outputs;

[0025] Step 6.3: Perform residual connections between the attention weights and the original features, and feed them into the fully connected layer. Use the channel attention mechanism to perform weighted fusion of features at different scales.

[0026]

[0027] In the formula, Let represent the enhanced feature of the i-th channel, D be the feature dimension, and N be the total number of spatial locations involved in the attention calculation. This represents the original feature of the i-th channel. Let be the learnable parameters in the residual connection of the i-th channel. Let be the attention weight of the i-th channel at the j-th spatial location.

[0028] Furthermore, in step 7, the AdamW optimizer is used with an initial learning rate of 1e-4. Cosine annealing learning rate scheduling is adopted, and an early stopping mechanism is implemented based on the validation set accuracy. The loss value and accuracy of each training cycle are recorded in real time, and training loss curves and accuracy curves are generated and automatically saved as PNG format files.

[0029] Furthermore, in step 8, batch inference for a single image or the entire catalog outputs the predicted category of each image as defective or defect-free, along with the corresponding confidence score.

[0030] Further, in step 9, a CSV file containing the file name, predicted category, confidence level, and probability of each category is created, and a statistical summary of the prediction results and category distribution analysis are provided.

[0031] Furthermore, the specific process of step 10 is as follows:

[0032] Step 10.1: Define task tuples to dynamically construct training tasks from the dataset, each task... Includes a support set and a query set ,Right now ;

[0033] Step 10.2: Set task parameters. For each training task, randomly select C categories, and each category contains K samples to form a C-way K-shot learning task.

[0034] Step 10.3: Construct the support set by randomly sampling from the has_defect and no_defect categories in the training set. Where m = C × K, For the input sample, Category labels;

[0035] Step 10.4: Construct a query set by randomly selecting a number of samples from the remaining samples of the same category. This is used to evaluate the model's generalization ability on this task;

[0036] Step 10.5: Task batch generation. In each training cycle, repeat the above process to generate several batches of tasks, ensuring that the model is exposed to a large number of different task configurations in order to learn transferable feature representations.

[0037] The present invention, by adopting the above-described technical solution, has the following beneficial effects:

[0038] This invention uses interactive annotation and automated data partitioning, which greatly reduces data preparation time. It effectively solves the sample imbalance problem by using image enhancement, which improves the model's generalization ability. It uses the Mobile JAM network to significantly reduce computational complexity while maintaining high accuracy. Combined with cosine annealing and early stopping mechanisms, it ensures the stability and convergence of the model training process. It can achieve accurate detection of blade tip defects in offshore wind turbines when the target sample categories are scarce, and has good application prospects in the field of fine detection of blade tip defects. Attached Figure Description

[0039] Figure 1 This is a flowchart of the method of the present invention;

[0040] Figure 2 This is a diagram of the overall network structure of the present invention;

[0041] Figure 3 This is a structural diagram of the global attention mechanism of this invention. Detailed Implementation

[0042] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and preferred embodiments. However, it should be noted that many details listed in the specification are merely to provide the reader with a thorough understanding of one or more aspects of the present invention, and these aspects of the invention can be implemented even without these specific details.

[0043] like Figure 1The diagram shown illustrates the network structure of this invention. This invention constructs a standardized dataset directory structure, uses interactive annotation tools for data annotation, and employs copy-paste data augmentation techniques to generate balanced training samples. A Mobile JAM detection model based on MobileViT and attention mechanisms is built. This model uses bi-branch feature extraction and attention mechanisms to achieve accurate detection of blade tip defects in offshore wind turbines. The training process is optimized through progressive learning rate scheduling and an early stopping mechanism, ultimately providing a complete model deployment interface to support online detection and batch processing.

[0044] A method for detecting blade tip defects in offshore wind turbines based on a Mobile JAM network includes the following steps:

[0045] S1: Create a standardized dataset directory structure, including subdirectories for training, validation, and test sets;

[0046] To strictly distinguish between training, validation, and testing data, prevent data leakage during model evaluation, adapt to the data loading requirements of deep learning frameworks, and utilize the framework's automatic data loading function, a reliable data foundation is provided for model training, parameter tuning, and final performance testing, ensuring the repeatability and credibility of experimental results.

[0047] Step S1 includes the following sub-steps:

[0048] S11: Create three subdirectories, train, val, and test, under the data root directory. In each subdirectory, create two category directories, has_receptor and no_receptor, to form a complete six-level directory structure.

[0049] S2: Use interactive annotation tools to visually annotate the original offshore wind turbine blade images and generate label files;

[0050] In order to provide accurate supervision signals for model training, a high-quality dataset is constructed to solve the problem of imbalanced samples and improve the accuracy and reliability of leaf tip defect detection.

[0051] Step S2 includes the following sub-steps:

[0052] S21: Displays images through an OpenCV visualization interface, allowing users to annotate using keyboard keys: 'y' key marks as defective, 'n' key marks as defect-free, 's' key skips the current image, and 'q' key exits and saves the annotation results.

[0053] S3: Clean up special characters in image filenames to ensure file system compatibility;

[0054] To ensure cross-platform file system compatibility and prevent training interruptions or data loading anomalies due to path resolution failures.

[0055] Step S3 includes the following sub-steps:

[0056] S31: Use regular expressions to match and replace non-alphanumeric characters in filenames with underscores, while protecting the integrity of image file extensions.

[0057] S4: Divide the labeled data into training set, validation set and test set according to the preset ratio, and automatically organize them into the corresponding directories;

[0058] To prevent data leakage during model training and ensure the authenticity and reliability of evaluation results, automatic organization into corresponding directories facilitates data management and model tuning, thereby improving experimental efficiency.

[0059] Step S4 includes the following sub-steps:

[0060] S41: Divide the data into training, validation and test sets in a 7:2:1 ratio, while maintaining the balance of class distribution in each set.

[0061] S5: Capture positive sample images of leaf tip defects and extract defect area patches, then uniformly scale them to a standard size;

[0062] To build a patch library and standardize input size, and to address the problem of insufficient positive samples, data augmentation was used to improve the model's detection accuracy and robustness for small target leaf tip defects.

[0063] Step S5 includes the following sub-steps:

[0064] S51: Crop the defective area from the center of the defective image, adjust the patch to a uniform size of 128×128 pixels, and save it as a PNG file.

[0065] S6: Using data augmentation technology, the defect area patch is randomly transformed and pasted onto the background image;

[0066] To effectively address the imbalance between positive and negative samples, data augmentation techniques are used to synthesize the transformed defect region patches into the background image, thereby expanding the training dataset and improving the model's generalization ability.

[0067] Step S6 includes the following sub-steps:

[0068] S61: Randomly rotate the patch (-30° to +30°), randomly scale it (0.6-1.4 times), and randomly adjust its brightness and contrast. Then, randomly paste the transformed patch into a reasonable position on the background image.

[0069] S7: Build a Mobile JAM detection model architecture based on MobileViT and attention mechanisms;

[0070] To combine the global perception of MobileViT with the local feature extraction advantages of CNN, we enhance the features of defect regions through an attention mechanism to improve the detection accuracy of small targets, while keeping the model lightweight to adapt to actual deployment needs.

[0071] Step S7 includes the following sub-steps:

[0072] S71: Feature Extraction Perform two-branch convolution operations;

[0073] S72: After convolution of the two branches, the features are summed and then smoothed using softmax.

[0074] ;

[0075] In the formula, For one branch output, It has two branch outputs;

[0076] S73: The attention weights are residually connected to the original features and then fed into the fully connected layer. The channel attention mechanism is used to perform weighted fusion of features at different scales.

[0077]

[0078] In the formula, Let represent the enhanced feature of the i-th channel, D be the feature dimension, and N be the total number of spatial locations involved in the attention calculation. This represents the original feature of the i-th channel. Let be the learnable parameters in the residual connection of the i-th channel. Let be the attention weight of the i-th channel at the j-th spatial location.

[0079] S8: Implement a network training optimization method that combines cosine annealing scheduling and early stopping mechanism;

[0080] To achieve dynamic adjustment of the learning rate and prevent overfitting, cosine annealing scheduling is combined to help the model escape local optima, while early stopping mechanism is used to terminate training when performance is saturated, thereby improving convergence efficiency and saving computational resources.

[0081] Step S8 includes the following sub-steps:

[0082] S81: Uses the AdamW optimizer with an initial learning rate of 1e-4, employs cosine annealing for learning rate scheduling, and implements an early stopping mechanism based on validation set accuracy.

[0083] S9: Monitor loss and accuracy in real time during training and visualize training curves;

[0084] To enable real-time diagnosis and optimization of the training process, monitoring loss and accuracy can promptly detect gradient anomalies or overfitting, while visualizing curves helps analyze convergence trends and guide hyperparameter adjustments.

[0085] Step S9 includes the following sub-steps:

[0086] S91: Records the loss value and accuracy for each training cycle in real time, generates training loss curves and accuracy curves, and automatically saves them as PNG format files.

[0087] S10: Perform test set inference on the trained model and output the predicted class and confidence level;

[0088] To evaluate the model's generalization ability in real-world scenarios, test set inference can verify the model's recognition performance on unseen data, while the output of predicted categories and confidence levels provides a reliable basis for decision-making and performance analysis in practical applications.

[0089] Step S10 includes the following sub-steps:

[0090] S101: Supports batch inference for a single image or an entire catalog, outputting the predicted category (defective / no defect) and the corresponding confidence score for each image.

[0091] S11: Generate a detailed forecast report, including filename, forecast category, and confidence statistics;

[0092] To provide traceable records of detection results, the original data is linked by file name, the predicted category gives the judgment conclusion, and the confidence score quantifies the reliability of the model, providing data support for subsequent error analysis and system optimization.

[0093] Step S11 includes the following sub-steps:

[0094] S111: Create a CSV file containing the filename, predicted category, confidence level, probability of each category, and provide a statistical summary of the prediction results and category distribution analysis.

[0095] S12: Provides a complete model deployment interface, supporting leaf tip defect detection on new input images;

[0096] To transform the trained model into practical productivity, automated detection is achieved through deployment interfaces, supporting batch processing of offshore wind turbine blade images, outputting standardized results, and improving engineering application efficiency and system scalability.

[0097] Step S12 includes the following sub-steps:

[0098] S121: Define task tuples: Dynamically construct training tasks from the dataset, each task Includes a support set and a query set ,Right now ;

[0099] S122: Set task parameters: For each training task, C classes are randomly selected, and each class contains K samples, forming a C-way K-shot learning task.

[0100] S123: Constructing the support set: Randomly sample from the has_defect and no_defect categories in the training set to form the support set. Where m = C × K, For the input sample, Category labels;

[0101] S124: Construct a query set: Randomly select several samples from the remaining samples of the same category to form a query set. This is used to evaluate the model's generalization ability on this task;

[0102] S125: Task Batch Generation: In each training cycle, the above process is repeated to generate multiple task batches, ensuring that the model is exposed to a large number of different task configurations to learn transferable feature representations.

[0103] Matters not covered in this invention are common knowledge.

[0104] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for detecting blade tip defects in offshore wind turbines based on a Mobile JAM network, characterized in that, The method includes the following steps: Step 1: Create a standardized dataset directory structure, including subdirectories for training, validation, and test sets; Step 2: Use interactive annotation tools to visually annotate the original offshore wind turbine blade images, generate label files, and clean up special characters in the image file names to ensure file system compatibility; Step 3: Divide the labeled data into training set, validation set and test set according to the preset ratio, and automatically organize them into the corresponding directories; Step 4: Capture positive sample images of leaf tip defects and extract defect area patches, then uniformly scale them to a standard size; Step 5: Using data augmentation techniques, randomly transform the defective area patch and paste it onto the background image; Step 6: Build a Mobile JAM detection model architecture based on MobileViT and attention mechanisms; Step 7: Optimize network training by combining cosine annealing scheduling and early stopping mechanism, monitor loss and accuracy in real time during training, and visualize training curves; Step 8: Perform test set inference on the trained model and output the predicted class and confidence score; Step 9: Generate a detailed prediction results report, including file name, prediction category, and confidence statistics; Step 10: Perform leaf tip defect detection on the new input image.

2. The method for detecting blade tip defects of offshore wind turbines based on Mobile JAM networks according to claim 1, characterized in that: In step 1, three subdirectories, train, val, and test, are created under the data root directory. Under each subdirectory, two category directories, has_defect and no_defect, are created to form a complete six-level directory structure.

3. The method for detecting blade tip defects of offshore wind turbines based on Mobile JAM networks according to claim 1, characterized in that: In step 2, the image is displayed using an OpenCV visualization interface, allowing users to annotate it using keyboard keys: the y key marks a defective image, the n key marks a defect-free image, the s key skips the current image, and the q key exits and saves the annotation results. Regular expressions are used to match and replace non-alphanumeric characters in filenames with underscores, while protecting the integrity of the image file extension.

4. The method for detecting blade tip defects of offshore wind turbines based on Mobile JAM networks according to claim 1, characterized in that: In step 4, the defective area is cropped from the center of the defective image, the patch is uniformly adjusted to 128×128 pixels, and saved as a PNG format.

5. The method for detecting blade tip defects of offshore wind turbines based on Mobile JAM networks according to claim 1, characterized in that: In step 5, the patch is randomly rotated from -30° to +30°, randomly scaled by 0.6 to 1.4 times, and its brightness and contrast are randomly adjusted. Then, the transformed patch is randomly pasted into a reasonable position on the background image. The composite image is checked, and images with obvious problems are removed.

6. The method for detecting blade tip defects of offshore wind turbines based on Mobile JAM networks according to claim 1, characterized in that, The specific process of step 6 is as follows: Step 6.1: Feature Extraction Perform two-branch convolution operations; Step 6.2: After convolution of the two branches, sum the features and then perform softmax smoothing. ； In the formula, For one branch output, It has two branch outputs; Step 6.3: Perform residual connections between the attention weights and the original features, and feed them into the fully connected layer. Use the channel attention mechanism to perform weighted fusion of features at different scales. In the formula, Let represent the enhanced feature of the i-th channel, D be the feature dimension, and N be the total number of spatial locations involved in the attention calculation. This represents the original feature of the i-th channel. Let be the learnable parameters in the residual connection of the i-th channel. Let be the attention weight of the i-th channel at the j-th spatial location.

7. The method for detecting blade tip defects of offshore wind turbines based on Mobile JAM networks according to claim 1, characterized in that, In step 7, the AdamW optimizer is used with an initial learning rate of 1e-4. Cosine annealing is used for learning rate scheduling, and an early stopping mechanism is implemented based on the validation set accuracy. The loss value and accuracy of each training cycle are recorded in real time, and training loss curves and accuracy curves are generated and automatically saved as PNG format files.

8. The method for detecting blade tip defects of offshore wind turbines based on Mobile JAM networks according to claim 1, characterized in that: In step 8, batch inference is performed on a single image or the entire catalog, outputting the predicted category of each image as defective or defect-free, along with the corresponding confidence score.

9. The method for detecting blade tip defects of offshore wind turbines based on Mobile JAM networks according to claim 1, characterized in that: In step 9, a CSV file containing the file name, predicted category, confidence level, and probability of each category is created, along with a statistical summary of the prediction results and category distribution analysis.

10. The method for detecting blade tip defects of offshore wind turbines based on Mobile JAM networks according to claim 1, characterized in that, The specific process of step 10 is as follows: Step 10.1: Define task tuples to dynamically construct training tasks from the dataset, each task... Includes a support set and a query set ,Right now ; Step 10.2: Set task parameters. For each training task, randomly select C categories, and each category contains K samples to form a C-way K-shot learning task. Step 10.3: Construct the support set by randomly sampling from the has_defect and no_defect categories in the training set. Where m = C × K, For the input sample, Category labels; Step 10.4: Construct a query set by randomly selecting a number of samples from the remaining samples of the same category. This is used to evaluate the model's generalization ability on this task; Step 10.5: Task batch generation. In each training cycle, repeat the above process to generate several batches of tasks, ensuring that the model is exposed to a large number of different task configurations in order to learn transferable feature representations.

Images