Fan blade fault small sample intelligent diagnosis method based on TIP-Adapter

Abstract

The invention discloses a wind turbine blade fault small sample intelligent diagnosis method and system based on TIP-Adapter, aims to solve the problems of high blade fault marking cost, long training time and insufficient generalization ability, and realizes rapid deployment without training. According to the method, a field-specific text prompt template library is constructed for the categories of cracks, erosion, normality, peeling, caving and the like, and normalization fusion is carried out on the multi-prompt text features of the same category by adopting a multi-template feature integration strategy, so that the semantic robustness of the text side is enhanced. Meanwhile, a small number of support set images are used for constructing feature cache, support set image features serve as cache keys, and label one-hot codes serve as cache values; during reasoning, calculating the similarity between query features and cache keys, obtaining attention weight through temperature parameter normalization, generating cache prediction logits, and fusing the cache prediction logits with CLIP zero sample prediction according to the cache weight to output a final diagnosis result. The invention also provides a hyper-parameter grid search strategy to automatically determine and recommend configuration, so that stable diagnosis performance is obtained under different shot settings, and the method is suitable for industrial deployment scenes such as unmanned aerial vehicle inspection, intelligent operation and maintenance platforms and mobile terminals.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and computer vision, and specifically addresses the fault identification task of wind turbine blade images. It presents a training-free, small-sample intelligent diagnostic method based on a TIP-Adapter. This method is used to classify and diagnose blade conditions with very few labeled samples, supporting outputs for categories such as cracks, erosion, normal, spalling, and structural damage, and is adapted to the rapid deployment requirements of industrial sites. Background Technology

[0002] Wind turbine blades are exposed to complex natural environments, including wind, sand, rain, snow, and ultraviolet radiation, making them prone to cracking, erosion, and spalling. The morphology of these faults varies significantly with the shooting angle, lighting, and distance. Traditional inspections rely heavily on manual labor, resulting in low efficiency and high subjectivity. While deep learning methods can improve automation, they often require large amounts of labeled data and lengthy training times, hindering rapid on-site deployment. Common issues in practical implementation include high annotation costs (requiring domain expert involvement), long training times, insufficient model generalization ability, and limited accuracy due to imprecise semantic representation of specialized domains when using general visual-language pre-trained models with generic prompts. Few-shot learning allows adaptation to be completed with only 1–16 labeled samples per class. TIP-Adapter, a CLIP-based training-free adaptation method, improves few-shot performance by building feature caches, but its performance on blade fault tasks still has room for improvement if generic text prompts are used directly. Summary of the Invention

[0003] This invention primarily addresses how to diagnose wind turbine blade faults using only 1-16 labeled samples per category, while simultaneously meeting industrial requirements for immediate deployment without training. To solve these problems, this invention employs the following technical solution, comprising the following steps:

[0004] Step 1: Design a dedicated text prompt library for wind turbine blade faults, including three categories: fault-specific templates, general templates, and technical templates. For each type of fault (crack, erosion, normal, spalling, structural damage), precise descriptive text is designed. For example, for cracks, prompts include "wind turbine blade showing crack damage" and "blade structuralcrack damage." Compared to the general prompt "a photo of crack," domain-specific prompts accurately express the semantic features of the fault, improving CLIP's zero-shot recognition capability.

[0005] Step 2: Perform CLIP text encoding on multiple text prompts of the same fault category, and generate a unified text feature prototype for that category using a fusion method of "L2 normalization - feature mean - re-normalization". This strategy can improve accuracy and enhance the robustness and expressiveness of text features compared to the single-template method.

[0006] Step 3: Use the support set image features (CLIP visual encoding) as the cache key and the label one-hot encoding as the cache value. During inference, calculate the similarity between the query feature and the cache key, scale using the temperature parameter β, and normalize using softmax to obtain the attention weight; then multiply it with the cache value to obtain the cache prediction. This mechanism requires no training process, only forward propagation, and cache construction takes very little time.

[0007] Step 4: Based on Steps 2 and 3, the CLIP zero-sample prediction output and the cached prediction output are weighted and fused according to the cache weight α to obtain the final classification result. The optimal α and β parameter configuration is determined through grid search to ensure stable diagnostic performance across different shot numbers.

[0008] Compared with the prior art, the present invention has the following advantages:

[0009] Traditional deep learning requires over a hundred labeled images per class, while this invention only requires 1-16 labeled samples per class for deployment, significantly reducing labeling costs and time investment. This invention is a training-free method; model adaptation only requires building a feature cache. Compared to methods requiring hours of GPU training, it significantly reduces the engineering deployment cycle, making it particularly suitable for rapid deployment in industrial settings. When adding new fault types, only text prompts and a small number of samples need to be added, without modifying the underlying model, facilitating on-demand expansion and iteration in industrial settings. Under different shot count settings, the method maintains stable performance, with basic performance comparable to lightweight methods requiring training, while exhibiting better generalization ability. Inference requires only a single CLIP forward pass and matrix operation, without GPU acceleration, supporting deployment on edge devices and mobile devices, making it suitable for scenarios where the computing power of field devices is limited.

[0010] Modular design. The hint library, cache, and hyperparameters can be maintained and expanded independently, facilitating migration to other industrial defect diagnosis tasks (such as composite material surface inspection, equipment appearance evaluation, etc.), and has broad application prospects. Attached Figure Description

[0011] Figure 1 This is the overall architecture diagram.

[0012] Figure 2 This is a schematic diagram of the structure of a dedicated text prompt template library.

[0013] Figure 3 This is a flowchart of the feature integration process.

[0014] Figure 4 This is a diagram illustrating the working principle of the TIP-Adapter caching mechanism.

[0015] Figure 5 This is the flowchart for adaptive fusion inference. Detailed Implementation

[0016] Step 1: System Initialization and Model Loading

[0017] Load the CLIP vision-language model with frozen parameters (such as ViT-B / 32), initialize the TIP-Adapter module, and configure the hyperparameters α and β. For more refined multilingual support or better transfer performance, a lightweight cross-language alignment adapter can be loaded.

[0018] Step 2: Preparation of the Domain Hint Library

[0019] Prepare domain-specific text prompts for each fault category, with approximately eight descriptive texts for each category, covering semantic dimensions such as fault appearance characteristics, location, process, and severity. Save them as a prompt library configuration file (JSON format) for easy version management and on-site updates.

[0020] Step 3: Generation of Category Text Features

[0021] For each fault category, all text prompts are CLIP-encoded. The encoded results are then L2-normalized, the mean of the features is taken, and L2-normalized again to obtain the final feature prototype vector for that category. This vector is used as the text-side representation in subsequent inference for zero-sample similarity calculation.

[0022] Step 4: Support set sampling and cache construction

[0023] K high-quality samples from each class are selected from the training data as the support set according to the k-shot setting (k∈[1,16]), ensuring that the samples cover different shooting conditions. The support set images are CLIP visually encoded and normalized as cache keys, and the labels are one-hot encoded as cache values. The cache construction process is a one-time computation, and can be directly used for inference after completion.

[0024] Step 5: Online Reasoning and Diagnosis

[0025] The input query image is CLIP visually encoded and normalized. The similarity to all categories of text features is calculated to obtain zero-sample logits. Simultaneously, the similarity to the cached key is calculated, adjusted by a temperature parameter β to obtain attention weights, which are then multiplied by the cached value to obtain cached logits. Finally, the two logits are fused using an α weight to output the predicted category and confidence score. When the confidence score is below a threshold, a "manual review required" flag is output.

[0026] Step 6: Hyperparameter Optimization

[0027] A grid search is performed on a small validation set to identify and record the optimal parameter combinations for α and β. Based on the number of shots and wind field characteristics in the actual application, recommended parameters are selected and embedded into the deployment configuration. Multiple configuration files can be maintained for different scenarios.

[0028] This invention achieves training-free, small-sample diagnosis of wind turbine blade faults by designing domain-specific text prompts, employing multi-template feature integration, constructing a caching adaptation mechanism, and introducing automatic hyperparameter optimization. This method features low annotation costs, short deployment cycles, and minimal resource consumption, making it particularly suitable for rapid field deployment and on-demand expansion. It represents a significant improvement over both the CLIP zero-sample method and the basic TIP-Adapter method, and can be widely applied to intelligent operation and maintenance and preventative maintenance in wind power generation companies, playing a crucial role in reducing wind turbine failure rates and extending blade life.

Claims

1. A small-sample intelligent diagnostic method for wind turbine blade faults based on TIP-Adapter, used to achieve rapid diagnosis and classification of wind turbine blade faults. The implementation process mainly includes the following steps: Step 1) Construct a domain-specific text prompt template library. Design multiple descriptive text prompt templates for the five states of wind turbine blades: cracks, erosion, coating peeling, material peeling, and normal conditions, forming a correspondence between fault categories and text descriptions. Step 2) Load the pre-trained CLIP vision-language model, use CLIP's image encoder to extract the visual features of the leaf image, and use CLIP's text encoder to encode the text prompt template in Step 1) to obtain the text feature vector. Step 3) Integrate the features of multiple text prompt templates for each fault category, and use a feature averaging strategy to average the multiple text features of the same category and then perform L2 normalization to generate a unified category text representation. Step 4) Construct a support set feature cache, extract CLIP visual features from a small number of labeled sample images (1-16 images per class) and perform L2 normalization as the cache key, and encode the corresponding labels one-hot as the cache value; Step 5) Perform fault diagnosis on the query image, calculate the attention weights of the query image features and cache keys, and perform adaptive fusion by combining zero-shot CLIP prediction and cache prediction to output the final fault classification result. Step 6) Based on the diagnostic results, output the fault category and corresponding confidence level for the condition assessment and maintenance decision of wind turbine blades.

2. According to claim 1, the TIP-Adapter model includes the following features: (1) Utilize the zero-shot capability of the CLIP pre-trained model to achieve semantic understanding of blade faults through domain-specific text prompts; (2) By supporting the feature caching mechanism, the feature information of a small number of labeled samples is stored as key-value pairs, so as to achieve fast adaptation without training.

3. According to claim 1 or 2, wherein the adaptive fusion in step 5) is determined by the following formula: (1) Zero-sample CLIP prediction: ,in For CLIP's temperature parameters, To query image features, The text feature matrix; (2) Caching attention prediction: , , ,in For cache key, β is the cache value, and β is the attention temperature parameter; (3) Fusion output: ,in This is for caching weight parameters.

4. According to claim 3, wherein the hyperparameters α and β are determined by performing a grid search on the validation set. The search range is [0.5, 3.0]. The search range is [1.0, 50.0], and the optimal configuration is... =1.6, =6.

0.

5. According to claim 4, the domain-specific text prompt template library includes four categories: fault-specific templates, general templates, technical templates, and contextual templates. Eight professional descriptive texts are designed for each fault type to enhance the CLIP model's semantic recognition capability for wind turbine blade faults.

6. According to claim 5, the output of the TIP-Adapter model is the classification probability distribution of wind turbine blades under different fault categories, which is used for fault diagnosis and maintenance decision-making.

7. According to claim 1, the TIP-Adapter model uses CLIP ViT-B / 32 as the basic vision-language model, with an image feature dimension of 512, and supports small sample learning scenarios with 1-16 labeled images per class.

8. The intelligent diagnosis method for small-sample wind turbine blade faults based on TIP-Adapter according to claim 7, characterized in that, The method described is a plug-and-play approach that requires no training. Model adaptation only requires building a feature cache, which reduces annotation costs by approximately [amount missing] compared to traditional deep learning methods and saves training time. It is suitable for rapid deployment scenarios in industrial settings.

Images