Construction method of field special text prompt template library for fan blade fault detection

Abstract

The invention relates to a method for constructing a text prompt template library special for the field of wind turbine blade fault detection. According to the method, text prompt templates are systematically designed from three dimensions of professional description, damage features and detection scenes, and the text prompt templates comprise fault special templates, universal templates, technical templates and context templates. All the templates contain field qualifiers, and the universal semantics are implemented in a wind power blade detection scene. According to the method, differential expression design is adopted, a plurality of templates of the same fault category are different in sentence pattern structure and description angle, and the robustness of feature representation is enhanced. Compared with the use of general text prompt, the domain-specific template library constructed by the method can improve the zero sample classification accuracy of the vision-language model, supports modular extension, and can be combined with various vision-language model adaptation methods for use.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and natural language processing, and in particular to a method for industrial equipment fault detection using a vision-language pre-trained model. Specifically, this invention provides a method for constructing a domain-specific text prompt template library for wind turbine blade fault detection, in order to improve the fault identification accuracy in zero-shot and few-shot learning scenarios. Background Technology

[0002] Wind turbines, as crucial equipment for clean energy, are prone to various blade failures during long-term operation, including cracks, erosion, coating peeling, and material spalling. Timely and accurate detection of these failures is essential for preventing energy loss and ensuring operational safety. Traditional failure detection methods mainly rely on manual inspection and self-supervised learning based on convolutional neural networks (CNNs), but these methods have several limitations. High data acquisition costs are a significant issue; wind turbine blade failure datasets are limited in size, and labeled samples for certain failure types are extremely scarce, making it difficult to meet the large-scale labeled data requirements of deep learning models. Furthermore, traditional CNN models have poor adaptability to new failure types and different shooting scenarios, making it difficult to achieve zero-shot or small-shot failure identification, which limits their generalization ability. In new application scenarios, data re-labeling and model fine-tuning are also required, which is costly and time-consuming, significantly reducing the efficiency of transfer learning. In recent years, the emergence of vision-language pre-trained models (such as CLIP) has provided a new solution for zero-shot learning. CLIP learns a general vision-language feature space through pre-training on large-scale image-text pairs, exhibiting excellent zero-shot generalization ability. However, applying general pre-trained models directly to specific industrial fields (such as wind turbine blade fault detection) still presents adaptation challenges. Using generic text prompts (such as "a photo of a crack") fails to fully leverage domain knowledge and cannot accurately capture industry-specific fault characteristics. Single or limited text descriptions cannot encompass the diversity of fault manifestations, easily leading to one-sided feature representations. More critically, existing literature lacks systematic methods and best practices for constructing text prompt libraries specifically for industrial fault detection, leaving significant room for in-depth research into domain adaptation. Summary of the Invention

[0003] The purpose of this invention is to provide a method for constructing a domain-specific text prompt template library system for wind turbine blade fault detection. By designing text templates in a multi-dimensional and multi-level manner, the advantages of vision-language pre-trained models are fully utilized to improve the accuracy of zero-shot and few-shot fault identification, reduce annotation costs, and achieve efficient transfer applications.

[0004] The method provided by this invention includes the following core steps:

[0005] Step 1: Fault Type and Visual Feature Analysis

[0006] First, the main failure modes and their visual characteristics of wind turbine blades are systematically analyzed. By reviewing industry standards, failure case databases, and field testing experience, a set of failure categories to be identified is determined. Based on the International Wind Energy Council (IEC) standards and industry practices, this method focuses on the following five types of failures: Crack – Linear fractures occurring on or within the blade material, typically caused by fatigue stress, impact, or manufacturing defects; Erosion – Wear on the leading edge surface of the blade due to prolonged contact with raindrops, ice particles, or dust, resulting in a rough, pitted surface morphology; Paint Peeling – The protective coating peeling off the blade surface, exposing the underlying material and affecting waterproofing and corrosion resistance; Material Delamination – Delamination between layers within the composite material, typically manifesting as surface protrusions or depressions; Normal – A reference sample of a blade without obvious defects.

[0007] Step 2: Design a dedicated text template for troubleshooting

[0008] Based on the fault categories determined in Step 1, text prompt templates are systematically designed for each fault category from three dimensions. Professional expressions are extracted from technical reports, inspection manuals, and expert descriptions, including the definition, causes, and impacts of the fault. For example, for crack faults, the template can include various English expressions, each describing the characteristics of the crack from a different perspective, such as "A turbine blade with visible cracks in the surface" or "Fractures on the rotor blade composite material." The visual morphology of the fault is described, including its appearance, degree of damage, and location. For example, for crack faults, expressions such as "Blade surface showing surface-level crack damage" or "Deep fissures affecting the structural integrity of the turbine blade" can be used, emphasizing the physical damage caused by the fault. The actual inspection environment is considered, including contextual information such as the imaging method, lighting conditions, and distance. For example, "Close-up view of blade edge with visible crack" or "UAV inspection image showing blade surface cracks".

[0009] Step 3: Building a Parametric Generic Template Framework

[0010] To improve the maintainability and scalability of the template library, a parameterized general template framework is designed. A placeholder mechanism is used to allow the same template architecture to be applied to different fault categories, enabling template reuse. The general template framework includes four basic templates: Basic description templates use the format "{domain term} with {fault type}", for example, combining "turbineblade" and "crack" into "turbine blade with crack". Its purpose is to directly and clearly describe the fault type; Damage state templates use the format "{domain term} showing {fault type} damage", for example, "rotorblade showing erosion damage". Its purpose is to emphasize the physical damage state caused by the fault; Detection result templates use the format "{fault type} defect detected on {fault location}", for example, "crack defect detected on blade edge". Its purpose is to express the discovery and location of the detection and diagnosis; Impact description templates use the format "{domain term} affected by {fault type}", for example, "turbine blade affected by material delamination". Its purpose is to describe the impact of the fault on equipment performance and safety.

[0011] The placeholders are explained as follows: domain terms include "turbine blade", "rotor blade", "windturbine blade", etc.; fault types include "crack", "erosion", "paint peeling", "delamination", etc.; fault locations include "blade edge", "blade root", "blade tip", "blade surface", etc.

[0012] Step 4: Multi-level supplementary template design

[0013] Building upon the fault-specific templates, two supplementary templates were designed, employing professional inspection and diagnostic terminology, including: inspection method terms such as "visual inspection," "NDT inspection," and "ultrasonic inspection"; diagnostic terms such as "defect," "fault," "anomaly," and "damage"; engineering terms such as "surface condition," "structural integrity," and "load capacity"; and fault severity terms such as "incipient defect," "moderate damage," and "critical failure." These terminology templates enhance the professionalism and academic rigor of the descriptions, better aligning text features with domain knowledge.

[0014] The detection scenario context template contains information about the actual detection environment, including: detection equipment such as "UAVinspection", "visual survey", and "drone inspection"; shooting angle such as "close-up view", "wide view", "side view", and "top view"; surface conditions such as "wet surface", "dusty surface", and "clean surface"; and distance information such as "surface-level", "near-field", and "far-field".

[0015] Step 5: Text Feature Encoding and Alignment

[0016] A text encoder using a vision-language pre-trained model (taking CLIP as an example in this method) encodes features of all text in the constructed template library. After encoding, the template library is transformed from a discrete set of natural language texts into a continuous set of high-dimensional feature vectors, providing a unified mathematical representation for subsequent similarity calculations and classification.

[0017] Step 6: Template Feature Library Storage and Application

[0018] The text feature vector encoded in step 5 and its corresponding template metadata (including fault type, template category, original text, etc.) are stored as a template feature library. The storage method adopts a structured format, recording the number of each template, its fault type, template category, original text content, corresponding feature vector, and vector dimension. Attached Figure Description

[0019] Appendix Figure 1 : Flowchart of the method of this invention.

[0020] Appendix Figure 2 : Schematic diagram of the parametric template framework.

[0021] Appendix Figure 3 Multi-level template system diagram.

[0022] Appendix Figure 4 Fault classification application flowchart. Detailed Implementation

[0023] Example 1: Template Design for Wind Turbine Blade Crack Failure

[0024] Based on the above method, taking wind turbine blade cracks as an example, the specific implementation steps are explained in detail.

[0025] Step 1: Fault type confirmation

[0026] Cracks were identified as the target failure type in this embodiment. Through industry standards (IEC 61400 series) and failure case analysis, the causes of cracks were identified as fatigue, impact, manufacturing defects, etc.; visual characteristics include linear fractures and dark lines; and the degree of harm includes affecting blade strength and lifespan.

[0027] Step 2: Design of a dedicated template for troubleshooting

[0028] For crack faults, design no fewer than eight differentiated English text description templates. In terms of professional descriptions, phrases such as "A turbine blade with visible cracks in the surface," "Wind turbine rotor blade containing fractures," and "Fissures observed within the composite blade material" can be designed. In terms of damage characteristics, phrases such as "Blade surface showing surface-level crack damage," "Deep linear fractures affecting the blade structure," and "Multiple fissures distributed across the blade surface" can be designed. In terms of inspection scenarios, phrases such as "Close-up view of wind turbine blade edge with cracks" and "UAV inspection image revealing structural fissures on the rotor blade" can be designed.

[0029] Step 3: Application of the parameterized framework

[0030] Based on the parameterized framework, four types of templates for crack failure can be generated through placeholder replacement. The basic description template generates "turbine blade with crack" by combining "turbine blade" and "crack"; the damage state template generates "blade showing crack damage" by combining "blade" and "crack damage"; the detection result template generates "crack defect detected on blade edge" by combining "crack defect detected on" and "blade edge"; and the impact description template generates "turbine blade affected by crack" by combining "turbine blade affected by" and "crack".

[0031] Step 4: Multi-level supplementary template

[0032] Based on the above templates, supplement them with technical terminology templates, such as "Structural integrity affected by surface cracks" and "Critical defect in the blade composite material"; supplement them with scenario templates, such as "Surface-level cracks detected in close-up inspection" and "Bladecracks visible in drone monitoring footage".

[0033] Step 5: Feature Encoding

[0034] All crack-related templates are encoded using a text encoder based on a vision-language pre-trained model. Each text input is converted into a 512-dimensional (or higher) feature vector, which represents the semantic information of the text in the feature space.

[0035] Step 6: Feature library storage

[0036] The encoded feature vectors and their corresponding metadata are stored. The stored content includes a unique ID for each template, the fault type (crack), the template classification (domain-specific, technical terminology, scenario template, etc.), the original English text, the corresponding feature vector values, and the dimension information of the vectors.

[0037] Application Demonstration

[0038] For a new blade detection image, the image is first encoded to obtain its feature vector; then the cosine similarity between the feature vector and all crack-related templates in the template feature library is calculated; finally, the similarity values ​​are counted. If most similarity values ​​are high (e.g., all are above 0.8), it is determined that the blade corresponding to the image has a crack fault, and a confidence score is given.

[0039] Example 2: Dynamic Expansion of the Template Library

[0040] Suppose we need to add a new fault category, "paint fading," to the template library.

[0041] Design new category templates—Following the design principles described in claims 1-4, independently design fault-specific templates, technical terminology templates, and scenario templates for paint fading. For example, fault-specific templates may include "Turbine blade with color fading" and "Wind turbine rotor blade showing paint discoloration," etc.; technical terminology templates may include "Surface appearance defect caused by UV exposure," etc.; scenario templates may include "Long-term weathering effects visible on blade surface," etc. Use a parametric framework to automatically generate four types of templates. Taking the basic description template as an example, by combining "turbine blade" and "color fading," "turbine blade with color fading" is generated; taking the damage state template as an example, "blade showing color fading issue" is generated; taking the detection result template as an example, "color fading observed on blade surface" is generated; and taking the impact description template as an example, "turbineblade affected by color fading" is generated. Text encoding only requires re-performing the text encoding operation. The text encoder of the vision-language model encodes the features of all newly added paint fading-related text templates, without requiring modification or retraining of the vision-language model itself. The newly added feature vectors and their meta-information are appended to the existing template feature library. The new content includes all text feature vectors, template numbers, and text content related to paint fading faults, and is managed uniformly with the existing template libraries for the five fault categories (cracks, erosion, coating peeling, material peeling, and normal state). Dynamic management of the template library content is supported through structured configuration files (such as YAML or other plain text formats). For example, the configuration file can list all supported fault categories, including the original five and the newly added paint fading, facilitating automatic system identification and loading of the corresponding templates. Through this mechanism, the entire expansion process does not involve model modification or retraining, significantly reducing system maintenance costs. New fault categories can be integrated into the system within days, representing a significant efficiency improvement compared to traditional methods (which require 1-3 months of data annotation and model training).

Claims

1. A method for constructing a domain-specific text prompt template library for wind turbine blade fault detection, characterized in that, Includes the following steps: Analyze the failure modes and visual characteristics of wind turbine blades to determine the set of failure categories, including cracks, erosion, coating peeling, material peeling, and normal condition; For each type of fault, a dedicated text prompt template for faults is designed from three dimensions: professional description, damage characteristics, and detection scenario. Each type of fault has no fewer than 8 differentiated English description texts. Design a parameterizable general template framework and use a placeholder mechanism to make the same template architecture applicable to different fault categories; design technical terminology templates and detection scenario context templates to form a multi-level semantic description system. A text encoder using a vision-language pre-trained model encodes the features of text in a template library, mapping natural language text to a high-dimensional feature vector space to form a set of text feature vectors aligned with the image feature space. The text feature vectors and their corresponding template metadata are stored as a template feature library for image-text similarity matching and zero-shot fault classification.

2. The method according to claim 1, characterized in that, The fault-specific text templates are designed with domain-specific features, with each template containing domain-specific terms such as "wind turbine blade," "turbine blade," or "rotor blade," anchoring general semantics to the wind turbine blade inspection environment; they also have lexical variation features, with each template containing fault type keywords and their synonyms or near-synonyms in diverse expressions, such as "fracture," "fissure," and "split" for the "crack" fault type; and they exhibit structural differences, with multiple templates for the same fault category differing in sentence structure, grammatical form, and descriptive perspective.

3. The method according to claim 1 or 2, characterized in that, The parameterized general template framework includes: a basic description template "{domain_term} with {fault_type}", a damage state template "{domain_term} showing {fault_type} damage", a detection result template "{fault_type} defect detected on {target_location}", and an impact description template "{domain_term} affected by {fault_type}"; where {domain_term} represents a domain qualifier placeholder, {fault_type} represents a fault type placeholder, and {target_location} represents a fault location placeholder.

4. The method according to any one of claims 1 to 3, characterized in that, The technical terminology template uses professional terms such as "inspection", "defect", "fault", "condition", "diagnostic", "surface condition" or "structural integrity"; the inspection scenario template includes inspection scenario information such as "UAV inspection", "close-up view" or "surface-level"; the fault-specific template, technical terminology template and scenario template form a progressive and complementary relationship.

5. The method according to any one of claims 1 to 4, characterized in that, The dynamic expansion of the template library includes: when expanding to new fault categories, designing corresponding templates for the new categories according to the design principles described in claims 1 to 4; adding new templates by feature encoding through the text encoder of the visual-language model and appending them to the template feature library without modifying the model structure or retraining; and dynamically loading, updating and managing template content through configuration files in JSON or YAML format.

6. The method according to any one of claims 1 to 5, characterized in that, The text feature encoding uses a text encoder of a vision-language pre-trained model to encode each text template into a fixed-dimensional vector representation; the dimension of the encoded text feature vector is the same as the output dimension of the image encoder, ensuring alignment of the image-text feature space; the text encoder is a frozen module and does not participate in the subsequent training process.

7. The method according to any one of claims 1 to 6, characterized in that, The template library is not dependent on a specific vision-language model and can be used in conjunction with various models such as CLIP, TIP-Adapter, CoOp, or CLIP-Adapter. This method is suitable for fault detection of wind turbine blades and can also be applied to fault detection tasks of other industrial equipment such as solar panels, power equipment, or mechanical parts. This method is independent of hardware deployment architecture and inference framework and supports application in various computing environments such as CPU, GPU, or edge computing devices.

Images