Self-adaptive text-guided fan blade defect image super-resolution reconstruction method

Abstract

The invention provides a fan blade defect image super-resolution reconstruction method guided by a self-adaptive text, and relates to the technical field of fan blade polling, and the method comprises the following steps: preprocessing a fan blade image, obtaining a high-quality blade sub-image, constructing a multi-modal prior knowledge base, and generating a global style cue word; defining a defect type, training a defect detection model, and generating a local content cue word; carrying out fusion according to a global-local-quality semantic hierarchy sequence, and constructing forward and reverse cue words; dynamically optimizing the potential features; and constructing a multi-dimensional evaluation system and combining a causal reasoning analysis framework. Through a self-adaptive text guidance mechanism, accurate enhancement of fan blade defect features is realized in a super-resolution reconstruction process, and the visual identification degree and the detection precision of a tiny defect area are remarkably improved on the premise of keeping the consistency of the overall structure of an image. And a high-fidelity and task-adaptive image reconstruction solution is provided for intelligent inspection of the fan blade.

Description

Technical Field

[0001] This invention relates to the field of wind turbine blade inspection technology, and in particular to an adaptive text-guided super-resolution reconstruction method for wind turbine blade defect images. Background Technology

[0002] Driven by the global "dual-carbon" strategy, wind power, as an important component of clean energy, has seen continuous growth in installed capacity, with wind turbines exhibiting trends towards larger sizes and more complex deployment areas. As a key component of wind turbines, the structural health of the turbine blades directly affects the safety and power generation efficiency of the entire unit. Under harsh conditions such as high speed and high corrosion, the blade surface is prone to damage such as cracks, erosion, and delamination. Traditional manual inspection methods suffer from low efficiency, high cost, and significant safety risks, making it difficult to meet the demands of the modern wind power industry for intelligent operation and maintenance.

[0003] Currently, using drones for automated inspection of wind turbine blades has become a mainstream technology. However, in practical engineering applications, drone inspection systems still face significant challenges: limited by factors such as shooting distance, complex lighting conditions, weather conditions, and image compression and storage, the acquired blade images often suffer from quality issues such as insufficient resolution, blurred edges, and missing details. These low-quality images severely restrict the performance of subsequent computer vision-based defect detection algorithms, especially in identifying minute defects such as cracks and paint peeling, leading to high rates of missed and false detections, and failing to meet the practical needs of high-precision, high-reliability intelligent inspection.

[0004] To improve image quality, existing technologies often employ Real-ISR (Real Image Super-Resolution) methods based on Generative Adversarial Networks (GANs), such as SRGAN, ESRGAN, and Real-ESRGAN. These methods enhance the model's generalization ability to some extent by introducing perceptual loss and adversarial training mechanisms, or by simulating complex degradation processes to generate synthetic data. However, GANs inherently suffer from training instability, and their generated results are prone to artifacts and structural distortions. When faced with severely degraded regions, they struggle to effectively recover semantic information that conforms to the true physical structure.

[0005] In recent years, diffusion models have gradually become the mainstream paradigm for generative vision tasks due to their excellent detail generation capabilities and training stability, and have shown great potential in image super-resolution tasks. Researchers have successively proposed a series of advanced methods based on diffusion priors, such as StableSR, ResShift, SUPIR, SeeSR, and DiffBIR. These methods have achieved significant results in the reconstruction fidelity and perceptual quality of general natural images by introducing the powerful generative priors of pre-trained diffusion models or by innovating in areas such as efficient sampling and multimodal guidance. However, these diffusion models for general scenarios still have obvious limitations when directly applied to the specific industrial inspection scenario of wind turbine blades: First, they lack specific constraints on the macroscopic structure and texture style of wind turbine blades, which can easily lead to structural distortion or style inconsistency during reconstruction; second, they fail to effectively integrate prior knowledge of defect regions, resulting in insufficient detail restoration capabilities for minor defects and failing to significantly improve defect identifiability; third, existing guidance control mechanisms (such as fixed prompts) lack fine-grained, task-oriented adaptive capabilities, making it difficult to achieve the optimal balance between overall structural fidelity and local defect enhancement.

[0006] Therefore, there is an urgent need in this field for a novel super-resolution reconstruction method that can deeply integrate knowledge of wind turbine blades and has fine-grained semantic guidance and adaptive control capabilities, so as to accurately solve the key problems of low resolution and blurred details in UAV inspection images in defect detection tasks.

[0007] Current research on super-resolution reconstruction methods for wind turbine blade images is insufficient and suffers from the following problems:

[0008] ① Existing super-resolution methods based on diffusion models lack domain knowledge guidance. General text prompts are difficult to accurately describe the macroscopic structural features and microscopic defect morphology of wind turbine blades, resulting in deficiencies in the reconstruction results in terms of structural consistency and defect authenticity.

[0009] ② The prior knowledge introduction method based on retrieval is inefficient and lacks an efficient cross-modal retrieval mechanism for the characteristics of wind turbine blade images, making it difficult to meet the real-time processing needs in actual inspections while ensuring semantic relevance.

[0010] ③ The prompt word fusion mechanism is relatively fixed and lacks the adaptive ability to dynamically adjust the global style and local detail weights based on image content, making it unable to accurately adapt the reconstruction strategy when faced with defects of different sizes and types. Summary of the Invention

[0011] The purpose of this invention is to address the shortcomings and deficiencies in existing super-resolution reconstruction methods for wind turbine blade images.

[0012] To achieve the above objectives, the present invention adopts the following technical solution:

[0013] An adaptive text-guided super-resolution reconstruction method for wind turbine blade defect images, characterized by comprising the following steps:

[0014] S1: Image preprocessing to obtain high-quality leaf sub-images covering typical defect scenarios;

[0015] S2: Utilize the high-quality blade subgraphs in S1 to construct a multimodal prior knowledge base for wind turbine blades;

[0016] S3: A two-stage retrieval strategy guided by sparse features and ordered by CLIP semantics generates global style cue words that characterize the macroscopic structure and texture features of the image.

[0017] S4: Define various defect types for wind turbine blades and train a blade defect detection model;

[0018] S5: Locate the defect area on the blade surface and generate local content prompts describing the defect attributes and spatial distribution based on the visual language big model;

[0019] S6: Construct a multi-level adaptive text prompt generator, integrate global style prompts, local content prompts and preset image quality prompts, fuse them in the semantic hierarchy of "global-local-quality", dynamically adjust the guidance intensity of each prompt through an adaptive weight allocation mechanism, and construct composite positive prompts and reverse prompts to suppress redundant information for diffusion model inference.

[0020] S7: In the iterative denoising process of the diffusion model, the latent features are dynamically optimized by combining bidirectional conditional guidance to achieve a balance between low-frequency structural stability and high-frequency defect detail enhancement, and output a high-resolution reconstructed image.

[0021] S8: Establish a multi-dimensional evaluation system and causal reasoning analysis framework to assess the effectiveness and robustness of the method in defect identification and detail recovery.

[0022] Preferably, the steps in S6 are as follows:

[0023] S6-1: Construct a multi-level adaptive text cue generator, where image quality cue words include positive quality cue words and negative quality cue words;

[0024] S6-2: When there are obvious defective areas in the image, increase the weight of local content cues; when there are few or no defects, increase the weight of global style cues.

[0025] S6-3: The weighted and fused prompt word sequence is constructed into a composite positive prompt word, and the preset negative quality prompt word is used as a negative prompt word to suppress redundant information, together forming a two-way conditional guidance signal.

[0026] Preferably, the specific steps of S1 are as follows:

[0027] S1-1: Acquire high-definition raw image data under different wind power scenarios;

[0028] S1-2: Use the YOLO leaf segmentation model to extract the foreground from the original image, generate a leaf region mask, and determine the cropping candidate region based on the mask shape;

[0029] S1-3: An adaptive trimming method is used to segment the mask area so that the trimming frame size meets the preset requirements and maximizes the coverage of the effective area of ​​the blade. After trimming and quality screening, high-quality wind turbine blade sub-images covering typical defect scenarios such as cracks, paint peeling, and surface contamination are obtained.

[0030] Preferably, step S2 is as follows:

[0031] S2-1: Using the acquired high-quality wind turbine blade sub-graphs, construct a multimodal prior knowledge base containing image-text pairs and their feature indices;

[0032] S2-2: The knowledge base construction process consists of a text generation module and an index construction module. First, the text generation module processes the input image through normalization and tensor quantization. This image, along with prompts, is then input into the multimodal Transformer framework. The model generates semantic representations highly correlated with the overall color tone, texture distribution, and macroscopic style attributes through a cross-modal alignment mechanism. These representations are then decoded to obtain a global description in natural language, ensuring that the generated results highlight global style information.

[0033] S2-3: After text generation is complete, the index building module is entered to construct structural and semantic feature indexes:

[0034] In the index construction stage, a self-trained sparse autoencoder is first used to compress the features of the input image, extracting low-dimensional sparse feature vectors that preserve the overall geometric structure and global texture distribution.

[0035] Subsequently, for each image-text pair, the pre-trained CLIP model is used to extract image features and text features respectively, and a semantic embedding representation with uniform dimensions is obtained through normalization processing.

[0036] Finally, an index mapping is established between image-text pairs and their corresponding structural and semantic features, and efficient vector retrieval is achieved through FAISS. The first stage completes structural recall based on sparse features, and the second stage completes semantic ranking based on CLIP features, thereby constructing a logically consistent and scalable multimodal prior knowledge base.

[0037] Preferably, step S3 is as follows:

[0038] Based on a multimodal prior knowledge base, a two-stage cascaded retrieval is performed;

[0039] First, in the first-stage structural-level recall, the input low-resolution wind turbine blade image extracts structural feature vectors through a self-trained sparse autoencoder, performs similarity retrieval in the FAISS sparse feature index, and selects the top-10 candidate samples with the highest similarity and their text descriptions.

[0040] Subsequently, in the second-stage semantic ranking, the CLIP cross-modal visual-language alignment model is used to extract semantic features of the input image and candidate texts. Cross-modal semantic alignment is achieved through cosine similarity calculation, and the top-3 text descriptions with the highest semantic matching degree are selected.

[0041] Finally, a weighted fusion strategy is used to generate global style cues that represent the macroscopic structure, texture distribution, and overall visual style of the input image.

[0042] Preferably, step S4 includes the following steps:

[0043] S4-1: Based on the typical damage modes of wind turbine blades and the distribution of key components in the actual operating environment, clarify the target category system of the detection model. The target categories include defect categories and device categories.

[0044] S4-2: Based on the high-quality wind turbine blade sub-images that have been preprocessed and screened, a training set in the format of a rotating frame is constructed using a manual fine annotation method, and data augmentation strategies are introduced for the training samples.

[0045] S4-3: Supervised training of the YOLOv8-OBB rotating object detection model is performed using the training set, and the model performance is systematically evaluated based on the independent validation set. Optimization strategies are introduced during the training process.

[0046] Preferably, step S5 is as follows:

[0047] S5-1: Use the trained YOLOv8-OBB model to detect the input image, crop each detected defect region, and input it into the visual language large model LLaVA. Generate a single-sentence technical description containing defect type, morphological features and location information through the preset prompt word template.

[0048] S5-2: When there are multiple defects in the same image, a description synthesis strategy is used to merge multiple single-sentence descriptions into a unified local content prompt word;

[0049] S5-3: When no defects are detected, call LLaVA to generate state descriptions for local regions of the image, ensuring that each input image corresponds to a valid local content prompt.

[0050] Preferably, step S7 is as follows:

[0051] S7-1: Encode the low-resolution input image into the latent space using a pre-trained variational autoencoder to obtain the initial latent feature representation. The encoding process is represented as follows:

[0052] (8)

[0053] S7-2: During the T-step iterative denoising process, the composite positive cue word and the negative cue word are input into the IRControlNet conditional control module to adjust the semantic constraints and quality suppression effects in each denoising step.

[0054] S7-3: Dynamic optimization is implemented through a bidirectional classifier free guidance mechanism: the forward branch uses compound positive cue words to enhance the semantic matching with the target features, and the reverse branch uses reverse quality cue words to suppress artifacts and the generation of non-target content;

[0055] S7-4: In each diffusion sampling step, the balance weights of structural fidelity and detail enhancement are adaptively adjusted according to the current noise level to achieve stable recovery of the low-frequency region and fine restoration of high-frequency defect features. Its dynamic balance coefficient can be expressed by equation (10):

[0056] (10)

[0057] In the formula, For the spread of the first The structural fidelity weight of the step is used to preserve low-frequency structural information such as blade profile and main ridge line; For the spread of the first The detail enhancement weight is used to strengthen high-frequency defect features such as cracks, paint peeling, and corrosion textures; This represents the current diffusion step number; This represents the total number of diffusion steps.

[0058] S7-5: The optimized latent features are reconstructed into pixel space through the decoder, outputting the final high-resolution reconstructed image.

[0059] (11).

[0060] Preferably, step S8:

[0061] S8-1: To assess the realism, structural fidelity, and defect identifiability of super-resolution reconstruction results, establish a multi-dimensional evaluation system covering subjective, objective, and task performance aspects, including the following:

[0062] a. Subjective perception evaluation based on expert ratings, used to measure the naturalness, texture reproduction and overall realism of an image;

[0063] b. Objective quantitative evaluation based on PIQE, BRISQUE, NIQE, and CLIP-IQA without reference metrics, used to measure the structural consistency and perceptual quality of reconstructed images. The mathematical expressions for PIQE, BRISQUE, and NIQE are shown below:

[0064] (12)

[0065] (13)

[0066] (14)

[0067] Wherein, BM is the blockiness index, used to evaluate the blocky structure in an image caused by compression artifacts; NM is the noise index, used to measure the noise level caused by image loss or transmission errors. This is a statistical feature vector of a natural scene obtained by statistical fitting of the MSCN coefficients and their products. This represents a pre-trained support vector regression model used to map feature vectors to the final quality score, where , and , These are the mean vector and covariance matrix of the multivariate Gaussian model for the natural image and the multivariate Gaussian model for the distorted image, respectively.

[0068] c. Performance evaluation of the YOLO-based defect detection model, used to measure the improvement effect of reconstructed images on the accuracy of defect identification.

[0069] S8-2: Treat the prompting and guidance strategy as an intervention variable, construct a causal graph framework between the prompting strategy, the super-resolution model and the realism of the reconstructed image, and clarify the causal transmission path between variables;

[0070] S8-3: In the actual experiment, the test image and different prompting guidance strategies are input into the super-resolution model, and the model output results are evaluated by experts' subjective evaluation, objective index calculation and defect detection performance assessment respectively.

[0071] S8-4: In counterfactual experiments, keep the model structure, inference process and data input consistent, remove only the prompting and guidance strategy, so that the super-resolution model only relies on low-resolution images for reconstruction, and perform multi-dimensional evaluation on the output results.

[0072] S8-5: By comparing the differences between factual and counterfactual experiments, the causal enhancement effect of prompting and guidance strategies on image authenticity across various dimensions is quantified.

[0073] Compared with the prior art, this application has the following beneficial effects:

[0074] This invention achieves a technological breakthrough from general enhancement to semantically guided reconstruction by constructing a multi-level adaptive text prompt generator that deeply integrates domain knowledge of wind turbine blades into the reconstruction process. It innovatively proposes an adaptive weight allocation mechanism based on defect detection, overcoming the limitations of traditional fixed-weight fusion and achieving a dynamic balance between structure preservation and detail enhancement. Furthermore, by constructing a multimodal prior knowledge base and a two-stage retrieval strategy, it establishes an efficient channel for utilizing domain knowledge. Experiments show that this method significantly enhances the accuracy of identifying minute defects while improving image visual quality, providing a practical technical solution for intelligent inspection of wind turbine blades. Attached Figure Description

[0075] Figure 1 This is a flowchart of an adaptive text-guided super-resolution reconstruction method for wind turbine blade defect images according to an embodiment of the present invention.

[0076] Figure 2 This is a schematic diagram of the algorithm flow of the adaptive text-guided super-resolution reconstruction method for wind turbine blade defect images according to one embodiment of the present invention.

[0077] Figure 3 This is an example of a dataset obtained through standardization, cropping, and quality screening in one embodiment of the present invention.

[0078] Figure 4 This is a schematic diagram illustrating the construction process of a multimodal prior knowledge base for wind turbine blade images in one embodiment of the present invention.

[0079] Figure 5 This is a YOLO annotation category for wind turbine blade images in one embodiment of the present invention.

[0080] Figure 6 This is a causal reasoning framework diagram for adaptive text guidance strategy analysis in one embodiment of the present invention.

[0081] Figure 7 This is a schematic diagram comparing the results of factual and counterfactual experiments without reference indicators in one embodiment of the present invention.

[0082] Figure 8This is a schematic diagram comparing test results on wind turbine images in different scenarios according to one embodiment of the present invention.

[0083] Figure 9 This is a schematic diagram comparing the enhanced image and the original image obtained by YOLO detection using the adaptive text-guided super-resolution reconstruction method for wind turbine blade defect images in one embodiment of the present invention. Detailed Implementation

[0084] The present invention will be further described in detail below with reference to specific embodiments.

[0085] Please see Figure 1 and Figure 2 This application provides an adaptive text-guided super-resolution reconstruction method for wind turbine blade defect images, which includes the following steps:

[0086] S1: Preprocessing of wind turbine blade images captured by the drone through standardized cropping and quality screening, resulting in images such as... Figure 3 As shown.

[0087] In one embodiment, step S1 is as follows:

[0088] S1-1: Acquire high-definition raw image data under different wind power scenarios, including static shutdown close-range data and dynamic operation and inspection data;

[0089] S1-2: The foreground of the original image is extracted using a self-trained YOLO leaf segmentation model to generate a leaf region mask, and the cropping candidate region is determined based on the shape of the mask.

[0090] S1-3: An adaptive trimming method is used to segment the mask area so that the trimming frame size meets the preset requirements and maximizes the coverage of the effective area of ​​the blade. After trimming and quality screening, high-quality wind turbine blade sub-images covering typical defect scenarios such as cracks, paint peeling, and surface contamination are obtained.

[0091] S2: Using the high-quality blade subgraphs obtained from preprocessing, construct a multimodal prior knowledge base for wind turbine blades that includes images, text, and structural features.

[0092] In one implementation, please refer to Figure 4 , Figure 4 The paper demonstrates the pre-processing steps for generating global style cue words based on a multimodal prior knowledge base. The construction steps of the multimodal prior knowledge base for wind turbine blades are as follows:

[0093] S2-1: Using the acquired high-quality wind turbine blade sub-graphs, construct a multimodal prior knowledge base containing image-text pairs and their feature indices.

[0094] S2-2: The knowledge base construction process consists of a text generation module and an index construction module, with the text generation module proceeding first. The input image, after normalization and tensor quantization, is input along with prompts into the multimodal Transformer framework. The model generates semantic representations highly correlated with the overall color tone, texture distribution, and macroscopic style attributes through a cross-modal alignment mechanism. These representations are then decoded to obtain a global description in natural language form, ensuring that the generated results highlight global style information. The generation process can be formalized as a conditional probability expansion, calculated as follows:

[0095] (1)

[0096] In the formula, This indicates the input of a wind turbine blade image. Constraints for prompt words, This is the generated semantic description sequence.

[0097] S2-3: After the text is generated, the index building module is entered to build structural and semantic feature indexes.

[0098] In the index building stage, the input image is first compressed using a self-trained sparse autoencoder to extract low-dimensional sparse feature vectors that can preserve the overall geometric structure and global texture distribution, providing high-quality feature representations for subsequent structure-level recall.

[0099] Subsequently, for each image-text pair, a pre-trained CLIP model is used to extract image features and text features respectively, and a semantic embedding representation with unified dimensions is obtained through normalization processing, thereby ensuring consistent representation of cross-modal data.

[0100] Finally, an index mapping is established between image-text pairs and their corresponding structural and semantic features, and efficient vector retrieval is achieved through FAISS. The first stage completes structural recall based on sparse features, and the second stage completes semantic ranking based on CLIP features, thereby constructing a logically consistent and scalable multimodal prior knowledge base.

[0101] The calculation formula for the index building phase is as follows:

[0102] (2)

[0103] (3)

[0104] (4)

[0105] In the formula, ,in , and It depicts the distribution characteristics of images and text in semantic space. This represents the multimodal prior knowledge base for wind turbine blades.

[0106] S3: A two-stage retrieval strategy guided by sparse features and CLIP semantic ranking is used to generate global style cue words that characterize the macroscopic structure and texture features of the image.

[0107] In one embodiment, the global style cue word generation steps are as follows: based on a multimodal prior knowledge base, a two-stage cascaded retrieval is performed to generate global style cue words.

[0108] First, in the first-stage structural-level recall, the input low-resolution wind turbine blade image extracts structural feature vectors through a self-trained sparse autoencoder, performs similarity retrieval in the FAISS sparse feature index, and selects the top-10 candidate samples with the highest similarity and their text descriptions.

[0109] Subsequently, in the second-stage semantic ranking, the CLIP cross-modal visual-language alignment model is used to extract semantic features of the input image and candidate texts. Cross-modal semantic alignment is achieved through cosine similarity calculation, and the top-3 text descriptions with the highest semantic matching degree are selected.

[0110] Finally, a weighted fusion strategy is used to generate global style cues that represent the macroscopic structure, texture distribution, and overall visual style of the input image. The calculation formula is as follows:

[0111] (5)

[0112] (6)

[0113] (7)

[0114] In the formula, For sparse feature space similarity scores This represents the similarity score between the image and the text in the semantic space. This indicates a global style hint.

[0115] S4: Combining common blade damage patterns in the wind power industry, define multiple defect types for wind turbine blades and train a blade defect detection model.

[0116] In one embodiment, S4 includes the following steps:

[0117] S4-1: Based on the typical damage modes of wind turbine blades and the distribution of key components in actual operating environments, define the target category system for the detection model. (Please refer to...) Figure 5 , Figure 5This illustrates the definition of defect categories in wind turbine blade images. The defect categories include six types of surface damage: surface contamination, paint peeling, cracks, gel coat peeling, fiberglass corrosion, and lightning strikes. The device categories include three types of structural components: lightning arresters, location markers, and power-enhancing components.

[0118] S4-2: Based on high-quality wind turbine blade sub-images that have undergone preprocessing and screening, a training set in rotated frame format is constructed using manual fine-grained annotation. To improve the model's adaptability to complex lighting, pose changes, and background noise, data augmentation strategies such as random rotation, brightness perturbation, local cropping, and noise injection are introduced into the training samples, thereby expanding the model's generalization ability under different inspection conditions.

[0119] S4-3: Supervised training of the YOLOv8-OBB rotating target detection model was performed using the training set, and the model performance was systematically evaluated based on the independent validation set. During training, optimization strategies such as cosine annealing learning rate scheduling, loss weight balancing, and class resampling were introduced to improve the model's detection sensitivity and recognition stability for small target defects. Through continuous training and validation, the final model met the requirements for engineering deployment in terms of key indicators such as mAP, recall, localization accuracy, and small target detection capability.

[0120] S5: Use the trained detection model to locate the defect area on the blade surface, and generate local content prompts describing the defect attributes and spatial distribution based on the visual language big data model.

[0121] In one embodiment, S5 includes the following steps:

[0122] S5-1: The trained YOLOv8-OBB model is used to detect the input image. Each detected defect region is cropped and input into the visual language large model LLaVA. A single-sentence technical description containing defect type, morphological features and location information is generated through preset prompt word templates.

[0123] S5-2: When there are multiple defects in the same image, a description synthesis strategy is used to merge multiple single-sentence descriptions into a unified local content prompt word.

[0124] S5-3: When no defects are detected, call LLaVA to generate state descriptions for local regions of the image, ensuring that each input image corresponds to a valid local content prompt.

[0125] S6: Construct a multi-level adaptive text prompt generator, which integrates global style prompts, local content prompts, and preset image quality prompts. The prompts are fused in a semantic hierarchy of "global-local-quality". The guiding strength of each prompt is dynamically adjusted through an adaptive weight allocation mechanism to construct composite positive prompts and negative prompts that suppress redundant information for diffusion model inference.

[0126] Please see Figure 6 , Figure 6 The document provides a causal reasoning framework diagram for the adaptive text guidance strategy analysis in the method of this application, used to describe the causal relationship between the adaptive text guidance strategy, the diffusion model, and the realism of the reconstructed image. In one embodiment, step S6 is as follows:

[0127] S6-1: Construct a multi-level adaptive text prompt generator that integrates three types of prompts according to the semantic hierarchy of "global-local-quality". The image quality prompts include positive quality prompts and negative quality prompts.

[0128] S6-2: Implement adaptive weight allocation based on defect detection results: When there are obvious defect areas in the image, increase the weight of local content prompts; when there are few defects or they are not visible, increase the weight of global style prompts.

[0129] S6-3: The weighted and fused prompt word sequence is constructed into a composite positive prompt word, and the preset negative quality prompt word is used as a negative prompt word to suppress redundant information, together forming a two-way conditional guidance signal.

[0130] S7: In the iterative denoising process of the diffusion model, the latent features are dynamically optimized by combining bidirectional conditional guidance to achieve a balance between low-frequency structural stability and high-frequency defect detail enhancement, and output a high-resolution reconstructed image.

[0131] In one embodiment, S7 includes the following steps:

[0132] S7-1: Encode the low-resolution input image into the latent space using a pre-trained variational autoencoder to obtain the initial latent feature representation. The encoding process is represented as follows:

[0133] (8)

[0134] S7-2: During the T-step iterative denoising process, the composite positive cue word and the negative cue word are input into the IRControlNet conditional control module to adjust the semantic constraints and quality suppression effects in each denoising step.

[0135] S7-3: Dynamic optimization is implemented through a bidirectional classifier free guidance mechanism: the forward branch uses compound positive cue words to enhance the semantic matching with the target features, and the reverse branch uses reverse quality cue words to suppress artifacts and the generation of non-target content.

[0136] S7-4: In each diffusion sampling step, the balance weights of structural fidelity and detail enhancement are adaptively adjusted according to the current noise level to achieve stable recovery of the low-frequency region and fine restoration of high-frequency defect features. Its dynamic balance coefficient can be expressed by equation (10):

[0137] (10)

[0138] In the formula, For the spread of the first The structural fidelity weight of the step is used to preserve low-frequency structural information such as blade profile and main ridge line; For the spread of the first The detail enhancement weight is used to strengthen high-frequency defect features such as cracks, paint peeling, and corrosion textures; This represents the current diffusion step number; This represents the total number of diffusion steps.

[0139] S7-5: The optimized latent features are reconstructed into pixel space through the decoder, and the final high-resolution reconstructed image is output.

[0140] (11)

[0141] S8: Establish a multi-dimensional evaluation system and causal reasoning analysis framework that includes subjective perception, objective evaluation, and detection performance to assess the effectiveness and robustness of the method in defect identification and detail recovery.

[0142] In one embodiment, S8 includes the following steps:

[0143] S8-1: Establish a multi-dimensional evaluation system covering subjective, objective, and task performance aspects to assess the realism, structural fidelity, and defect identification capability of super-resolution reconstruction results. This includes the following:

[0144] (1) Subjective perception evaluation based on expert ratings is used to measure the naturalness, texture reproduction and overall realism of the image.

[0145] (2) Objective quantitative evaluation based on non-reference metrics such as PIQE, BRISQUE, NIQE, and CLIP-IQA is used to measure the structural consistency and perceptual quality of reconstructed images. The mathematical expressions for PIQE, BRISQUE, and NIQE are shown below:

[0146] (12)

[0147] (13)

[0148] (14)

[0149] Among them, BM (Blocking Indicator) is used to evaluate the blocky structure in an image caused by compression artifacts; NM (Noise Indicator) is used to measure the noise level caused by image loss or transmission errors. This is a statistical feature vector of a natural scene obtained by statistical fitting of the MSCN coefficients and their products. This represents a pre-trained support vector regression model used to map feature vectors to final quality scores. , and , These are the mean vector and covariance matrix of the multivariate Gaussian model for the natural image and the multivariate Gaussian model for the distorted image, respectively.

[0150] (3) Task performance evaluation based on YOLO defect detection model, used to measure the improvement effect of reconstructed image on defect recognition accuracy, thereby reflecting the effectiveness of reconstruction results in actual inspection scenarios.

[0151] S8-2: Treat the prompting and guidance strategy as an intervention variable, construct a causal graph framework between the prompting strategy, the super-resolution model, and the realism of the reconstructed image, and clarify the causal transmission path between variables.

[0152] S8-3: In the field experiment, test images were input into the super-resolution model along with different prompting strategies, including global style prompts, local content prompts, and image quality prompts, as well as multi-level adaptive prompting after the fusion of the three types of prompts. The model output results were evaluated through expert subjective evaluation, objective index calculation, and defect detection performance assessment, evaluating the actual improvement effect of adaptive prompting on image realism from three dimensions.

[0153] S8-4: In the counterfactual experiment, maintain consistency between the model structure, inference process, and data input, only removing the cue-guided strategy, so that the super-resolution model relies solely on low-resolution images for reconstruction. Perform the same multi-dimensional evaluation on the output as in the factual experiment.

[0154] S8-5: By comparing the differences between factual and counterfactual experiments, the causal enhancement effect of the prompting strategy on image realism across various dimensions is quantified. The results verify that the multi-level adaptive prompt generator can significantly enhance the detail restoration, realism, and defect identifiability of reconstructed images, confirming the direct causal contribution of adaptive prompting.

[0155] In this application Figure 4 and Figure 5 This paper outlines the key construction process of the multi-level adaptive text prompt generator in the method provided in this application. Together, these two aspects constitute the core foundation of the prompt generation module Y.

[0156] Please see Figure 7-9 , Figure 7-9 This paper presents a multi-dimensional evaluation and analysis of the "Adaptive Text-Guided Super-Resolution Reconstruction of Wind Turbine Blade Defect Images" method in wind turbine blade defect detection, and compares it from three dimensions: expert subjective perception, objective index evaluation, and YOLO detection performance.

[0157] The comparative results clearly demonstrate that the proposed "Adaptive Text-Guided Super-Resolution Reconstruction of Wind Turbine Blade Defect Images" method exhibits significant advantages in wind turbine blade defect detection and detail restoration. Through multimodal semantic cue word fusion and a bidirectional classifier-guided mechanism, this invention effectively suppresses artifacts and texture shifts while achieving high-fidelity reconstruction and detail enhancement of key defect areas, significantly improving the clarity and recognizability of the reconstructed image, as well as the accuracy and stability of downstream defect detection tasks. This method maintains excellent performance even under complex lighting, low resolution, and noise interference conditions, verifying the effectiveness and advancement of the adaptive text-guided strategy in wind turbine blade image super-resolution reconstruction.

Claims

1. A method for adaptive text-guided super-resolution reconstruction of fan blade defect images, characterized in that: Comprising the following steps: S1: image preprocessing, obtaining high-quality blade subgraphs covering typical defect scenarios; S2: using the high-quality blade subgraphs in S1, constructing a fan blade multi-modal prior knowledge base; S3: generating global style prompt words representing image macrostructure and texture features through a two-stage retrieval strategy of sparse feature guidance and CLIP semantic sorting; S4: defining multiple defect types of fan blades and training a blade defect detection model; S5: positioning the blade surface defect area, and generating local content prompt words describing defect attributes and spatial distribution based on visual language large model; S6: constructing a multi-level adaptive text prompt word generator, integrating global style prompt words, local content prompt words and preset image quality prompt words, and fusing them in the order of "global-local-quality" semantic hierarchy, dynamically adjusting the guidance intensity of each prompt word through adaptive weight distribution mechanism, constructing composite forward prompt words for diffusion model reasoning and reverse prompt words for suppressing redundant information; S7: in the iterative denoising process of the diffusion model, combine the bidirectional conditional guidance to dynamically optimize the latent features, achieve the balance of low-frequency structure stability and high-frequency defect detail enhancement, and output high-resolution reconstructed images; S8: establish a multi-dimensional evaluation system and causal reasoning analysis framework to evaluate the effectiveness and robustness of the method in defect identification and detail recovery.

2. The method of claim 1, wherein the method is characterized by: The steps in S6 are as follows: S6-1: Construct a multi-level adaptive text prompt word generator, wherein the image quality prompt word includes a forward quality prompt word and a reverse quality prompt word; S6-2: When there are obvious defect areas in the image, increase the weight of the local content prompt word; When the defects are less or invisible, increase the weight of the global style prompt word; S6-3: The prompt word sequence after weighted fusion is constructed as a composite forward prompt word, and the preset reverse quality prompt word is used as a reverse prompt word for suppressing redundant information, which together constitute a bidirectional conditional guidance signal.

3. The method of claim 1, wherein the method is characterized by: The specific steps of S1 are as follows: S1-1: Obtain high-definition original image data under different wind power scenarios; S1-2: Use the YOLO blade segmentation model to extract the foreground of the original image, generate a blade area mask, and determine the candidate area based on the mask shape; S1-3: Use an adaptive cropping method to cut the mask area, so that the size of the cropped frame meets the preset requirements and maximizes the coverage of the effective blade area. After cropping and quality screening, high-quality fan blade subgraphs covering typical defect scenarios such as cracks, paint peeling, and surface contamination are obtained.

4. The method of claim 1, wherein the method is characterized by: The steps of S2 are as follows: S2-1: Use the obtained high-quality fan blade subgraphs to construct a multi-modal prior knowledge base containing image-text pairs and their feature indexes; S2-2: The knowledge base construction process is composed of a text generation module and an index construction module. The text generation module is first performed. After the input image is normalized and tensorized, it is input into the multi-modal Transformer framework together with the prompt word. The model generates semantic representations highly related to the overall color tone, texture distribution, and macro-style attributes through cross-modal alignment mechanism, and obtains the global description in natural language form through decoding, ensuring that the generated results can highlight the global style information, S2-3: After the text generation is completed, the index construction module is entered to construct the structure level and semantic level feature index: In the index construction stage, first, the self-training sparse autoencoder is used to compress the features of the input image, extract low-dimensional sparse feature vectors that can preserve the overall geometric structure and global texture distribution, Then, for each image-text pair, the pre-trained CLIP model is used to extract image features and text features, and the normalized processing is used to obtain the unified dimension semantic embedding representation; Finally, the image-text pair and its corresponding structure level features and semantic level features are indexed and mapped, and the efficient vector retrieval is realized through FAISS. The structure level recall is completed in the first stage based on sparse features, and the semantic level sorting is completed in the second stage based on CLIP features, so as to construct a logically consistent and scalable multi-modal prior knowledge base.

5. The adaptive text guided fan blade defect image super-resolution reconstruction method according to claim 1, characterized in that: The S3 step is as follows: Based on the multi-modal prior knowledge base, two-stage cascade retrieval is performed, First, in the first-stage structure level recall, the input low-resolution fan blade image extracts the structure feature vector through the self-training sparse autoencoder, and performs similarity retrieval in the FAISS sparse feature index library, and selects the top-10 candidate samples with the highest similarity and their text descriptions; Then, in the second-stage semantic level sorting, the CLIP cross-modal visual-linguistic alignment model is used to extract the semantic features of the input image and the candidate text, and the cross-modal semantic alignment is realized through the cosine similarity calculation, and the top-3 text descriptions with the highest semantic matching degree are selected; Finally, a weighted fusion strategy is used to generate a global style prompt representing the macro-structure, texture distribution and overall visual style of the input image.

6. The method of claim 1, wherein the method is a self-adaptive text guided super-resolution reconstruction method for fan blade defect images. The S4 includes the following steps: S4-1: According to the typical damage modes and key component distribution of the fan blade in the actual running environment, the target category system of the detection model is determined, including defect categories and device categories; S4-2: Based on the pre-processed high-quality fan blade subgraph, the training set in the form of rotated box is constructed by using artificial fine labeling, and data enhancement strategy is introduced for the training samples; S4-3: The YOLOv8-OBB rotating target detection model is supervised trained by using the training set, and the model performance is systematically evaluated based on the independent validation set. Optimization strategies are introduced in the training process.

7. The method of claim 1, wherein the method is characterized by: The S5 step is as follows: S5-1: Detect the input image using the trained YOLOv8-OBB model, crop each detected defect area, and input it into the visual language large model LLaVA to generate a single sentence technical description containing defect type, morphological characteristics, and location information through a pre-set prompt word template; S5-2: When there are multiple defects in the same image, use a description synthesis strategy to fuse multiple single sentence descriptions into a unified local content prompt; S5-3: When no defects are detected, call LLaVA to generate a state description for the local area of the image, ensuring that each input image corresponds to an effective local content prompt.

8. The method of claim 1, wherein the method is a self-adaptive text guided super-resolution reconstruction method for fan blade defect images. The S7 steps are as follows: S7-1: Encode the low-resolution input image into the latent space through the pre-trained variational autoencoder to obtain the initial latent feature representation, and the encoding process is represented as: （8） S7-2: In the T-step iterative denoising process, the composite forward prompt and the reverse prompt are jointly input into the IRControlNet conditional control module to adjust the semantic constraints and quality suppression in each step of the denoising process; S7-3: Implement dynamic optimization through the bidirectional classifier free guidance mechanism: the forward branch uses the composite forward prompt to enhance the semantic matching with the target features, and the reverse branch uses the reverse quality prompt to suppress artifacts and non-target content generation; S7-4: In each step of the diffusion sampling, the balance weight between structure fidelity and detail enhancement is adaptively adjusted according to the current noise level to realize stable recovery of low-frequency areas and fine restoration of high-frequency defect features, and the dynamic balance coefficient can be represented by formula (10): （10） In the formula, is the structure fidelity weight of the first diffusion step, used to maintain the low-frequency structure information such as the blade profile, the main ridge line, etc.; is the structure fidelity weight of the first diffusion step, used to maintain the low-frequency structure information such as the blade profile, the main ridge line, etc.; is the detail enhancement weight of the first diffusion step, used to strengthen the high-frequency defect features such as cracks, paint peeling, corrosion textures, etc.; is the detail enhancement weight of the first diffusion step, used to strengthen the high-frequency defect features such as cracks, paint peeling, corrosion textures, etc.; is the current diffusion step number; is the total diffusion step number; S7-5: Reconstruct the optimized latent features to the pixel space through the decoder to output the final high-resolution reconstructed image: （11）。 9. The method of claim 1, wherein the method is a self-adaptive text guided super-resolution reconstruction method for fan blade defect images. The S8 steps are as follows: S8-1: For the authenticity, structure restoration, and defect recognition ability of the super-resolution reconstruction result, a multi-dimensional evaluation system covering subjective, objective, and task performance is established, including the following: a. Subjective perception evaluation based on expert scoring, used to measure the naturalness, texture restoration, and overall realism of the image; b. Objective quantitative evaluation based on PIQE, BRISQUE, NIQE, and CLIP-IQA no-reference indicators, used to measure the structural consistency and perceptual quality of the reconstructed image, where the mathematical expressions of PIQE, BRISQUE, and NIQE are as follows: （12） （13） （14） where BM is a blocking artifact metric used to evaluate the blocky structure in the image caused by compression artifacts; NM is a noise metric used to measure the noise level caused by image loss or transmission error, is a natural scene statistical feature vector obtained by statistical fitting from the MSCN coefficients and their products, denotes a pre-trained support vector regression model used to map the feature vector to a final quality score, where , and , are the mean vector and covariance matrix of the multivariate Gaussian model of the natural image and the distorted image, respectively. c. Task performance evaluation based on the YOLO defect detection model, used to measure the improvement of defect recognition accuracy of the reconstructed image; S8-2: Treat the prompt guidance strategy as an intervention variable and construct a causal graph framework between the prompt strategy, the super-resolution model, and the authenticity of the reconstructed image to clarify the causal transmission path between variables; S8-3: In the factual experiment, input the test image and different prompt guidance strategies into the super-resolution model, and perform expert subjective evaluation, objective indicator calculation, and defect detection performance evaluation on the model output results; S8-4: In the counterfactual experiment, keep the model structure, inference process, and data input consistent, only remove the prompt guidance strategy, and make the super-resolution model rely only on low-resolution images for reconstruction, and perform multi-dimensional evaluation on the output results. S8-5: Quantify the causal promotion effect of the prompt guidance strategy on the image authenticity in each dimension by comparing the difference between the fact and counter-fact experiments.

Images