A fan blade damage identification method based on structural vibration modal characteristic transmission and image recognition

Abstract

The application discloses a fan blade damage identification method based on structural vibration modal feature transmission and image recognition. The method constructs a three-layer deep learning architecture composed of an upper structure dynamic network, a lower visual network and a joint fusion network, realizing deep fusion of multi-modal information. Among them, the upper structure dynamic network implicitly extracts the structural dynamics characteristics of the blade through the acceleration sensor signal to perceive the internal state anomaly; the lower visual network identifies the surface damage of the blade by fusing the visible light and infrared image features collected by the unmanned aerial vehicle; the joint fusion network realizes deep fusion and collaborative decision of the above-mentioned structural features and visual features through cross-modal attention and Bayesian inference. The application significantly improves the accuracy, robustness and environmental adaptability of damage identification by fusing the internal dynamic response of the blade and the external visual information, and provides an efficient solution for realizing intelligent state monitoring and operation and maintenance of the fan blade.

Description

Technical Field

[0001] This invention relates to a damage identification method, and more particularly to a wind turbine blade damage identification method based on structural vibration modal feature transfer and image recognition. Background Technology

[0002] Offshore wind power is an important component of clean energy. As the core component for capturing wind energy, wind turbine blades are subjected to complex alternating loads and erosion from the harsh marine environment over long periods, making them highly susceptible to damage such as cracks, corrosion, and coating peeling. Timely and accurate identification of blade damage is crucial for preventing catastrophic failures, ensuring the safe and stable operation of wind farms, and reducing operation and maintenance costs.

[0003] Existing wind turbine blade damage detection technologies mainly include machine vision-based detection technologies, optical frequency domain reflectance-based detection methods, deep learning algorithm-based blade damage identification methods, and vibration monitoring data-based blade damage identification methods. However, most existing technologies rely on a single data source, either purely visual or purely vibration-based, and under complex and variable marine operating conditions, they all have shortcomings in terms of reliability, robustness, or implementation cost.

[0004] 1. Machine vision-based inspection technology uses drones or fixed cameras to capture images of the blade surface and then uses image processing or deep learning algorithms to identify visible damage. However, visual inspection is severely limited by shooting distance, viewing angle, and lighting conditions. For large blades, multiple sensors are often required, increasing system complexity and cost, and it cannot detect internal or invisible damage.

[0005] 2. Detection methods based on optical frequency domain reflectance technology utilize fiber optic sensors to sense strain or temperature changes. However, in long-distance monitoring applications, their spatial resolution and sensing performance degrade. Furthermore, phase noise and nonlinear tuning effects of the light source can affect measurement accuracy, requiring additional technical compensation and increasing implementation difficulty.

[0006] 3. While deep learning-based blade damage identification methods can automatically extract features, their performance is highly dependent on the scale and quality of the training data. In practical applications, obtaining sufficient high-quality labeled data covering various damage types and operating conditions is costly, and insufficient data can easily lead to poor model generalization ability and poor identification results for unknown or rare damage.

[0007] 4. Blade damage identification methods based on vibration monitoring data infer structural damage by analyzing changes in blade vibration response. However, vibration signals are highly susceptible to strong interference from environmental factors (such as wind speed, wind direction, and temperature changes). This interference may mask or confuse the characteristic changes caused by actual damage, leading to false alarms or missed alarms, making it difficult to guarantee the accuracy and reliability of damage identification. Summary of the Invention

[0008] Purpose of the invention: The purpose of this invention is to provide a wind turbine blade damage identification method with high recognition accuracy, good environmental robustness, and strong feature fusion adaptability.

[0009] Technical Solution: The wind turbine blade damage identification method based on structural vibration modal feature transfer and image recognition described in this invention adopts a three-layer deep learning architecture consisting of an upper-layer structural dynamic network, a lower-layer visual network, and a joint fusion network, and includes the following steps:

[0010] (1) Simultaneously acquire vibration acceleration signals of wind turbine blades and blade surface images containing visible light and infrared information; preprocess the vibration acceleration signals and the blade surface images respectively;

[0011] (2) The structural features of the wind turbine blade under test are extracted from the preprocessed vibration acceleration signal through the upper-layer dynamic network; the visual features of the wind turbine blade under test are extracted from the preprocessed blade surface image through the lower-layer visual network; the upper-layer dynamic network adopts a deep neural network based on convolution and self-attention mechanism; the lower-layer visual network adopts a multimodal convolutional neural network based on weight allocation.

[0012] (3) Input the structural features and the visual features into the joint fusion network; the joint fusion network performs the following operations:

[0013] (31) The structural features and the visual features are interacted and fused based on the cross-modal attention mechanism to obtain preliminary fusion features, and the fusion layer damage probability is output based on the preliminary fusion features;

[0014] (32) A structural vibration classifier outputs a damage prior probability based on the vibration signal based on the structural features; a vision classifier outputs a damage likelihood probability based on the image appearance based on the visual features; and the damage prior probability and the damage likelihood probability are fused based on Bayes' theorem to calculate the final posterior damage probability.

[0015] (4) Output damage diagnosis results, which include at least the damage type and damage probability determined based on the posterior damage probability, and the damage location information determined based on the visual features.

[0016] Preferably, in step (1), the vibration acceleration signal is collected by at least three acceleration sensors arranged along the length of the wind turbine blade, and the at least three acceleration sensors are respectively arranged in the root region, the middle region and the tip region of the blade.

[0017] Preferably, the preprocessing of the vibration acceleration signal includes time-frequency joint coding using wavelet transform.

[0018] Preferably, the lower-layer visual network performs adaptive weighted fusion of visible light features and infrared features, and the fusion weight coefficients are dynamically calculated and generated by a lightweight sub-network based on the image quality evaluation index of visible light and infrared images.

[0019] Preferably, the adaptive weighted fusion is configured such that when a local thermal anomaly region is identified in the infrared image, the fusion weight of the visible light features of that region is reduced accordingly to guide the priority detection of that region.

[0020] Preferably, the upper-layer dynamic network includes a one-dimensional convolutional neural network layer for extracting local vibration mode features and a Transformer encoder layer for capturing dynamic correlation features across sensors.

[0021] Preferably, in step (1), the image of the blade surface is acquired by a drone equipped with a visible light camera and an infrared thermal imaging device.

[0022] Preferably, the preprocessing of the vibration acceleration signal also includes bandpass filtering and channel normalization.

[0023] Preferably, the preprocessing of the blade surface image includes contrast enhancement and noise filtering.

[0024] Preferably, the method further includes a system update step (5): maintaining a dynamic database containing historical blade images and using online learning technology to update the model in the three-layer deep learning architecture in real time with newly acquired data in order to learn new damage patterns.

[0025] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages: (1) It can capture the internal dynamic anomaly and the visible damage on the surface of the blade at the same time, realize the mutual verification of internal structural anomaly and surface damage, and overcome the disadvantage of easy misjudgment of a single data source; (2) The cross-modal adaptive fusion mechanism can dynamically adjust the dependency weight according to the working conditions, ensuring the stability of monitoring performance under harsh conditions; (3) The discovery and assessment of damage has both high real-time performance and accuracy, and can accurately locate the specific location and extent of damage in the first time. Attached Figure Description

[0026] Figure 1 This is a flowchart of the present invention. Detailed Implementation

[0027] The technical solution of the present invention will be further described below with reference to the accompanying drawings.

[0028] This invention provides a method for wind turbine blade damage identification based on structural vibration modal feature transfer and image recognition. This method achieves multimodal fusion intelligent identification of wind turbine blade damage by constructing a three-layer deep learning architecture consisting of an upper-layer structural dynamic network, a lower-layer visual network, and a joint fusion network.

[0029] I. Methods and Steps

[0030] As shown in the attached figure, the wind turbine blade damage identification method based on structural vibration mode feature transfer and image recognition of the present invention includes the following steps:

[0031] Step 1: Synchronous Acquisition and Preprocessing of Multimodal Data

[0032] 1. Data Collection

[0033] (1) Arrange a sensor network on the wind turbine blades to collect vibration data. At least three acceleration sensors are arranged along the length of a single blade to form a sensor arrangement unit with the smallest identifiable structural dynamic characteristics. Specifically, the three sensors are respectively located near the blade root (approximately 0.2L~0.3L from the blade root), in the middle of the blade (approximately 0.45L~0.55L from the blade root), and near the blade tip (approximately 0.15L~0.25L from the blade tip), where L is the total length of the blade. The number of measuring points can be expanded to 4~6 points depending on the required monitoring accuracy.

[0034] (2) The UAV is equipped with a high-resolution visible light camera and infrared imaging equipment to conduct flight inspections to simultaneously acquire visible light images and infrared thermal imaging data of the blade surface.

[0035] 2. Data Preprocessing

[0036] (1) Vibration acceleration signal preprocessing: Bandpass filtering is applied to the acceleration signal to reduce background noise. Subsequently, time-series slicing is performed using a sliding time window, with the time window length set to 1s~2s and the overlap rate between adjacent time windows being 50%~75%. Next, zero-mean normalization or maximum-minimum normalization is performed on each acceleration sensor channel to eliminate the influence of different installation conditions and dimensional differences at different measurement points. Finally, time-frequency joint encoding is performed on the signal, and wavelet transform is used to convert the signal from the time domain to the time-frequency domain to generate features suitable for network input.

[0037] (2) Leaf surface image preprocessing: For the acquired visible light and infrared images, Gaussian filtering (bilateral filtering) is first used to remove noise while preserving edge information. Then, contrast enhancement is performed, specifically using the CLAHE (Contrast Limit Adaptive Histogram Equalization) method, which divides the image into multiple small blocks for independent histogram equalization and sets a contrast limit parameter (e.g., setting the limit parameter to 40) to prevent over-enhancement. Finally, the pixel intensity values ​​of the image are rescaled to the standard range of [0, 1] using a scale normalization method.

[0038] Step 2: Feature extraction based on a three-layer deep learning architecture

[0039] This step involves inputting the preprocessed data into a three-layer network architecture for feature extraction.

[0040] 1. Extracting structural features

[0041] The preprocessed vibration acceleration signal sequence is input into the upper-level dynamic network. This network employs a structure combining a one-dimensional convolutional neural network and a multi-head self-attention mechanism. First, three one-dimensional convolutional layers are used to extract local vibration mode features, with the number of convolutional kernels in each layer being 32, 64, and 128 respectively, a kernel size of 3, and a stride of 1. Each convolutional layer is followed by a ReLU activation function and a max-pooling layer. The feature sequence extracted by the one-dimensional convolutional layers is converted into a feature embedding sequence through a linear mapping layer, and then input into a Transformer module containing 2 to 4 encoding layers. Each encoding layer includes a multi-head self-attention mechanism with 4 to 8 attention heads to capture cross-sensor dynamic correlation features. The network finally outputs structural features Z representing the internal structural state of the blade. upper .

[0042] 2. Visual Feature Extraction: The preprocessed visible light image and infrared thermal image are input into the lower-level visual network. This network includes a dual-branch feature extraction sub-network, an adaptive fusion sub-network, and a damage detection head. The dual-branch feature extraction sub-network extracts multi-scale feature maps from the visible light and infrared images respectively; the adaptive fusion sub-network performs weighted fusion of the feature maps at corresponding scales to obtain fused multi-scale visual features; and the damage detection head outputs the damage category and bounding box coordinates based on the fused multi-scale visual features.

[0043] (1) Dual-branch feature extraction: The dual-branch feature extraction sub-network adopts a dual-branch convolutional architecture. One branch processes visible light images, and the other branch processes infrared images. Both use deep residual networks (such as ResNet-50) as backbone networks to extract high-dimensional spatial features F respectively. vis (Visible light image) and F ir (Infrared images), which may include mechanisms such as attention to optimize feature representation.

[0044] (2) Adaptive feature fusion: The adaptive fusion sub-network uses a weight-based multimodal convolutional neural network to adaptively weight and fuse the extracted bimodal features to generate a unified visual feature representation Z. lower The formula is:

[0045]

[0046] The weighting coefficient α is used to adjust the global fusion weights of the visible light mode and the infrared mode. For local thermal anomaly regions, the feature response of the corresponding spatial location can be further enhanced through local attention mapping. Its adaptive calculation is completed by a lightweight sub-network based on indicators such as image sharpness, signal-to-noise ratio, and infrared thermal anomaly intensity. The formula is as follows:

[0047]

[0048] Q vis Q ir α and W1 represent the image quality evaluation metrics for visible light and infrared images, respectively. σ is the sigmoid function, and W1 and W2 are weighting coefficients that are adaptively adjusted based on image quality. When the visible light image quality is high, α approaches 1, and the network relies more on visible light features; when thermal anomalies in the infrared image are very significant, α decreases, and the network relies more on infrared features. Simultaneously, this strategy guides the network to prioritize thermal anomaly regions detected in the infrared image.

[0049] (3) Damage detection and localization: based on the fused visual features Z lower The damage detection head utilizes the YOLOv11 model for real-time detection, classification, and bounding box localization of various damage types, including cracks, corrosion, and coating peeling. Damage localization accuracy is improved by introducing an additional bounding box regression module. The detection head independently outputs the damage category and bounding box coordinates.

[0050] Step 3: Cross-modal feature fusion and damage identification and diagnosis

[0051] The structural features Z extracted from the upper-level dynamic network upper (Capturing the inherent dynamic characteristics of the structure, such as natural frequencies and mode shape changes) and visual features Z extracted by the lower-level visual network. lower (Capture damage appearance characteristics, such as morphology, texture, and location) and input them into the joint fusion network. Its workflow is as follows:

[0052] 1. Feature Alignment and Interaction

[0053] The network first uses a feature alignment layer (such as linear transformation and attention mapping) to separate two types of heterogeneous features (Z... upper and Z lowerThe features are projected onto the same semantic space to eliminate modal differences. Then, a cross-modal attention mechanism is used to calculate the mutual importance weights between the two types of features (e.g., the attention weight of structural features on visual features, and the attention weight of visual features on structural features), and a preliminary fusion feature Z is obtained through weighted summation. fused The fused features are input into a fully connected classifier, and after Softmax normalization, the damage probability P of the fusion layer is generated. fused (That is, the direct output of feature fusion, reflecting the possibility of damage supported by both structural and visual features).

[0054] 2. Independent probability estimation and Bayesian fusion

[0055] Structural prior probability estimation: This involves estimating the structural features Z... upper Input a dedicated structural vibration classifier and output a damage prior probability P based on the vibration signal. prior This probability reflects the damage assessment based on the inherent characteristics of structural dynamics.

[0056] Visual likelihood probability estimation: This involves estimating the visual feature Z... lower Input a dedicated vision-specific classifier and output a damage likelihood probability P based on the image appearance. likelihood This probability reflects the assessment of damage based on external, visual evidence.

[0057] Bayesian probabilistic fusion decision: P prior With P likelihood Input the Bayesian probability fusion module. This module uses P... prior Let P be the prior probability. likelihood Let P be the likelihood function, and calculate the posterior damage probability P according to Bayes' theorem. posterior The joint fusion network output is used only for feature alignment and intermediate representation learning, and is no longer repeatedly included in the posterior calculation as an independent probability term. The calculation formula is:

[0058]

[0059] in Type of damage To represent all possible injury categories (including health status), this P posterior This is the final system-adopted approach that integrates the inherent characteristics of structural vibration with visual evidence, and its reliability is higher than any single-mode probability.

[0060] 3. System Output: Integration of multi-dimensional diagnostic results

[0061] The system ultimately outputs a comprehensive diagnostic result, with the generation logic for each element as follows:

[0062] Damage type: Determined by the damage category corresponding to the highest posterior probability output by the Bayesian module;

[0063] Damage region coordinates: directly using the damage target bounding box or pixel-level mask output by the lower-layer visual network;

[0064] Damage probability: Directly outputs the posterior damage probability P calculated by the Bayesian module. posterior ;

[0065] Damage severity level: The damage severity level is determined comprehensively based on the damage geometric scale parameters, thermal anomaly intensity parameters, and structural response anomaly degree; the posterior damage probability is only used to characterize the model's confidence in the existence of damage.

[0066] II. Model Training and System Deployment

[0067] 1. Data preparation and annotation

[0068] To train the three-layer deep learning architecture, a high-quality, paired multimodal dataset needs to be constructed.

[0069] (1) Data acquisition and damage setting: Three test blades were selected, and healthy state and three preset damage states (leading edge crack, trailing edge corrosion, and coating peeling) were set respectively. The damage parameters are shown in the table below:

[0070]

[0071] During the data acquisition process, the fan should be kept in a stable operating condition (speed maintained at 1500 r / min, constant load) to eliminate interference. Vibration sensors (sampling frequency set to 1000 Hz) were used to collect blade vibration signals, with each state lasting 30 minutes. Simultaneously, a high-definition industrial camera (resolution 1920×1080) was used to acquire images of the blade surface at 1-second intervals, and a timestamp synchronization mechanism was used to ensure the synchronization of vibration data and image data to form training sample pairs.

[0072] (2) Vibration data labeling: The vibration data is a time-series signal, and the overall state labeling is performed at the segment level. Each vibration segment is 10 seconds long, corresponding to an overall state label. The labeling adopts a two-level labeling system: the first-level label is divided into healthy (0) and damaged (1), and the second-level label is the specific damage type (healthy (0), leading edge crack (1), trailing edge corrosion (2), coating peeling (3)). The labeling output format is: (time-series signal matrix, overall state label, timestamp, working condition information), where the time-series signal matrix is ​​a 10000×1 one-dimensional vector (10s×1000Hz).

[0073] (3) Image data annotation: The LabelImg annotation tool was used to perform fine annotation on the image. The annotation content included the damage category (crack, surface corrosion, coating peeling, no damage (background)) and the precise axis-aligned bounding box coordinates (x, y, z). min ,y min , x max , y max ), where x min y min , where x is the coordinate of the top-left corner of the bounding box. max y max The coordinates are the bottom right corner coordinates of the bounding box, expressed in pixels. The output uses COCO format for easy reading and training of the visual network. When multiple damages exist in a single image, each damage category is labeled with an independent bounding box; images without damage are uniformly labeled as background / healthy, and no bounding box is drawn. During the labeling process, avoid making the bounding boxes too large or too small, ensuring that the damaged area is completely contained within the box and does not contain too much background information.

[0074] (4) Multimodal data pairing: Using timestamps, blade numbers, and rotor azimuth angles as joint indexes, and combining them with image spatial positioning information, vibration time-series segments collected near the same blade, the same blade segment, and the same time point are paired with image samples to construct standard training sample pairs in the format of (vibration sample, image sample, overall state label, image annotation file). During the pairing process, samples with mismatched timestamps, missing data, or incorrect annotations are removed. Finally, a multimodal dataset containing 10,000 sets of samples is constructed and divided into training set, validation set, and test set in a ratio of 7:2:1.

[0075] 2. Model Training Process

[0076] The training adopts a two-stage strategy that combines pre-training of separate networks with joint fine-tuning to make full use of data and achieve deep fusion of cross-modal features.

[0077] Phase 1: Sub-network pre-training

[0078] (1) Pre-training of the upper-level dynamic network: Using labeled vibration time segments, the hybrid architecture network of 1D-CNN and Transformer encoder is trained in a purely supervised manner. The goal is to learn a general feature representation that distinguishes between the healthy and damaged states of the blade from the vibration signal, realize the classification of the overall health status of the blade, and output the prediction result of the overall state of the blade. Training hyperparameter settings: learning rate 1 The batch size is 32, the training rounds are 50, and an early stopping strategy is used to prevent overfitting. When the accuracy of the validation set does not improve for 10 consecutive rounds, training is stopped and the pre-trained model parameters are saved.

[0079] (2) Lower-layer visual network pre-training: A two-stage pre-training method was adopted. In the first stage, the ResNet50 backbone network was initialized and pre-trained using the ImageNet large-scale general image dataset to learn the general visual feature extraction capabilities. In the second stage, the network parameters were fine-tuned using a self-annotated wind turbine blade image dataset to adapt it to the feature distribution of wind turbine blade surface damage. Training hyperparameter settings: The learning rate was 1. The batch size is 16, the training rounds are 30, and the early stopping strategy is also adopted to save the pre-trained model parameters.

[0080] Phase Two: Joint Fine-tuning

[0081] (1) Network architecture splicing: The pre-trained upper-layer network, lower-layer network and joint fusion network are spliced ​​together to form a complete three-layer architecture. Among them, the vibration dynamics network outputs a global state feature vector (dimension 128) to represent the overall health of the blade; the vision network outputs a local damage visual feature vector (dimension 256) to represent the damage category and location information; the joint fusion network adopts a cross-modal attention fusion mechanism to weight and fuse the dual-modal features and output a unified fusion feature vector (dimension 384) for subsequent overall state classification and local damage detection.

[0082] (2) Parameter initialization: To avoid destroying the pre-trained general features, a layered unfreezing strategy is used for parameter initialization:

[0083] Freeze bottom layer parameters: Freeze the bottom and shallow layer parameters of the vibration dynamics network and the vision network (such as the first 80% of the layers) to retain the general features they have learned.

[0084] Unfreeze key parameters: Unfreeze the top-level parameters (last 20% of the layers) of the vibration dynamics network and the vision network, as well as all parameters of the joint fusion network, so that it can be used for cross-modal feature adaptation and fusion.

[0085] (3) Training process and hyperparameter specification

[0086] (31) Forward propagation: Paired multimodal training samples are input into the model. Vibration data is processed by a vibration dynamics network to extract global features, and image data is processed by a vision network to extract local features. The two are then input into a joint fusion network to generate deep fusion features. The fusion features are then input into the overall state classification head and the damage classification and bounding box regression head, respectively, and the prediction results are output.

[0087] (32) Loss calculation and backpropagation: Calculate the total loss of the multi-task operation and update the parameters of all unfrozen layers using the backpropagation algorithm. Mini-batch gradient descent is used, and the batch size is set to 16.

[0088] (33) Hyperparameter settings: The key hyperparameter settings for joint fine-tuning are as follows:

[0089] (331) Learning rate: Use a small learning rate (e.g., 1) for the unfrozen pre-trained layers (higher-level parameters). ); a higher learning rate (e.g., 1) is used for joint fusion networks. ).

[0090] (332) Learning rate scheduling: adopt cosine annealing strategy, and gradually reduce the learning rate as the training rounds increase.

[0091] (333) Regularization: Set the weight decay coefficient to 1 This is to prevent the model from overfitting.

[0092] (334) Gradient processing: Gradient clipping is used with a threshold of 1.0 to avoid gradient explosion during training.

[0093] (335) Training rounds: A total of 50 rounds of joint fine-tuning training.

[0094] (34) Fine-tuning in the later stages of training: In the later stages of training (e.g., after the 30th round), some shallow parameters of the vibration dynamics network and the visual network are gradually unfrozen, and a smaller learning rate (e.g., 1) is adopted. Fine-tuning is performed to further improve model performance and achieve global parameter optimization.

[0095] (4) Loss function design: The total loss function is a weighted sum of the multi-task losses, integrating the vibration overall classification loss, image damage classification loss, and bounding box regression loss. The formula is as follows:

[0096]

[0097] in, This is the task weighting coefficient, used to balance the loss contribution of different tasks; For overall vibration classification loss, For image damage classification loss, Image bounding box regression loss is used; cross-entropy loss function is used for both vibration overall classification and image damage classification; Smooth L1 Loss is used for image bounding box regression. After experimental optimization, the weights can be set to... To balance different tasks.

[0098] (5) Optimizer selection: The AdamW optimizer is selected. This optimizer adds a weight decay term to the Adam optimizer, which can effectively prevent model overfitting and improve the model's generalization ability. Optimal optimizer parameter settings: The weight decay coefficient is 1. .

[0099] 3. Online learning and dynamic updates

[0100] To continuously improve system performance, a dynamic standard database containing images of both normal and abnormal blades is established and maintained. Online learning technology is employed to enable the model to update in real time using newly acquired data, thereby learning new damage patterns and improving its adaptability to unknown damage types.

[0101] 4. System Deployment

[0102] In terms of implementation and deployment, a unified intelligent diagnostic system is deployed at the wind turbine monitoring center, integrating an accelerometer sensor network and a drone image acquisition system. Edge computing technology is used to process most of the data on-site, uploading only critical anomaly data to the cloud to reduce data transmission costs and latency. Maintenance personnel can view monitoring data and diagnostic results in real time via mobile devices or desktop systems, ensuring timely maintenance measures are taken.

Claims

1. A method for identifying wind turbine blade damage based on structural vibration modal feature transfer and image recognition, characterized in that, A three-layer deep learning architecture consisting of an upper-layer dynamic network, a lower-layer visual network, and a joint fusion network is adopted, including the following steps: (1) Simultaneously acquire vibration acceleration signals of wind turbine blades and blade surface images containing visible light and infrared information; preprocess the vibration acceleration signals and the blade surface images respectively; (2) The structural features of the wind turbine blade under test are extracted from the preprocessed vibration acceleration signal through the upper-layer dynamic network; the visual features of the wind turbine blade under test are extracted from the preprocessed blade surface image through the lower-layer visual network; the upper-layer dynamic network adopts a deep neural network based on convolution and self-attention mechanism; the lower-layer visual network adopts a multimodal convolutional neural network based on weight allocation. (3) Input the structural features and the visual features into the joint fusion network; the joint fusion network performs the following operations: (31) The structural features and the visual features are interacted and fused based on the cross-modal attention mechanism to obtain preliminary fusion features, and the fusion layer damage probability is output based on the preliminary fusion features; (32) A structural vibration classifier outputs a damage prior probability based on the vibration signal based on the structural features; a vision classifier outputs a damage likelihood probability based on the image appearance based on the visual features; and the damage prior probability and the damage likelihood probability are fused based on Bayes' theorem to calculate the final posterior damage probability. (4) Output damage diagnosis results, which include at least the damage type and damage probability determined based on the posterior damage probability, and the damage location information determined based on the visual features.

2. The method according to claim 1, characterized in that, In step (1), the vibration acceleration signal is collected by at least three acceleration sensors arranged along the length of the wind turbine blade, and the at least three acceleration sensors are respectively arranged in the root region, the middle region and the tip region of the blade.

3. The method according to claim 1 or 2, characterized in that, The preprocessing of the vibration acceleration signal includes time-frequency joint coding using wavelet transform.

4. The method according to claim 1, characterized in that, The lower-layer visual network performs adaptive weighted fusion of visible light features and infrared features. The fusion weight coefficients are dynamically calculated and generated by a lightweight sub-network based on the image quality evaluation indicators of visible light and infrared images.

5. The method according to claim 4, characterized in that, The adaptive weighted fusion is configured such that when a local thermal anomaly region is identified in the infrared image, the fusion weight of the visible light features of that region is reduced accordingly to guide the priority detection of that region.

6. The method according to claim 1, characterized in that, The upper-layer dynamic network includes a one-dimensional convolutional neural network layer for extracting local vibration mode features and a Transformer encoder layer for capturing dynamic correlation features across sensors.

7. The method according to claim 1, characterized in that, In step (1), the image of the blade surface is acquired by a drone equipped with a visible light camera and an infrared thermal imaging device.

8. The method according to claim 1, characterized in that, The preprocessing of the vibration acceleration signal also includes bandpass filtering and channel normalization.

9. The method according to claim 1, characterized in that, The preprocessing of the blade surface image includes contrast enhancement and noise filtering.

10. The method according to claim 1, characterized in that, It also includes a system update step (5): maintaining a dynamic database containing historical blade images and using online learning technology to update the model in the three-layer deep learning architecture in real time with the newly acquired data in order to learn new damage patterns.

Images