Long flexible blade monitoring method and system based on multi-source data fusion

Abstract

The present invention relates to the field of wind power generation technology, and in particular to a long flexible blade monitoring method and system based on multi-source data fusion. The present invention adopts multi-source data fusion technology, arranges sensor nodes on the wind turbine blades to collect optical fiber strain, vibration, environment and image data, generates comprehensive feature data through preprocessing, and then extracts potential feature vectors by multi-modal autoencoder. Combined with offline finite element simulation data and physical constraints, physical information neural network mapping is used to predict force data. Subsequently, the multi-modal deep fusion network and hierarchical reinforcement learning generate control decisions, adjust the blade status in real time, and optimize the model through a closed-loop feedback mechanism. The present invention realizes the precise monitoring and active regulation of special working conditions such as blade contamination and icing.

Description

Technical Field

[0001] The present invention relates to the technical field of wind power generation, and in particular to a long flexible blade monitoring method and system based on multi-source data fusion. Background Art

[0002] As a key load-bearing component in modern wind turbines, the health of long, flexible blades directly impacts the turbine's power generation efficiency and operational safety. Existing monitoring methods (Chinese Invention Patent Publication No. CN118361358A, Title: Wind Turbine Blade Load Monitoring Method Based on Transmission System Vibration Signal Characteristic Calibration) rely primarily on transmission system vibration signal characteristic calibration technology. By collecting gearbox vibration data, turbine speed, and blade root stress, they use fast Fourier transforms, frequency band energy calculations, and linear regression models to establish a correlation between blade root force and vibration signals, thereby indirectly monitoring blade loads.

[0003] However, this traditional method has significant shortcomings: First, its monitoring method is single-minded, relying solely on transmission system vibration signals, making it difficult to accurately reflect the blade's true stress state under complex operating conditions such as blade contamination and icing. Second, the use of linear regression models to model the stress characteristics of nonlinear and variable operating conditions has limitations and is prone to large deviations due to environmental noise, equipment installation errors, and inconsistent data collection. Furthermore, existing technologies require additional sensors to be installed on the blades, which may negatively impact the blade structure and aerodynamic performance. The complex calibration process also increases maintenance difficulty and testing costs. Therefore, there is an urgent need for a new monitoring method that can achieve real-time and accurate monitoring of blade status through multi-source data fusion without interfering with the normal operation of the wind turbine, especially with higher recognition accuracy and response speed under special operating conditions such as contamination and icing. Summary of the Invention

[0004] In response to the many problems existing in the above-mentioned prior art, the present invention provides a long flexible blade monitoring method and system based on multi-source data fusion. The present invention fuses the optical fiber strain, vibration, environment and image data collected by various sensors through advanced preprocessing and multimodal autoencoders, extracts representative potential feature vectors, and uses physical information neural networks to map them into predicted blade force data under preset physical constraints. Subsequently, a multimodal deep fusion and hierarchical reinforcement learning decision module are used to jointly analyze the predicted data and the comprehensive feature data to generate an assessment vector reflecting the health status and risk level of the blades, and then automatically generate a control decision to adjust the blade status. The present invention can achieve early warning through data fusion and physical constraints when the blades are contaminated or frozen, and timely adjust the working status of the wind turbine blades, thereby ensuring efficient and stable operation of the wind turbine.

[0005] A long flexible blade monitoring method based on multi-source data fusion includes the following steps:

[0006] By arranging multi-source sensor nodes on wind turbine blades, optical fiber strain data, vibration raw data, environmental monitoring data and image data are collected, and all data are pre-processed to generate comprehensive sensor feature data;

[0007] Inputting the integrated sensor feature data into a multimodal autoencoder to extract potential feature vectors, and combining the data with offline finite element simulation data, using a physical information neural network under preset physical constraints to map the potential feature vectors into predicted blade force data;

[0008] The predicted blade force data is fused with the integrated sensor feature data to generate a blade health assessment vector, and hierarchical reinforcement learning is used to generate a control decision based on the health assessment vector. The control decision is used to adjust the blade state, and the control decision model is updated online through a closed-loop feedback mechanism.

[0009] Preferably, the preprocessing step includes applying discrete wavelet transform to the original vibration data, performing multi-level decomposition using a preset wavelet basis to extract the time-frequency characteristics of the signal, removing high-frequency noise in the vibration signal using a Kalman filter, and realizing signal smoothing processing using a state space model.

[0010] Preferably, the preprocessing step also includes using a sliding average filter to smooth the temperature, humidity and wind speed data of the environmental monitoring data to eliminate sudden fluctuations, and using a pre-trained convolutional neural network to extract image edge and texture features from the image data, and using principal component analysis technology to perform dimensionality reduction processing to generate image feature data.

[0011] Preferably, the multimodal autoencoder is provided with a plurality of data processing branches, which are respectively used to process vibration data, optical fiber strain data, environmental monitoring data and image data, wherein the vibration data branch adopts a one-dimensional convolutional network combined with a long short-term memory network to extract and encode time series features of the vibration data; the optical fiber strain data branch adopts a one-dimensional convolutional network to extract local strain features and uses a fully connected network to integrate features; the environmental monitoring data branch adopts a multi-layer perceptron to process the temperature, humidity and wind speed data to extract time-varying features; the image data branch adopts a pre-trained convolutional neural network to extract edge and texture features in the image and output a low-dimensional feature vector; the feature vectors output by each branch are dynamically weighted and fused in the middle layer through a cross-modal attention mechanism to generate a potential feature vector to comprehensively represent the blade state information.

[0012] Preferably, the physical information neural network adopts physical constraints during the training process, establishes a blade dynamics model based on offline finite element simulation data to simulate the stress and vibration response of the blade under different wind speed, temperature and humidity conditions, and uses a regression loss function to optimize the neural network to ensure that the output predicted blade force data is consistent with the actual force distribution.

[0013] Preferably, the predicted blade force data and the integrated sensor feature data are input into a multimodal deep fusion network together. The multimodal deep fusion network uses convolutional layers, fully connected layers and long short-term memory networks to encode and extract features from each input data, thereby generating a unified blade health assessment vector, which reflects the blade health status and current risk level.

[0014] Preferably, the multimodal deep fusion network adopts an adaptive weighting algorithm when generating a blade health assessment vector, and dynamically weightedly fuses the predicted blade force data input from the physical information neural network output with the environmental monitoring data, optical fiber strain data and image feature data contained in the comprehensive sensor feature data to achieve an accurate assessment of the blade health status and risk level.

[0015] Preferably, the hierarchical reinforcement learning includes low-level decision-making and high-level decision-making, wherein the low-level decision-making is used to respond to short-term abnormalities in blade status in real time and issue adjustment instructions for instantaneous vibration or drastic changes in strain, while the high-level decision-making combines long-term health assessment vectors with predicted blade force data to formulate a global control strategy, and realizes collaborative work between low-level and high-level decision-making through a feedback mechanism.

[0016] Preferably, the closed-loop feedback mechanism collects feedback data in real time after executing the control decision, compares the feedback data with the predicted data to calculate the deviation, and optimizes the parameters of the hierarchical reinforcement learning through an online adaptive update mechanism to ensure real-time adjustment and long-term stability of the blade state.

[0017] A long flexible blade monitoring system based on multi-source data fusion is used to implement the long flexible blade monitoring method based on multi-source data fusion. The system includes the following units:

[0018] Sensor node units, used to collect optical fiber strain data, vibration raw data, environmental monitoring data and image data on wind turbine blades;

[0019] a data preprocessing unit, configured to perform a discrete wavelet transform on the collected data to extract time-frequency characteristics of the vibration signal, remove noise from the vibration signal using a Kalman filter, process the environmental monitoring data using a sliding average filter, and process the image data using a pretrained convolutional neural network and principal component analysis technology, thereby generating comprehensive sensor feature data;

[0020] a multimodal autoencoder unit for processing the integrated sensor feature data through each branch separately and fusing them through a cross-modal attention mechanism to generate a latent feature vector;

[0021] A physical information neural network unit is used to map the potential feature vector into predicted blade force data by combining offline finite element simulation data and preset physical constraints;

[0022] a multimodal deep fusion unit for fusing the predicted blade force data with the integrated sensor feature data, extracting features through a convolutional layer, a fully connected layer, and a long short-term memory network encoding, thereby generating a unified blade health assessment vector;

[0023] A hierarchical reinforcement learning decision unit generates control decisions based on the blade health assessment vector, wherein the low-level decision unit responds to short-term anomalies in real time, and the high-level decision unit formulates a global control strategy to adjust the blade state;

[0024] The closed-loop feedback unit is used to collect feedback data in real time after the control decision is executed, compare the feedback data with the predicted data to calculate the deviation, and adjust the parameters of the hierarchical reinforcement learning decision model through an online adaptive update mechanism.

[0025] Compared with the prior art, the advantages and beneficial effects of the present invention are:

[0026] The present invention uses sensor nodes arranged on the blades to simultaneously collect optical fiber strain data, raw vibration data, environmental monitoring data, and image data. After preprocessing steps such as discrete wavelet transform, Kalman filtering, sliding average filtering, and pre-trained convolutional neural network and principal component analysis, comprehensive sensor feature data is generated.

[0027] The present invention uses a multimodal autoencoder branch network, a one-dimensional convolutional network, a long short-term memory network and a multi-layer perceptron to extract features from different modal data respectively, and realizes dynamic weighted fusion through a cross-modal attention mechanism to generate a potential feature vector;

[0028] The present invention combines offline finite element simulation data with preset physical constraints and uses a physical information neural network to map potential eigenvectors to accurately predict blade force data, thus overcoming the limitations of traditional methods that rely solely on a single vibration signal and avoiding monitoring errors caused by sensor noise and nonlinear working conditions.

[0029] This invention utilizes a multimodal deep fusion network and a hierarchical reinforcement learning decision module to automatically generate blade health assessments and control decisions, and incorporates a closed-loop feedback mechanism for online adaptive updates. This technology not only accurately identifies special operating conditions such as blade contamination and icing, but also ensures the system's real-time and robustness under dynamic conditions, significantly improving wind turbine operational safety and maintenance efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 Schematic diagram of the process of the present invention;

[0031] Figure 2 Schematic diagram of hierarchical reinforcement learning and closed-loop feedback decision-making in the present invention;

[0032] Figure 3 This is a system structure block diagram of the present invention. DETAILED DESCRIPTION

[0033] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of well-known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0034] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise," "include," etc. used herein indicate the presence of the features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0035] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0036] like Figure 1 As shown, a long flexible blade monitoring method based on multi-source data fusion includes the following steps:

[0037] By arranging multi-source sensor nodes on wind turbine blades, optical fiber strain data, vibration raw data, environmental monitoring data and image data are collected, and all data are pre-processed to generate comprehensive sensor feature data;

[0038] By deploying multi-source sensor nodes on wind turbine blades, this invention simultaneously acquires optical fiber strain data, raw vibration data, environmental monitoring data, and image data. After a series of preprocessing steps, this data is transformed into comprehensive sensor feature data. This eliminates the potential monitoring blind spots associated with a single data source and provides a more targeted and timely analysis basis for complex operating conditions such as blade contamination and icing.

[0039] By rationally distributing sensors at various locations on and within the blade, this invention enables comprehensive capture of blade structural strain changes, vibration characteristics, and external environmental conditions. During this process, each type of data undergoes independent acquisition channels to ensure information integrity and sampling synchronization. Data alignment and cleaning are then performed on edge or central computing platforms, ultimately providing a solid foundation for subsequent multimodal fusion.

[0040] Because long, flexible blades are susceptible to multiple factors at high altitudes, including wind loads, rain, snow, ice, and dust, especially in extreme climates or remote offshore environments, surface contamination, icing, and wear often lead to significant changes in the blade's aerodynamic and mechanical properties. Failure to promptly monitor and identify these changes can impact the turbine's overall operating efficiency and even threaten system safety.

[0041] To this end, the present invention fundamentally focuses on multi-source information fusion: on the one hand, optical fiber strain data can reflect the internal stress distribution of the blade body; on the other hand, raw vibration data can show the mechanical coupling characteristics between the blade and the transmission system; in addition, environmental monitoring data can indicate the dynamic changes of external conditions such as temperature, humidity and wind speed, and image data can capture visual clues of surface contamination, icing or local damage on the blade. Aggregating this information within the same monitoring framework helps to grasp the health status of the blade at a macro level. The comprehensive sensor feature data obtained based on multi-source data fusion can also be further input into a multimodal autoencoder to extract higher-level feature representations by mining the potential correlations between various types of data. Combined with offline finite element simulation data and physical constraints, a physical information neural network is used to map out predicted blade force data, providing a basis for subsequent health assessment and decision-making control.

[0042] When the blades are in special working conditions such as contamination or icing, these sensor nodes will immediately capture the blade surface temperature, strain anomalies, and image changes, and output feature quantities that can be fused through the preprocessing link, ultimately achieving accurate identification and real-time feedback of complex working conditions.

[0043] Preferably, the preprocessing step includes applying discrete wavelet transform to the original vibration data, performing multi-level decomposition using a preset wavelet basis to extract the time-frequency characteristics of the signal, removing high-frequency noise in the vibration signal using a Kalman filter, and realizing signal smoothing processing using a state space model.

[0044] In a preferred implementation of the present invention, a multi-layered and multi-faceted approach is employed to process the raw vibration data. The core concept is to utilize discrete wavelet transforms and Kalman filters for time-frequency resolution and noise suppression, respectively. The discrete wavelet transform is a processing method that localizes signal analysis in the time and frequency domains. Using wavelet bases of varying scales, the raw vibration signal can be decomposed into several subbands, thereby extracting time-frequency features relevant to the blade's dynamic characteristics.

[0045] If the blade is contaminated or frozen, the blade vibration mode will have specific high-frequency components or waveform distortions. Wavelet decomposition can highlight these abnormal features on a certain sub-band and compare them with the baseline features under normal conditions. In order to accurately capture subtle changes when implementing wavelet transform, it is necessary to select a wavelet basis (such as Daubechies or Symlet) that matches the blade vibration characteristics, and the number of decomposition layers must also be considered in combination with the blade speed range and the number of structural modes. After completing the discrete wavelet decomposition, the high-frequency sub-bands obtained by decomposition often contain more noise components. In order to further eliminate environmental interference and sensor errors, the present invention uses a Kalman filter to perform smooth estimation of the high-frequency sub-bands.

[0046] The Kalman filter dynamically predicts and corrects the system state by establishing a state-space model, allowing noise components to gradually attenuate over multiple iterations. For identifying blade contamination or icing, if certain high-frequency features persist in the vibration subband and the Kalman filter struggles to identify them as noise attenuation, this indicates that these features are likely true vibration anomalies and need to be marked and passed to subsequent fusion steps. In practical applications, for example, for vibration signals with sampling frequencies between 1kHz and 5kHz, discrete wavelet transforms are used to decompose the signals into four to six layers each time. Kalman filters are then applied to the high-frequency components of the top two layers, with the prediction-update period set between 10 and 20 milliseconds to balance computational efficiency and filtering effectiveness. This multi-level processing allows blades to achieve high recognition and accuracy of localized vibration anomalies caused by icing or contamination, both in the wavelet subband and in the Kalman filter output.

[0047] Preferably, the preprocessing step also includes using a sliding average filter to smooth the temperature, humidity and wind speed data of the environmental monitoring data to eliminate sudden fluctuations, and using a pre-trained convolutional neural network to extract image edge and texture features from the image data, and using principal component analysis technology to perform dimensionality reduction processing to generate image feature data.

[0048] For environmental monitoring data and image data, the present invention also adopts a preferred processing strategy in order to provide more dimensional support in the identification of special working conditions. First, in terms of environmental monitoring, since the instantaneous fluctuations of temperature, humidity and wind speed may be affected by factors such as sudden weather and airflow disturbances, directly inputting them into the subsequent analysis module will lead to unstable results or even misjudgment. Therefore, the present invention uses a sliding average filtering method to smooth the environmental monitoring data. By calculating the average value within a continuous time window, the extreme abnormal points are moderately smoothed, thereby obtaining a more stable temperature, humidity and wind speed curve.

[0049] If the monitored temperature is consistently low and the humidity is high, combined with wind speed fluctuations, it can be preliminarily determined that the blades may be in an icing risk area. If the monitored wind speed is low but the dust or salt spray content is high, it may indicate pollution accumulation. To capture the specific pollution or icing patterns on the blade surface in the image data, the present invention further uses a pre-trained convolutional neural network to extract edge and texture features from the image.

[0050] Pre-training involves initial feature learning on a common dataset. In this method, only a small amount of transfer training is required to adapt the model to the blade surface image. Using convolution kernels and pooling operations, the convolutional neural network effectively extracts edge information of contamination patches or ice contours and identifies changes in blade surface texture, such as local highlights or increased roughness caused by ice.

[0051] Based on this, principal component analysis (PCA) is used to reduce the dimensionality of high-dimensional feature vectors, concentrating key features onto a few principal components, thereby reducing computational burden and noise interference. In practical applications, for example, image resolution can be set between 640×480 and 1280×720, and convolutional neural networks can use ResNet or MobileNet architectures. The extracted features typically have dimensions ranging from hundreds to thousands. PCA is then used to compress these features to 20 to 50 dimensions, which are then combined with other sensor features to form the subsequent integrated sensor feature data. When contamination or ice accumulates on the blade surface, the texture features in the image data significantly deviate from normal operating conditions. The convolutional neural network automatically extracts and labels these abnormal patterns in the feature map. PCA then makes these abnormal patterns separable in a low-dimensional space, laying the foundation for subsequent multimodal fusion and anomaly identification.

[0052] The concept of multi-source data fusion is particularly crucial in applications requiring identification of special operating conditions such as blade contamination and icing. When blades are iced, surface temperature and humidity monitoring data will show a trend of long-term low temperatures and high humidity. White or gray ice marks may appear in the image data, and the vibration signal may also exhibit additional vibration frequency bands caused by the attached ice layer.

[0053] The integrated multi-sensor layout ensures that strain, temperature, and image information are captured at the blade root, mid-section, and tip. Combined with a discrete wavelet transform and a Kalman filter, this method highlights high-frequency vibration components caused by ice accumulation. A sliding average filter and convolutional neural network, combined with principal component analysis, can reveal clues to frost accumulation based on environmental parameters and image features.

[0054] Ultimately, this information is uniformly represented in the integrated sensor feature data. A multimodal autoencoder then extracts the latent feature vectors, and a physical information neural network performs constraint mapping to produce predicted blade force data. If the ice layer continues to thicken, the additional load on the blades will increase, and the predicted blade force data will exhibit an abnormal distribution that differs significantly from normal. At this stage, the fusion network maps these differences into a blade health assessment vector, triggering alarms or de-icing actions during hierarchical reinforcement learning decisions.

[0055] Similarly, increased surface roughness and decreased aerodynamic performance caused by blade contamination can be identified using the same approach. If environmental monitoring data indicates persistent dry weather, while image data shows dust or sand coverage, vibration data may exhibit slight imbalances in the low and mid-frequency bands, while fiber strain data shows no significant stress changes, the system will comprehensively identify contamination.

[0056] When it is detected that the degree of blade contamination or icing exceeds the safety threshold, hierarchical reinforcement learning will combine the health assessment vector with long-term operating condition information to make a load reduction or shutdown decision, and promptly feedback the situation to the operation and maintenance team for cleaning or de-icing operations. Doing so can significantly reduce the accumulation of damage to blades in harsh environments and improve the overall operating efficiency and safety of wind turbines. Combining the sensor layout, preprocessing method and multimodal fusion principle mentioned above, it can be seen that the present invention has extremely strong practical value and scalability in the identification of complex working conditions of long and flexible blades. It can not only accurately detect common anomalies such as icing and contamination, but also adapt to blade fault monitoring needs in more special scenarios through the same framework.

[0057] The comprehensive sensor feature data is input into a multimodal autoencoder to extract potential feature vectors, and combined with offline finite element simulation data, the potential feature vectors are mapped into predicted blade force data using a physical information neural network under preset physical constraints.

[0058] In the present invention, by inputting the comprehensive sensor feature data into a multimodal autoencoder and combining it with offline finite element simulation data, a physical information neural network is used to map the predicted blade force data under preset physical constraints, thereby realizing real-time monitoring and status evaluation of long flexible blades under various complex working conditions.

[0059] The core idea is to first use multi-source sensors (including fiber optic strain, vibration, environmental monitoring, and image data) to form a set of comprehensive sensor feature data, then use a multimodal autoencoder to mine deep potential feature vectors, and finally combine the physical information neural network to predict the force distribution of the potential features. In this way, we can fully utilize the spatiotemporal complementarity of different types of sensors, and ensure that the prediction results have engineering credibility under physical constraints. When applied to the identification needs of special working conditions such as blade contamination and icing, the potential feature vectors extracted by the multimodal autoencoder will more prominently show the abnormal information of the blade surface or internal structure. For example, icing will show obvious texture changes in image features, bring additional high-frequency components in vibration features, and cause uneven local stress distribution in fiber optic strain data.

[0060] By comparing these abnormal features with offline finite element simulation data, the physical information neural network can automatically identify whether the abnormality exceeds the normal range, thereby outputting the actual force state distribution of the blade. If it is found that the force distribution deviates significantly from the normal working conditions, the system will mark this situation as an icing or pollution alarm and make corresponding adjustments in the subsequent control strategy. In order to make this process operational in engineering applications, the present invention will establish a set of finite element simulation libraries with wide coverage based on typical wind speed, temperature, humidity and other parameters in the offline stage, and retrieve or interpolate the corresponding simulation data according to the current sensor characteristics in the online stage, and cooperate with the physical information neural network to perform force mapping on the potential features.

[0061] Since long and flexible blades are subject to interference from multi-dimensional external factors during wind turbine operation, it is often difficult to accurately distinguish between icing, contamination, or other structural damage if relying solely on a single sensor. Therefore, the combination of multi-source fusion and physical constraints in the present invention is particularly critical. In actual application scenarios, the system can be deployed on an edge computing platform for fast computing, or the core data can be uploaded to the cloud for large-scale analysis, and the results can be returned to the wind turbine master control system for adjustment. Regardless of the deployment mode, the cooperation of multimodal autoencoders and physical information neural networks can achieve timely identification and safety assessment of special blade working conditions.

[0062] Preferably, the multimodal autoencoder is provided with a plurality of data processing branches, which are respectively used to process vibration data, optical fiber strain data, environmental monitoring data and image data, wherein the vibration data branch adopts a one-dimensional convolutional network combined with a long short-term memory network to extract and encode time series features of the vibration data; the optical fiber strain data branch adopts a one-dimensional convolutional network to extract local strain features and uses a fully connected network to integrate features; the environmental monitoring data branch adopts a multi-layer perceptron to process the temperature, humidity and wind speed data to extract time-varying features; the image data branch adopts a pre-trained convolutional neural network to extract edge and texture features in the image and output a low-dimensional feature vector; the feature vectors output by each branch are dynamically weighted and fused in the middle layer through a cross-modal attention mechanism to generate a potential feature vector to comprehensively represent the blade state information.

[0063] A multimodal autoencoder plays a key role in this invention. It features multiple data processing branches, each dedicated to processing vibration data, fiber strain data, environmental monitoring data, and image data. Each branch uses a deep learning model and its own adapted network structure to extract key features from the corresponding modality. These features are then fused at an intermediate layer using a cross-modal attention mechanism to generate a latent feature vector that comprehensively reflects the blade's state.

[0064] In the vibration data branch, a combination of a one-dimensional convolutional network and a long short-term memory network was employed to better capture the dynamic characteristics of the blades in the time and frequency domains. The one-dimensional convolutional network can locally perceive and extract features from the vibration signal, while the long short-term memory network can track the evolution of the blade's vibration characteristics over time, thereby highlighting vibration patterns that may be related to icing and contamination. The fiber optic strain data branch uses a one-dimensional convolutional network to extract the local strain distribution characteristics of the blades. The strain characteristics at different locations are then integrated through a fully connected network. This approach can restore the stress differences on the blade surface as much as possible, given the limited number of sensing points.

[0065] If blades experience stress concentration or uneven distribution due to contamination or icing, this will be reflected in the feature output of this branch. The environmental monitoring data branch uses a multi-layer perceptron to process time-varying parameters such as temperature, humidity, and wind speed. These parameters often exhibit significant extremes or abnormal distributions in icing and contamination conditions. For example, in an icy environment, temperatures are chronically low and humidity is relatively high, while in contaminated environments, there may be prolonged periods of dry, low wind speeds and high dust indices. The image data branch uses a pre-trained convolutional neural network to extract edge and texture features from blade surface images and output a low-dimensional feature vector to characterize localized damage, icing contours, or contamination patches on the blade surface.

[0066] When the feature vectors of all branches converge at the middle layer, the cross-modal attention mechanism dynamically weights each modal feature according to its importance to ensure that the abnormal information of the key modality occupies a greater weight in the fusion vector. For example, if the current blade is iced and the image features are abnormally significant, then the attention weight of the image features will be relatively increased during the weighting process, and the final generated latent feature vector will be able to highlight this abnormality, thereby making accurate judgments for the subsequent physical information neural network. This fusion method of a multi-branch network and cross-modal attention can not only stably extract the basic features of the blade under normal working conditions, but also sensitively capture abnormal signals under special working conditions such as icing and pollution, ensuring the system's adaptability to different scenarios.

[0067] Preferably, the physical information neural network adopts physical constraints during the training process, establishes a blade dynamics model based on offline finite element simulation data to simulate the stress and vibration response of the blade under different wind speed, temperature and humidity conditions, and uses a regression loss function to optimize the neural network to ensure that the output predicted blade force data is consistent with the actual force distribution.

[0068] In this paper, the physical information neural network performs force mapping and constraint correction on potential eigenvectors. To ensure that the neural network's output more closely reflects real-world physical laws, the system employs a physical constraint strategy based on offline finite element simulation data during training. This refers to the fact that when training the neural network, the system not only relies on conventional prediction errors to update parameters, but also incorporates the blade's actual stress and vibration response data under various wind speed, temperature, humidity, and other operating conditions into the loss function calculation.

[0069] Specifically, the stress distribution and vibration mode of the blade under various environmental conditions are simulated through offline finite element simulation data. Then, when training the neural network, the force data predicted by the network is compared with the benchmark values ​​under the corresponding working conditions in the simulation library, and the deviation is minimized in the form of a regression loss function. This can effectively prevent the network from overfitting without physical basis, and can also ensure that when the blade encounters extreme situations such as pollution or icing, the network can still give reasonable stress and vibration prediction results in a physical sense. In order to achieve this, it is necessary to cover a sufficiently rich range of simulation conditions in the training set, such as different wind speeds (from low wind to extremely high wind), different temperatures (including icing caused by low temperature and material softening caused by high temperature), different humidity (high salt fog and dust environment at sea), etc., to ensure that the network has sufficient generalization ability when facing real scenes.

[0070] During network training, each input latent feature vector corresponds to one or more simulation condition labels. If the network's output of the predicted force distribution differs significantly from the simulation label, the system will impose a higher penalty in the regression loss. If the network performs poorly under certain extreme conditions, more simulation data from those conditions can be added for enhanced training. This allows the network to accurately predict the force distribution when the blades experience contamination or icing similar to those in the simulation library during actual operation, relying on previously learned physical laws. This provides a solid basis for subsequent evaluation and control.

[0071] In applications requiring identification of special operating conditions such as blade contamination and icing, the collaborative work of this multimodal autoencoder and the physical information neural network can effectively improve the system's monitoring accuracy and practicality. The multimodal autoencoder is responsible for high-level fusion of various sensor features, while the physical information neural network maps this fused latent feature vector into a force prediction space using offline simulation data and physical constraints, and appropriately corrects for force changes under abnormal operating conditions.

[0072] In this collaborative working model, when ice forms on the blade surface, the image branch captures differences in ice edges and textures, the environmental monitoring branch detects deviations from normal temperature and humidity ranges, and the vibration and fiber strain branches output abnormal dynamic features. Ultimately, through a cross-modal attention mechanism, the weights of these abnormalities are amplified, making them prominently reflected in the latent feature vector. Subsequently, the physical information neural network determines whether the anomaly is caused by icing by comparing stress-strain relationships with blade modal characteristics from a simulation library of low-temperature, high-humidity conditions. If icing is the case, the network predicts significant deviations from normal blade force distribution, such as additional loads in certain areas or an overall force increase exceeding safety limits. For pollution scenarios, if dust or salt coverage on the surface degrades aerodynamic performance, the image features will show large dark spots or blurred areas. The environmental branch may indicate stable wind speed but poor air quality, and the vibration branch may show slight energy increases in the low and medium frequency bands. Combined with the simulation data, the network generates corresponding force predictions, indicating areas of concentrated force or the magnitude of the overall force increase.

[0073] During the training phase, the system can add several simulation scenarios for different degrees of icing and contamination conditions, allowing the network to gradually master the force distribution characteristics under various abnormal scenarios during learning, and quickly map the potential characteristics during actual operation. If the monitoring results show that the stress level at certain key locations continues to exceed the limit, the system will input the results into the next health assessment and control decision module to take active de-icing, cleaning or load reduction measures to avoid further damage to the blades or a significant decline in the operating efficiency of the wind turbine. In this way, the combination of multimodal autoencoders and physical information neural networks is no longer just a passive monitoring of the blades, but a comprehensive system with early warning and accurate identification capabilities. It provides a highly reliable protection method for wind turbines in complex and changing environments, and also provides a scientific basis for the decision-making of operation and maintenance personnel.

[0074] The predicted blade force data is fused with the integrated sensor feature data to generate a blade health assessment vector, and hierarchical reinforcement learning is used to generate a control decision based on the health assessment vector. The control decision is used to adjust the blade state, and the control decision model is updated online through a closed-loop feedback mechanism.

[0075] By fusing predicted blade force data with integrated sensor feature data to generate a blade health assessment vector, and using hierarchical reinforcement learning to generate control decisions based on this assessment vector, the present invention enables dynamic state monitoring and precise control scheduling during the actual operation of long and flexible blades. Overall, this process aims to integrate optical fiber strain data, raw vibration data, environmental monitoring data, and image data collected by multi-source sensor nodes. The integrated sensor feature data obtained through preliminary processing and feature extraction is combined with the predicted blade force data output by the physical information neural network, and then fused into a blade health assessment vector within a multimodal deep fusion network.

[0076] This evaluation vector not only characterizes the blade's operating characteristics under normal operating conditions but also detects abnormal behavior under special conditions such as icing and contamination, providing key input for subsequent hierarchical reinforcement learning. If the evaluation indicates a blade presents a potential risk or has entered a dangerous threshold, the system adjusts the control strategy based on the evaluation vector, issuing commands such as load reduction, yaw adjustment, or active de-icing to minimize blade damage accumulation in extreme environments and ensure overall turbine efficiency.

[0077] During this process, the closed-loop feedback mechanism is responsible for real-time sampling of the execution effect and comparing the actual response of the blade with the predicted data. If the deviation continues to expand, it will trigger the online adaptive update mechanism to dynamically correct the parameters of the hierarchical reinforcement learning. This self-correction mechanism can effectively improve the system's adaptability to complex environments, enabling it to maintain high monitoring accuracy and control effects under various extreme conditions such as icy weather, marine salt spray environment or dust pollution. Combining the core concepts of the aforementioned multi-source data fusion and physical information neural network, the present invention has good scalability and stability in engineering applications, and is particularly suitable for long-term health management of large wind turbines at high altitudes, at sea or in arid and dusty environments.

[0078] Preferably, the predicted blade force data and the integrated sensor feature data are input into a multimodal deep fusion network together. The multimodal deep fusion network uses convolutional layers, fully connected layers and long short-term memory networks to encode and extract features from each input data, thereby generating a unified blade health assessment vector, which reflects the blade health status and current risk level.

[0079] Multimodal deep fusion networks play a crucial role in fusing predicted blade force data with integrated sensor feature data to generate a blade health assessment vector. These networks typically employ various structural units, including convolutional layers, fully connected layers, and long short-term memory networks, to encode and extract features from the input force data and integrated sensor feature data, respectively.

[0080] In application scenarios, working conditions such as pollution or icing often have cross-influences on different modal data: for example, local icing on the blade will not only change the texture in the image, but also cause asymmetric strain distribution in the optical fiber strain data. The vibration signal may increase energy in a specific frequency band, and the environmental monitoring data will also show abnormal combinations of temperature and humidity. The multimodal deep fusion network can capture this cross-modal coupling relationship, and perform local perception of data with spatial or temporal structures such as images and vibrations through the convolution layer. The feature vectors extracted by different branches are integrated into the same vector space through the fully connected layer, and the long short-term memory network is used to track the dynamic evolution of the time series. The advantage of this is that when the blade surface is iced or polluted, abnormal signals will appear in various modal data at the same time, and the fusion network can extract these anomalies from the multimodal features, thereby forming significant feature items in the health assessment vector.

[0081] In order to make this process feasible in engineering, several intermediate layers can be set in the fusion network to align the timestamps and resolutions of each mode, and perform appropriate normalization or alignment processing based on the data differences of the blades at different wind speeds and different rotation angles. Through this global information integration, the present invention can detect the linked changes in image texture and temperature and humidity in the early stages of freezing, and accurately distinguish surface contamination features caused by dust, salt spray or oil in pollution scenarios. The blade health assessment vector obtained in this way will mark the risk level and anomaly type in detail, providing a clear data basis for the subsequent reinforcement learning decision module.

[0082] Preferably, the multimodal deep fusion network adopts an adaptive weighting algorithm when generating a blade health assessment vector, and dynamically weightedly fuses the predicted blade force data input from the physical information neural network output with the environmental monitoring data, optical fiber strain data and image feature data contained in the comprehensive sensor feature data to achieve an accurate assessment of the blade health status and risk level.

[0083] To maximize the advantages of each modal data when generating the blade health assessment vector, the present invention employs an adaptive weighting algorithm to dynamically weight the predicted blade force data output by the physical information neural network and the integrated sensor feature data. This algorithm adjusts the weight of each modality based on its contribution to the monitoring results, its confidence level, and its historical performance.

[0084] For example, in a blade icing scenario, image features may best reflect the icing form and extent, environmental monitoring data can provide background information on low temperatures and high humidity, and fiber optic strain data can reveal areas of stress concentration. If image features are detected as highly significant during actual operation, and environmental and force data also corroborate the possibility of icing, the adaptive weighting algorithm will assign higher weights to these modalities, thereby highlighting the icing risk in the assessment vector.

[0085] On the contrary, if the environmental data indicates that the temperature and humidity are normal, and the optical fiber strain and vibration data are relatively stable, then even if there is a small amount of false positives caused by noise in the image features, it will not be over-amplified by the system. In practical applications, for example: the algorithm will calculate the variance and cross-correlation indicators of each mode in real time in the middle layer of the fusion network, and dynamically update the weights based on the historical accuracy to ensure that it can adapt to changes in sensor status and fluctuations in the external environment in a timely manner during long-term monitoring. The embodiment shows that when the blades are slightly frozen and accompanied by light dust pollution, the image branch and the vibration branch are often the most capable of detecting anomalies. The environmental branch only provides auxiliary judgment when the temperature fluctuates greatly. The different feature distributions brought about by these two anomalies can be effectively distinguished through adaptive weighting, thereby outputting a more accurate health assessment vector.

[0086] Preferably, Figure 2 As shown in the figure, the hierarchical reinforcement learning includes low-level decision-making and high-level decision-making, wherein the low-level decision-making is used to respond to short-term abnormalities in blade status in real time and issue adjustment instructions for instantaneous vibration or drastic changes in strain, while the high-level decision-making combines the long-term health assessment vector with the predicted blade force data to formulate a global control strategy, and realizes the collaborative work between the low-level and high-level decision-making through the feedback mechanism.

[0087] In the control decision-making phase of the present invention, hierarchical reinforcement learning is used to generate and execute blade state adjustment commands. This hierarchical reinforcement learning consists of low-level and high-level decision-making. The low-level decision-making is responsible for quickly responding to short-term blade anomalies, while the high-level decision-making formulates a global control strategy based on the long-term health assessment vector and predicted blade force data.

[0088] When blade ice is detected, low-level decisions can quickly reduce the engine speed or activate local heating based on the surge in vibration and strain characteristics to prevent further spread of the ice. High-level decisions, after assessing overall wind conditions, determine whether to shut down the engine for de-icing or implement other protective measures. In pollution scenarios, low-level decisions can also instantaneously suppress vibration peaks, while high-level decisions may incorporate blade health assessment vectors to determine when to perform cleaning or maintenance to minimize power generation losses.

[0089] Such a hierarchical structure not only ensures the safety of the system within a short time scale, but also maintains the overall efficiency of the wind turbine within a long period. In order to better coordinate the two layers of decision-making, the present invention also introduces a feedback mechanism in hierarchical reinforcement learning, that is, the low-level and high-level decisions will exchange execution results and monitoring data with each other so that a comprehensive judgment can be made when making the next decision. For example, if the high-level instruction requires maintaining low-speed operation, but the low-level detects that the blade surface has completed de-icing, it will feedback this status to the high-level, thereby prompting the high-level to adjust the speed back to normal in the next stage to avoid losses caused by excessive conservatism.

[0090] Preferably, the closed-loop feedback mechanism collects feedback data in real time after executing the control decision, compares the feedback data with the predicted data to calculate the deviation, and optimizes the parameters of the hierarchical reinforcement learning through an online adaptive update mechanism to ensure real-time adjustment and long-term stability of the blade state.

[0091] After executing a control decision, the system activates a closed-loop feedback mechanism to collect feedback data in real time. This data is then compared and analyzed with the predicted data to calculate deviations and determine whether the system's performance meets expectations. If the deviation continues to increase or fluctuates significantly under certain critical conditions, an online adaptive update mechanism is triggered to modify the parameters of the hierarchical reinforcement learning, allowing the decision-making strategy to dynamically adapt to the current icing, contamination, or other unexpected abnormal scenarios.

[0092] At the end of each monitoring cycle, the actual vibration energy, strain distribution, image changes, and differences between environmental data and predicted data are recorded, and then the strategy is iteratively optimized using the reward function of reinforcement learning. When local heating is applied to the blades in the early stages of icing but the effect is not significant, the deviation analysis will show that there are still abnormal stress or vibration peaks. The system will then increase the weight of the heating intensity or angle adjustment strategy through online updates, and then try a new action plan in the next cycle. This closed-loop feedback mechanism ensures that the system has the ability to self-learn and self-adjust to environmental changes. In particular, under extreme working conditions, the optimal or suboptimal control solution can be found through multiple iterations, which can both avoid excessive loads on the blades and maximize the use of wind energy.

[0093] In pollution scenarios, if the system detects increased dust accumulation on the blade surface during extended periods of dry, low wind speeds, and that a single cleaning is ineffective, it may adjust the maintenance cycle or recommend preventive cleaning through an online update mechanism to ensure high blade health and turbine availability. By integrating an adaptively weighted multimodal deep fusion network, hierarchical reinforcement learning, and closed-loop feedback, this invention provides a complete monitoring and control process for long-flexible blades in complex and changing climates, enabling efficient and accurate identification and response to special operating conditions such as icing and pollution.

[0094] like Figure 3 As shown, a long flexible blade monitoring system based on multi-source data fusion is used to implement the long flexible blade monitoring method based on multi-source data fusion. The system includes the following units:

[0095] Sensor node units, used to collect optical fiber strain data, vibration raw data, environmental monitoring data and image data on wind turbine blades;

[0096] The sensor node units are deployed at key locations on the wind turbine blades, usually selecting areas such as the blade root, middle section and blade tip, so as to fully cover the entire length of the blade. Each node integrates a fiber optic strain sensor, a MEMS accelerometer and inertial measurement unit, an environmental monitoring sensor and an image sensor. Although the present invention also requires the arrangement of sensor nodes on the wind turbine blades to collect data, the arrangement and integration design of its sensor nodes have been optimized compared to the prior art. Traditional methods often require the installation of an additional single sensor on the blades, while the present invention uses integrated multi-source sensor nodes that can be embedded in the manufacturing process or installed in a non-invasive manner, thereby significantly reducing the negative impact on the blade structure and aerodynamic performance. At the same time, by utilizing advanced data preprocessing and multimodal fusion technology, the present invention can simplify the calibration process and reduce reliance on the high precision of individual sensors, thereby reducing maintenance difficulty and detection costs. In terms of hardware design, the sensor nodes utilize industrial-grade modules. Their fiber-optic strain sensors utilize high-precision fiber Bragg grating technology to capture the internal stress distribution and micro-deformations of the blades in real time. MEMS accelerometers and inertial measurement units (IMUs) are used to detect blade vibration characteristics, with sampling frequencies typically set between 1 kHz and 5 kHz to meet the requirements for collecting high-frequency vibration data. Environmental monitoring sensors primarily collect temperature, humidity, and wind speed data, which are critical for determining blade icing or contamination. Image sensors utilize high-definition cameras or infrared thermal imagers to capture the blade surface condition and obtain real-time visual information on blade contamination, icing, or damage. During hardware integration, each sensor is synchronized at the millisecond level using a unified clock synchronization device (e.g., using GPS signals or IEEE1588 precision clocks). This ensures consistent timestamps for data collected by different sensors, facilitating subsequent data alignment and fusion. Furthermore, the sensor nodes are typically powered by the wind turbine system's internal power management unit (PMU), and feature a waterproof and dustproof design to ensure long-term stable operation in harsh outdoor environments. In actual application scenarios, these sensor nodes can maintain high reliability of data collection under high-speed rotation of wind turbine blades and strong wind conditions, providing accurate basic data for subsequent processing.

[0097] a data preprocessing unit, configured to perform a discrete wavelet transform on the collected data to extract time-frequency characteristics of the vibration signal, remove noise from the vibration signal using a Kalman filter, process environmental monitoring data using a sliding average filter, and process image data using a pretrained convolutional neural network and principal component analysis technology, thereby generating comprehensive sensor feature data;

[0098] The data preprocessing unit serves as the front-end data processing module of the system, and usually uses an embedded processor, a digital signal processor or a field programmable gate array (FPGA) to implement high-speed data operations. The unit first performs format conversion and time-series alignment on the raw data collected from the sensor nodes. For the raw vibration data, the preprocessing unit uses discrete wavelet transform to perform multi-level decomposition of the signal using a preset wavelet basis to extract the time-frequency characteristics of the signal. The discrete wavelet transform can capture local changes in the vibration signal at different scales, so that when the blades are frozen or contaminated, the abnormal frequency bands that appear in the vibration signal can be accurately extracted. Afterwards, in response to the high-frequency noise that may exist in the vibration data, the preprocessing unit uses a Kalman filter to perform real-time smoothing on the vibration signal by establishing a state-space model, thereby eliminating errors caused by environmental interference or sensor noise. For environmental monitoring data, temperature, humidity, and wind speed data can experience dramatic, instantaneous fluctuations during the acquisition process. To prevent these fluctuations from affecting subsequent analysis, the preprocessing unit employs a sliding average filter, calculating the average of continuous data within a set time window to obtain stable environmental parameters. For example, during wind speed data acquisition, the window length can be set to 5 seconds, and a smooth curve is formed by averaging all collected values ​​within that 5-second window. For image data, the preprocessing unit utilizes an embedded GPU or dedicated acceleration chip to run a pretrained convolutional neural network to extract edge and texture features from the blade surface in the image. Principal component analysis is then used to reduce the high-dimensional features, resulting in output image feature data that accurately reflects the blade surface condition. These preprocessing operations ensure a high signal-to-noise ratio for all data types and lay a solid foundation for subsequent multimodal data fusion. In terms of hardware, the data preprocessing unit is typically integrated into the wind turbine edge computing platform, supporting real-time data stream processing and short-term caching, ensuring that collected data is preprocessed and transmitted to subsequent modules within tens of milliseconds.

[0099] a multimodal autoencoder unit for processing the integrated sensor feature data through each branch separately and fusing them through a cross-modal attention mechanism to generate a latent feature vector;

[0100] The multimodal autoencoder unit primarily runs on edge computing platforms or servers. Its core function is to perform deep feature extraction on preprocessed integrated sensor feature data. This unit utilizes a deep neural network architecture with multiple data processing branches, each of which independently encodes vibration data, fiber strain data, environmental monitoring data, and image data. The vibration data branch extracts local temporal features using a one-dimensional convolutional network and captures the dynamic evolution of the signal using a long short-term memory network. The fiber strain data branch extracts local strain features using a one-dimensional convolutional network and then integrates data from various regions using a fully connected network. The environmental monitoring data branch uses a multi-layer perceptron to perform nonlinear mapping of collected environmental parameters to extract time-varying features. The image data branch processes the image data using a pretrained convolutional neural network, extracting key edge and texture information and outputting a low-dimensional feature vector. Each branch is dynamically weighted and fused at the intermediate layer using a cross-modal attention mechanism. This adaptively adjusts the weights based on the contribution of each modal data, thereby enhancing the representation of abnormal or key features in the fused latent feature vector. For example, when the blade surface exhibits significant texture changes due to icing, the attention weight of the image branch automatically increases, significantly increasing the sensitivity of the latent feature vector to the icing state. In terms of hardware, multimodal autoencoder units are often deployed on GPU-accelerated servers or edge devices, utilizing deep learning frameworks for model training and real-time inference. This ensures millisecond-level response times for latent feature vector generation, providing high-dimensional data representation for subsequent force prediction.

[0101] A physical information neural network unit is used to map the potential feature vector into predicted blade force data by combining offline finite element simulation data and preset physical constraints;

[0102] The primary task of the physical information neural network unit is to map the latent feature vectors generated by the multimodal autoencoder into predicted blade force data. To this end, this unit combines offline finite element simulation data with pre-set physical constraints to introduce physical constraints into the neural network training process. In terms of hardware, this unit typically runs on a server with high computing power. Its core lies in constructing a set of loss functions based on physical laws. Specifically, offline finite element simulation data is used to build a dynamic model of the blade under different wind speed, temperature, and humidity conditions, simulating the stress distribution and vibration response of the blade under actual operating conditions. During training, the physical information neural network matches the latent feature vectors with the simulation data. A regression loss function is used to calculate the difference between the predicted value and the simulated calibration value. Based on this difference, the network parameters are adjusted to ensure that the output predicted blade force data matches the actual force distribution. For example, in the simulation data, the nominal force value for a specific blade region under a certain operating condition is F0. If the network output is F, the regression loss function can be expressed as L = (F-F0)². The network continuously reduces L using gradient descent. In this process, physical constraints not only prevent the non-physical predictions that can occur in purely data-driven models, but also enable the model to maintain high prediction accuracy even in the face of abnormal operating conditions such as icing and contamination. In an embodiment, when ice forms on wind turbine blades in a low-temperature, high-humidity environment, the simulation data will reflect the force changes caused by the additional load of the ice layer. The physical information neural network adjusts the model parameters to achieve a high degree of consistency between the predicted and simulated data, thus providing a basis for safe regulation.

[0103] a multimodal deep fusion unit for fusing the predicted blade force data with the integrated sensor feature data, extracting features through a convolutional layer, a fully connected layer, and a long short-term memory network encoding, thereby generating a unified blade health assessment vector;

[0104] The multimodal deep fusion unit is responsible for fusing the predicted blade force data output by the physical information neural network with the preprocessed integrated sensor feature data to generate a unified blade health assessment vector. In hardware, this unit is typically deployed on an edge computing platform and employs a deep neural network architecture, using convolutional layers to extract spatial features, fully connected layers to integrate features, and long short-term memory networks to capture temporal variations. During the fusion process, the system dynamically weights each input data using an adaptive weighting algorithm to ensure that abnormal features in each modal data under different environmental conditions are appropriately amplified. For example, in a blade contamination scenario, dust coverage in the image data and abnormal wind speed indicators in the environmental monitoring data will be given higher weights, ensuring that the fused health assessment vector accurately reflects the adverse effects of contamination on blade health. In practice, GPUs can be used on edge devices to accelerate the real-time inference of the deep fusion network, enabling the output of the health assessment vector within hundreds of milliseconds, providing real-time data support for the control decision-making module.

[0105] A hierarchical reinforcement learning decision unit generates control decisions based on the blade health assessment vector, wherein the low-level decision unit responds to short-term anomalies in real time, and the high-level decision unit formulates a global control strategy to adjust the blade state;

[0106] The hierarchical reinforcement learning decision-making unit in the system is responsible for generating control decisions based on the blade health assessment vector. Its hardware platform is typically an industrial-grade computing device with a built-in real-time control module. The decision-making process employs a hierarchical reinforcement learning architecture, consisting of two parts: low-level and high-level decision-making. The low-level decision-making module rapidly responds to short-term anomalies in blade status. For example, if it detects a rapid change in vibration or strain within a very short period of time, it immediately generates local adjustment instructions, such as reducing the speed or activating local heating. The high-level decision-making module, based on the long-term health assessment vector and predicted blade force data, formulates an overall control strategy, such as adjusting the blade pitch angle, reducing turbine output power, or issuing a shutdown command to prevent further damage. During the decision-making process, the hierarchical reinforcement learning module leverages real-time historical state and motion data to continuously optimize its strategy using a deep reinforcement learning algorithm, ensuring high responsiveness and accuracy. The system connects to the wind turbine's main control system via industrial Ethernet, enabling real-time transmission and execution of decision instructions. In practical applications, simulation experiments can be used to verify the response time and decision accuracy of each layer of the decision-making module under different operating conditions. Network parameters are then adjusted based on feedback data, enabling online adaptive updates.

[0107] The closed-loop feedback unit is used to collect feedback data in real time after the control decision is executed, compare the feedback data with the predicted data to calculate the deviation, and adjust the parameters of the hierarchical reinforcement learning decision model through an online adaptive update mechanism.

[0108] The closed-loop feedback unit serves as the feedback and adaptive adjustment mechanism for the entire system. Its hardware is typically integrated with the hierarchical reinforcement learning module on the same control platform. It is responsible for collecting real-time feedback data after executing control decisions. This feedback data includes the latest observed values ​​of vibration, strain, image, and environmental parameters. By comparing it with the predicted data, the deviation between the actual and expected values ​​is calculated. If the deviation exceeds a preset threshold, the closed-loop feedback unit adjusts the parameters of the hierarchical reinforcement learning decision module through an online adaptive update mechanism to optimize the decision strategy. To ensure the accuracy of the feedback data, the closed-loop feedback unit may utilize a high-precision analog-to-digital converter and a real-time data transmission interface to achieve millisecond-level data acquisition and processing. In one embodiment, if the system detects that feedback data still shows abnormal vibration after localized heating of the blade, the closed-loop feedback unit triggers parameter adjustments, lowering the response threshold of the lower-level decision module or adjusting the weighting of the health assessment vector in the higher-level decision module to achieve more accurate control in the next cycle. This closed-loop feedback mechanism ensures that the system maintains efficient and stable operation under changing environmental conditions and provides real-time status monitoring and decision-making for long-term operation and maintenance.

[0109] In summary, this system, through its hardware-based high-precision sensor nodes, real-time data preprocessing platform, autoencoders combining multimodal deep learning with physical constraints, and hierarchical reinforcement learning decision-making and closed-loop feedback mechanisms, achieves comprehensive monitoring and precise control of long, flexible blades under special operating conditions such as contamination and icing. The system not only maintains real-time performance and accuracy, but also continuously improves its performance through online adaptive updates, providing a scientific and reliable means of health monitoring and early warning for wind turbine operations.

[0110] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware.

[0111] The above are merely embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should be included within the scope of the claims of the present application.

Claims

1. A long flexible blade monitoring method based on multi-source data fusion, characterized in that: The following steps are involved: By arranging multi-source sensor nodes on wind turbine blades, optical fiber strain data, vibration raw data, environmental monitoring data and image data are collected, and all data are pre-processed to generate comprehensive sensor feature data; Inputting the integrated sensor feature data into a multimodal autoencoder to extract potential feature vectors, and combining the data with offline finite element simulation data, using a physical information neural network under preset physical constraints to map the potential feature vectors into predicted blade force data; The multimodal autoencoder is provided with multiple data processing branches, which are respectively used to process vibration data, optical fiber strain data, environmental monitoring data and image data. The vibration data branch uses a one-dimensional convolutional network combined with a long short-term memory network to extract and encode time series features of the vibration data; the optical fiber strain data branch uses a one-dimensional convolutional network to extract local strain features and integrates features using a fully connected network; the environmental monitoring data branch uses a multi-layer perceptron to process temperature, humidity and wind speed data to extract time-varying features; the image data branch uses a pre-trained convolutional neural network to extract edge and texture features in the image and output a low-dimensional feature vector; the feature vectors output by each branch are dynamically weighted and fused in the middle layer through a cross-modal attention mechanism to generate a potential feature vector to comprehensively represent the blade state information; The predicted blade force data is fused with the integrated sensor feature data to generate a blade health assessment vector, and hierarchical reinforcement learning is used to generate a control decision based on the health assessment vector. The control decision is used to adjust the blade state, and the control decision model is updated online through a closed-loop feedback mechanism. The predicted blade force data and the integrated sensor feature data are input into a multimodal deep fusion network. The multimodal deep fusion network uses convolutional layers, fully connected layers, and long short-term memory networks to encode and extract features from each input data, thereby generating a unified blade health assessment vector. The assessment vector reflects the blade health status and current risk level. The multimodal deep fusion network adopts an adaptive weighting algorithm when generating a blade health assessment vector, and dynamically weightedly fuses the predicted blade force data output from the physical information neural network with the environmental monitoring data, optical fiber strain data, and image feature data contained in the integrated sensor feature data to achieve an accurate assessment of the blade health status and risk level.

2. The method according to claim 1, characterized in that The preprocessing includes applying discrete wavelet transform to the original vibration data, performing multi-level decomposition using a preset wavelet basis to extract the time-frequency characteristics of the signal, removing high-frequency noise in the vibration signal using a Kalman filter, and realizing signal smoothing processing using a state space model.

3. The method according to claim 1, characterized in that The preprocessing also includes using sliding average filtering to smooth the temperature, humidity and wind speed data of the environmental monitoring data to eliminate sudden fluctuations, and using a pre-trained convolutional neural network to extract image edge and texture features from the image data, and using principal component analysis technology to perform dimensionality reduction processing to generate image feature data.

4. The method according to claim 1, wherein The physical information neural network adopts physical constraints during the training process, establishes a blade dynamics model based on offline finite element simulation data to simulate the stress and vibration response of the blade under different wind speed, temperature and humidity conditions, and optimizes the neural network using a regression loss function to ensure that the output predicted blade force data is consistent with the actual force distribution.

5. The method according to claim 1, wherein The hierarchical reinforcement learning includes low-level decision-making and high-level decision-making. The low-level decision-making is used to respond to short-term abnormalities in blade status in real time and issue adjustment instructions for instantaneous vibration or drastic changes in strain, while the high-level decision-making combines long-term health assessment vectors with predicted blade force data to formulate a global control strategy, and realizes the collaborative work between low-level and high-level decision-making through a feedback mechanism.

6. The method according to claim 1, wherein The closed-loop feedback mechanism collects feedback data in real time after executing the control decision, compares the feedback data with the predicted data to calculate the deviation, and optimizes the parameters of the hierarchical reinforcement learning through an online adaptive update mechanism to ensure real-time adjustment and long-term stability of the blade state.

7. A long flexible blade monitoring system based on multi-source data fusion, used to implement the long flexible blade monitoring method based on multi-source data fusion according to any one of claims 1 to 6, characterized in that: The system includes the following units: Sensor node units, used to collect optical fiber strain data, vibration raw data, environmental monitoring data and image data on wind turbine blades; A data preprocessing unit is used to perform discrete wavelet transform on the collected data to extract the time-frequency characteristics of the vibration signal, remove noise from the vibration signal using a Kalman filter, process environmental monitoring data using a sliding average filter, and process image data using a pre-trained convolutional neural network and principal component analysis technology to generate comprehensive sensor feature data; a multimodal autoencoder unit for processing the integrated sensor feature data through each branch separately and fusing them through a cross-modal attention mechanism to generate a latent feature vector; A physical information neural network unit is used to map the potential feature vector into predicted blade force data by combining offline finite element simulation data and preset physical constraints; a multimodal deep fusion unit for fusing the predicted blade force data with the integrated sensor feature data, extracting features through a convolutional layer, a fully connected layer, and a long short-term memory network encoding, thereby generating a unified blade health assessment vector; A hierarchical reinforcement learning decision unit generates control decisions based on the blade health assessment vector, wherein the low-level decision unit responds to short-term anomalies in real time, and the high-level decision unit formulates a global control strategy to adjust the blade state; The closed-loop feedback unit is used to collect feedback data in real time after the control decision is executed, compare the feedback data with the predicted data to calculate the deviation, and adjust the parameters of the hierarchical reinforcement learning decision model through an online adaptive update mechanism.

Images