Long flexible blade monitoring method and system based on multi-source data fusion

Abstract

The invention relates to the technical field of wind power generation, in particular to a long flexible blade monitoring method and system based on multi-source data fusion, which adopts a multi-source data fusion technology, collects optical fiber strain, vibration, environment and image data by arranging sensor nodes on a fan blade, and generates comprehensive feature data through preprocessing; and extracting potential feature vectors by a multi-modal auto-encoder, and mapping the potential feature vectors into predicted stress data by using a physical information neural network in combination with offline finite element simulation data and physical constraint conditions. Then, a control decision is generated through the multi-modal deep fusion network and hierarchical reinforcement learning, the blade state is adjusted in real time, and the model is optimized through a closed-loop feedback mechanism; according to the invention, precise monitoring and active regulation and control of special working conditions such as blade pollution and icing are realized.

Description

Technical Field

[0001] The present invention relates to the technical field of wind power generation, and in particular to a long flexible blade monitoring method and system based on multi-source data fusion. Background Art

[0002] As a key load-bearing component in modern wind turbines, the health of long flexible blades directly affects the power generation efficiency and operational safety of wind turbines. Existing monitoring methods (Chinese invention patent, publication number: CN118361358A, name: Wind turbine blade load monitoring method based on transmission system vibration signal feature calibration) mostly rely on transmission system vibration signal feature calibration technology, by collecting gearbox vibration data, wind turbine speed and blade root stress, using fast Fourier transform, frequency band energy calculation and linear regression model to establish the relationship between blade root force and vibration signal, thereby realizing indirect monitoring of blade load.

[0003] However, this traditional method has obvious shortcomings: first, its monitoring means is single and only relies on the vibration signal of the transmission system, which makes it difficult to accurately reflect the actual force state of the blade under complex working conditions such as blade contamination and icing; second, the use of linear regression models to model the force characteristics of nonlinear and variable working conditions has limitations, and is prone to large deviations due to environmental noise, equipment installation errors and inconsistent data collection; in addition, the existing technology requires additional sensors to be installed on the blades, which may have a negative impact on the blade structure and aerodynamic performance, and the complex calibration process increases the difficulty of maintenance and detection costs. Based on this, there is an urgent need for a new monitoring method that can achieve real-time and accurate monitoring of the blade status through multi-source data fusion without interfering with the normal operation of the wind turbine, especially under special working conditions such as contamination and icing, with higher recognition accuracy and response speed. Summary of the invention

[0004] In view of the many problems existing in the above-mentioned prior art, the present invention provides a long flexible blade monitoring method and system based on multi-source data fusion. The present invention fuses the optical fiber strain, vibration, environment and image data collected by various sensors through advanced preprocessing and multimodal autoencoders, extracts representative potential feature vectors, and uses physical information neural networks to map them to predicted blade force data under preset physical constraints. Subsequently, a multimodal deep fusion and hierarchical reinforcement learning decision module is used to jointly analyze the predicted data and the comprehensive feature data to generate an assessment vector reflecting the health status and risk level of the blades, and then automatically generate a control decision to adjust the blade status. The present invention can achieve early warning through data fusion and physical constraints when the blades are contaminated or frozen, and timely adjust the working status of the fan blades, thereby ensuring efficient and stable operation of the fan.

[0005] A long flexible blade monitoring method based on multi-source data fusion comprises the following steps:

[0006] By arranging multi-source sensor nodes on the wind turbine blades, optical fiber strain data, vibration raw data, environmental monitoring data and image data are collected, and all data are pre-processed to generate comprehensive sensor feature data;

[0007] Inputting the comprehensive sensor feature data into a multimodal autoencoder to extract potential feature vectors, and combining with offline finite element simulation data, mapping the potential feature vectors into predicted blade force data using a physical information neural network under preset physical constraints;

[0008] The predicted blade force data is fused with the comprehensive sensor feature data to generate a blade health assessment vector, and hierarchical reinforcement learning is used to generate a control decision based on the health assessment vector. The control decision is used to adjust the blade state, and the control decision model is updated online through a closed-loop feedback mechanism.

[0009] Preferably, the preprocessing step includes applying discrete wavelet transform to the original vibration data, performing multi-level decomposition using a preset wavelet basis to extract the time-frequency characteristics of the signal, removing high-frequency noise in the vibration signal using a Kalman filter, and realizing signal smoothing processing using a state space model.

[0010] Preferably, the preprocessing step also includes using a sliding average filter to smooth the temperature, humidity and wind speed data of the environmental monitoring data to eliminate sudden fluctuations, and using a pre-trained convolutional neural network to extract image edge and texture features from the image data, and using principal component analysis technology to perform dimensionality reduction processing to generate image feature data.

[0011] Preferably, the multimodal autoencoder is provided with a plurality of data processing branches, which are respectively used to process vibration data, optical fiber strain data, environmental monitoring data and image data, wherein the vibration data branch adopts a one-dimensional convolutional network combined with a long short-term memory network to extract and encode time series features of the vibration data; the optical fiber strain data branch adopts a one-dimensional convolutional network to extract local strain features and integrates features using a fully connected network; the environmental monitoring data branch adopts a multi-layer perceptron to process the temperature, humidity and wind speed data to extract time-varying features; the image data branch adopts a pre-trained convolutional neural network to extract edge and texture features in the image and output a low-dimensional feature vector; the feature vectors output by each branch are dynamically weighted fused in the middle layer through a cross-modal attention mechanism to generate a potential feature vector to comprehensively represent the blade state information.

[0012] Preferably, the physical information neural network adopts physical constraints during the training process, establishes a blade dynamics model based on offline finite element simulation data to simulate the stress and vibration response of the blade under different wind speed, temperature and humidity conditions, and optimizes the neural network using a regression loss function to ensure that the output predicted blade force data is consistent with the actual force distribution.

[0013] Preferably, the predicted blade force data and the comprehensive sensor feature data are input into a multimodal deep fusion network together. The multimodal deep fusion network uses convolutional layers, fully connected layers and long short-term memory networks to encode and extract features from each input data, thereby generating a unified blade health assessment vector, which reflects the blade health status and current risk level.

[0014] Preferably, the multimodal deep fusion network adopts an adaptive weighted algorithm when generating a blade health assessment vector, and dynamically weighted fuses the predicted blade force data output from the physical information neural network with the environmental monitoring data, optical fiber strain data and image feature data contained in the comprehensive sensor feature data to achieve an accurate assessment of the blade health status and risk level.

[0015] Preferably, the hierarchical reinforcement learning includes low-level decisions and high-level decisions, wherein the low-level decisions are used to respond in real time to short-term abnormalities in blade status and issue adjustment instructions for instantaneous vibrations or drastic changes in strain, while the high-level decisions combine long-term health assessment vectors with predicted blade force data to formulate global control strategies, and achieve collaborative work between low-level and high-level decisions through a feedback mechanism.

[0016] Preferably, the closed-loop feedback mechanism collects feedback data in real time after executing the control decision, compares the feedback data with the predicted data to calculate the deviation, and optimizes the parameters of the hierarchical reinforcement learning through an online adaptive update mechanism to ensure real-time adjustment and long-term stability of the blade state.

[0017] A long flexible blade monitoring system based on multi-source data fusion is used to implement the long flexible blade monitoring method based on multi-source data fusion, and the system includes the following units:

[0018] Sensor node units, used to collect optical fiber strain data, vibration raw data, environmental monitoring data and image data on wind turbine blades;

[0019] A data preprocessing unit, configured to perform discrete wavelet transform on the collected data to extract time-frequency characteristics of the vibration signal, remove noise from the vibration signal using a Kalman filter, process environmental monitoring data using a sliding average filter, and process image data using a pre-trained convolutional neural network and principal component analysis technology, thereby generating comprehensive sensor feature data;

[0020] A multimodal autoencoder unit, for processing the comprehensive sensor feature data through each branch separately, and fusing them through a cross-modal attention mechanism to generate a potential feature vector;

[0021] A physical information neural network unit, used to map the potential feature vector into predicted blade force data by combining offline finite element simulation data and preset physical constraints;

[0022] A multimodal deep fusion unit, used to fuse the predicted blade force data with the comprehensive sensor feature data, extract features through convolutional layers, fully connected layers and long short-term memory network encoding, so as to generate a unified blade health assessment vector;

[0023] A hierarchical reinforcement learning decision unit generates a control decision based on the blade health assessment vector, wherein the low-level decision unit responds to short-term anomalies in real time, and the high-level decision unit formulates a global control strategy to adjust the blade state;

[0024] The closed-loop feedback unit is used to collect feedback data in real time after the control decision is executed, compare the feedback data with the predicted data to calculate the deviation, and adjust the parameters of the hierarchical reinforcement learning decision model through an online adaptive update mechanism.

[0025] Compared with the prior art, the advantages and beneficial effects of the present invention are:

[0026] The present invention utilizes sensor nodes arranged on the blades to simultaneously collect optical fiber strain data, vibration raw data, environmental monitoring data and image data, and generates comprehensive sensor feature data through preprocessing steps such as discrete wavelet transform, Kalman filtering, sliding average filtering, pre-trained convolutional neural network and principal component analysis;

[0027] The present invention uses a multimodal autoencoder branch network, a one-dimensional convolutional network, a long short-term memory network and a multi-layer perceptron to extract features from different modal data respectively, and realizes dynamic weighted fusion through a cross-modal attention mechanism to generate a potential feature vector;

[0028] The present invention combines offline finite element simulation data with preset physical constraints, uses physical information neural network to map potential feature vectors, and accurately predicts blade force data, thereby overcoming the limitation of traditional methods that rely only on a single vibration signal and avoiding monitoring errors caused by sensor noise and nonlinear working conditions.

[0029] The present invention realizes the automatic generation of blade health assessment and control decisions through a multimodal deep fusion network and a hierarchical reinforcement learning decision module, and performs online adaptive updates in combination with a closed-loop feedback mechanism. This technical means not only realizes the accurate identification of special working conditions such as blade contamination and icing, but also ensures the real-time and robustness of the system under dynamic working conditions, thereby significantly improving the safety of wind turbine operation and maintenance efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is a schematic diagram of the method flow in the present invention;

[0031] Figure 2 It is a schematic diagram of hierarchical reinforcement learning and closed-loop feedback decision-making of the present invention;

[0032] Figure 3 This is a system structure block diagram of the present invention. DETAILED DESCRIPTION

[0033] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is obvious that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0034] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "include", "comprising", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0035] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification, and should not be interpreted in an idealized or overly rigid manner.

[0036] like Figure 1 As shown, a long flexible blade monitoring method based on multi-source data fusion includes the following steps:

[0037] By arranging multi-source sensor nodes on the wind turbine blades, optical fiber strain data, vibration raw data, environmental monitoring data and image data are collected, and all data are pre-processed to generate comprehensive sensor feature data;

[0038] The present invention can simultaneously acquire optical fiber strain data, vibration raw data, environmental monitoring data, and image data by arranging multi-source sensor nodes on the wind turbine blades, and form comprehensive sensor feature data after a series of preprocessing of these data. This not only breaks the monitoring blind spot that may be caused by a single data source, but also provides a more targeted and timely analysis basis under complex working conditions such as blade contamination and blade icing.

[0039] The present invention rationally distributes sensors at different positions on the blade surface and inside the blade, so that the blade structure strain changes, vibration characteristics and external environmental conditions can be fully captured. In this process, each type of data will go through an independent collection channel to ensure information integrity and sampling synchronization, and then the data will be aligned and cleaned on the edge or central computing platform, ultimately providing a solid original basis for subsequent multi-modal fusion.

[0040] Because long and flexible blades are easily affected by multiple factors such as wind load, rain, snow, ice and dust at high altitudes, especially in extreme climates or remote offshore environments, surface contamination, icing, and wear of the blades often cause significant changes in the aerodynamic and force characteristics of the blades. If these changes cannot be monitored and identified in a timely manner, the overall operating efficiency of the wind turbine will be affected and even the safety of the system will be threatened.

[0041] To this end, the present invention fundamentally focuses on multi-source information fusion: on the one hand, optical fiber strain data can reflect the internal stress distribution of the blade body; on the other hand, the raw vibration data can show the mechanical coupling characteristics between the blade and the transmission system; in addition, environmental monitoring data can indicate the dynamic changes of external conditions such as temperature, humidity and wind speed, and image data can capture visual clues of surface contamination, icing or local damage of the blade. Aggregating this information within the same monitoring framework helps to grasp the health status of the blade at the overall level. The comprehensive sensor feature data obtained based on multi-source data fusion can also be further input into a multimodal autoencoder to extract higher-level feature representations by mining the potential correlations between various types of data, and then combined with offline finite element simulation data and physical constraints, the physical information neural network is used to map out the predicted blade force data, providing a basis for subsequent health assessment and decision-making control.

[0042] When the blades are in special working conditions such as contamination or icing, these sensor nodes will capture the blade surface temperature, strain abnormalities and image changes in the first place, and output feature quantities that can be fused through the preprocessing link, ultimately achieving accurate identification and real-time feedback of complex working conditions.

[0043] Preferably, the preprocessing step includes applying discrete wavelet transform to the original vibration data, performing multi-level decomposition using a preset wavelet basis to extract the time-frequency characteristics of the signal, removing high-frequency noise in the vibration signal using a Kalman filter, and realizing signal smoothing processing using a state space model.

[0044] In the preferred implementation of the present invention, a multi-level and multi-angle processing method is adopted for the original vibration data. The core idea is to use discrete wavelet transform and Kalman filter to deal with time-frequency resolution and noise suppression respectively. Discrete wavelet transform is a processing method that can perform localized analysis of signals in the time and frequency domains. The original vibration signal can be decomposed into several sub-bands using wavelet bases of different scales, thereby extracting time-frequency features related to the dynamic characteristics of the blade.

[0045] If the blade is contaminated or frozen, the blade vibration mode will have specific high-frequency components or waveform distortions. Wavelet decomposition can highlight these abnormal features on a certain sub-band and compare them with the baseline features under normal conditions. In order to accurately capture subtle changes when implementing wavelet transform, it is necessary to select a wavelet basis (such as Daubechies or Symlet) that matches the blade vibration characteristics, and the number of decomposition layers must also be considered in combination with the blade speed range and the number of structural modes. After completing the discrete wavelet decomposition, the high-frequency sub-bands obtained by decomposition often contain more noise components. In order to further eliminate environmental interference and sensor errors, the present invention uses a Kalman filter to smooth the high-frequency sub-bands.

[0046] The Kalman filter dynamically predicts and corrects the system state by establishing a state space model, so that the noise component is gradually attenuated in multiple iterations. For the identification of blade contamination or icing, if certain high-frequency features in the vibration subband continue to appear, and the Kalman filter is difficult to regard it as noise attenuation, it means that this feature is very likely to be a real vibration anomaly, which needs to be marked and passed to the subsequent fusion link. In practical applications, for example: for vibration signals with a sampling frequency in the range of 1kHz to 5kHz, the discrete wavelet transform is used to decompose them into four to six layers each time, and then the Kalman filter is applied to the high-frequency components of the top two layers. The prediction-update cycle can be set between 10 milliseconds and 20 milliseconds to take into account both computational efficiency and filtering effect. Through such multi-level processing, when the blade suffers from local vibration anomalies caused by icing or contamination, its features can be highly recognized and accurate in both the wavelet subband and the Kalman filter output.

[0047] Preferably, the preprocessing step also includes using a sliding average filter to smooth the temperature, humidity and wind speed data of the environmental monitoring data to eliminate sudden fluctuations, and using a pre-trained convolutional neural network to extract image edge and texture features from the image data, and using principal component analysis technology to perform dimensionality reduction processing to generate image feature data.

[0048] For environmental monitoring data and image data, the present invention also adopts a preferred processing strategy in order to provide more dimensional support in the identification of special working conditions. First, in terms of environmental monitoring, since the instantaneous fluctuations of temperature, humidity and wind speed may be affected by factors such as sudden weather and airflow disturbances, directly inputting them into the subsequent analysis module will lead to unstable results or even misjudgment. Therefore, the present invention uses a sliding average filtering method to smooth the environmental monitoring data, and by calculating the average value in a continuous time window, the extreme abnormal points are moderately smoothed, thereby obtaining a more stable temperature, humidity and wind speed curve.

[0049] If the temperature is monitored to be low for a long time and the humidity is high, combined with the change in wind speed, it can be preliminarily determined that the blades may be in an icing risk area; if the wind speed is monitored to be low but the dust or salt fog content is high, pollution accumulation may occur. In order to capture the specific pollution or icing form on the blade surface in the image data, the present invention further uses a pre-trained convolutional neural network to extract edge and texture features in the image.

[0050] Pre-training means completing preliminary feature learning on a general data set, and then in the present invention, only a small amount of migration training is required to adapt to the blade surface image. The convolutional neural network can effectively extract the edge information of contamination patches or ice contours through convolution kernels and pooling operations, and identify changes in blade surface texture, such as local highlights or increased roughness caused by ice.

[0051] On this basis, the principal component analysis technology is used to reduce the dimensionality of high-dimensional feature vectors, and the main features are concentrated on a few principal components, thereby reducing the computational burden and reducing noise interference. In practical applications, for example, the image resolution can be set between 640×480 and 1280×720, and the convolutional neural network can choose the ResNet or MobileNet structure. The extracted feature dimensions are usually hundreds to thousands, and then compressed to 20 to 50 dimensions using principal component analysis, and then combined with other sensor features to form subsequent comprehensive sensor feature data. When there is contamination or ice on the blade surface, the texture features in the image data will obviously deviate from the normal working conditions. In this regard, the convolutional neural network will automatically extract and mark these abnormal patterns in the feature map, and then the principal component analysis can make the abnormal pattern still separable in the low-dimensional space, laying the foundation for subsequent multimodal fusion and abnormality recognition.

[0052] In the application scenarios of the present invention for identifying special working conditions such as blade contamination and icing, the concept of multi-source data fusion is particularly critical. When ice occurs on the blade, the surface temperature and humidity monitoring data will show a trend of long-term low temperature and high humidity. At the same time, white or gray ice marks may appear in the image data, and the vibration signal may also show additional vibration frequency bands caused by the attached ice layer.

[0053] At this time, the overall multi-sensor layout can ensure that multiple information such as strain, temperature, and image can be obtained at the root, middle, and tip of the blade. The combined method of discrete wavelet transform and Kalman filter can highlight the high-frequency components of vibration caused by ice adhesion. Sliding average filtering and convolutional neural network combined with principal component analysis can find clues of frost accumulation in environmental parameters and image features.

[0054] Finally, after this information is uniformly represented in the integrated sensor feature data, the potential feature vector is extracted through a multimodal autoencoder, and the physical information neural network is used for constraint mapping to obtain the predicted blade force data. If the ice layer continues to thicken, the additional load on the blade will increase, and the predicted blade force data will also show an abnormal distribution that is significantly different from the normal situation. At this stage, the fusion network will map these differences into the blade health assessment vector, thereby triggering an alarm or de-icing operation during the hierarchical reinforcement learning decision.

[0055] Similarly, the increased surface roughness and reduced aerodynamic performance caused by blade contamination can also be identified by the same idea. If the environmental monitoring data reflects continued dry weather but the image data shows dust or sand coverage, the vibration data may show slight imbalance characteristics in the low and medium frequency bands, while the optical fiber strain data does not show a large stress change, then the system will comprehensively determine that it is contaminated.

[0056] When it is detected that the degree of blade contamination or icing exceeds the safety threshold, hierarchical reinforcement learning will combine the health assessment vector with long-term operating condition information to make a load reduction or shutdown decision, and promptly feedback the situation to the operation and maintenance team for cleaning or de-icing operations. Doing so can significantly reduce the accumulation of damage to blades in harsh environments and improve the overall wind turbine operating efficiency and safety. Combining the sensor layout, preprocessing method and multimodal fusion principle mentioned above, it can be seen that the present invention has extremely strong practical value and scalability in the identification of complex working conditions of long and flexible blades. It can not only accurately detect common anomalies such as icing and contamination, but also adapt to blade fault monitoring needs in more special scenarios through the same framework.

[0057] The comprehensive sensor feature data is input into a multimodal autoencoder to extract potential feature vectors, and combined with offline finite element simulation data, the potential feature vectors are mapped into predicted blade force data using a physical information neural network under preset physical constraints.

[0058] In the present invention, by inputting the comprehensive sensor feature data into the multimodal autoencoder and combining it with the offline finite element simulation data, a physical information neural network is used to map the predicted blade force data under preset physical constraints, so that real-time monitoring and status evaluation of long flexible blades under various complex working conditions can be achieved.

[0059] The core idea is to first use multi-source sensors (including fiber strain, vibration, environmental monitoring, image and other data) to form a set of comprehensive sensor feature data, then use multimodal autoencoders to mine deep potential feature vectors, and finally combine physical information neural networks to predict the force distribution of potential features. In this way, we can fully utilize the spatiotemporal complementarity of different types of sensors, and ensure that the prediction results have engineering credibility under physical constraints. When applied to the identification of special working conditions such as blade contamination and icing, the potential feature vectors extracted by the multimodal autoencoder will more prominently show the abnormal information of the blade surface or internal structure. For example, icing will show obvious texture changes in image features, bring additional high-frequency components in vibration features, and cause uneven local stress distribution in fiber strain data.

[0060] By comparing these abnormal features with offline finite element simulation data, the physical information neural network can automatically identify whether the abnormality exceeds the normal range, thereby outputting the actual force state distribution of the blade. If it is found that the force distribution deviates significantly from the normal operating conditions, the system will mark this situation as an icing or contamination alarm and make corresponding adjustments in the subsequent control strategy. In order to make this process operational in engineering applications, the present invention will establish a set of finite element simulation libraries with wide coverage based on typical wind speed, temperature, humidity and other parameters in the offline stage, and retrieve or interpolate the corresponding simulation data according to the current sensor characteristics in the online stage, and cooperate with the physical information neural network to perform force mapping on the potential features.

[0061] Since long and flexible blades are affected by multi-dimensional external factors during the operation of the wind turbine, it is often difficult to accurately distinguish between icing, pollution or other structural damage if only a single sensor is relied upon. Therefore, the combination of multi-source fusion and physical constraints of the present invention is particularly critical. In actual application scenarios, the system can be deployed on an edge computing platform to achieve fast computing, or it can upload core data to the cloud for large-scale analysis, and return the results to the wind turbine master control system for adjustment. Regardless of the deployment mode, the cooperation of multimodal autoencoders and physical information neural networks can achieve timely identification and safety assessment of special working conditions of blades.

[0062] Preferably, the multimodal autoencoder is provided with a plurality of data processing branches, which are respectively used to process vibration data, optical fiber strain data, environmental monitoring data and image data, wherein the vibration data branch adopts a one-dimensional convolutional network combined with a long short-term memory network to extract and encode time series features of the vibration data; the optical fiber strain data branch adopts a one-dimensional convolutional network to extract local strain features and integrates features using a fully connected network; the environmental monitoring data branch adopts a multi-layer perceptron to process the temperature, humidity and wind speed data to extract time-varying features; the image data branch adopts a pre-trained convolutional neural network to extract edge and texture features in the image and output a low-dimensional feature vector; the feature vectors output by each branch are dynamically weighted fused in the middle layer through a cross-modal attention mechanism to generate a potential feature vector to comprehensively represent the blade state information.

[0063] The multimodal autoencoder plays a key role in the present invention. It has multiple data processing branches, which are used to process vibration data, optical fiber strain data, environmental monitoring data and image data. Each branch extracts the key features of the corresponding modality based on the deep learning model and combines its own adaptive network structure, and then fuses them in the middle layer using the cross-modal attention mechanism to generate a potential feature vector that can comprehensively reflect the state of the blade.

[0064] In the vibration data branch, in order to better capture the dynamic characteristics of the blades in the time domain and frequency domain, a combination of a one-dimensional convolutional network and a long short-term memory network is used: the one-dimensional convolutional network can locally perceive and extract features of the vibration signal, and the long short-term memory network can track the evolution of the blade vibration characteristics over time, thereby highlighting vibration modes that may be related to icing and pollution. The optical fiber strain data branch uses a one-dimensional convolutional network to extract the local strain distribution characteristics of the blade, and then integrates the strain characteristics at different positions through a fully connected network. This method can restore the stress difference on the blade surface as much as possible under the premise of limited sensing points.

[0065] If the blade has stress concentration or uneven distribution when it is polluted or frozen, it will be reflected in the feature output of this branch. The environmental monitoring data branch uses a multi-layer perceptron to process time-varying parameters such as temperature, humidity, and wind speed. These parameters usually have obvious extremes or abnormal distributions in ice and pollution conditions. For example, in an ice environment, the temperature is long-term low and the humidity is relatively high, while in a polluted scene, there may be long periods of dry, low wind speeds and high dust indexes. The image data branch uses a pre-trained convolutional neural network to extract edge and texture features in the blade surface image and output a low-dimensional feature vector to characterize local damage, ice contours or pollution patches on the blade surface.

[0066] When the feature vectors of all branches converge at the middle layer, the cross-modal attention mechanism will dynamically weight each modal feature according to its importance to ensure that the abnormal information of the key modality occupies a larger weight in the fusion vector. For example, if the current blade icing causes the image feature to be abnormally significant, then the attention weight of the image feature will be relatively increased during the weighting process, and the potential feature vector finally generated will be able to highlight this abnormality, thereby making accurate judgments for the subsequent physical information neural network. Such a fusion method of a multi-branch network and cross-modal attention can not only stably extract the basic features of the blade under normal conditions, but also sensitively capture abnormal signals under special conditions such as icing and pollution, ensuring the system's adaptability to different scenarios.

[0067] Preferably, the physical information neural network adopts physical constraints during the training process, establishes a blade dynamics model based on offline finite element simulation data to simulate the stress and vibration response of the blade under different wind speed, temperature and humidity conditions, and optimizes the neural network using a regression loss function to ensure that the output predicted blade force data is consistent with the actual force distribution.

[0068] The physical information neural network in the present invention plays the role of force mapping and constraint correction of potential feature vectors. In order to make the output of the neural network closer to the real physical laws, the system adopts a physical constraint strategy based on offline finite element simulation data during the training process. The so-called physical constraint strategy means that when training the neural network, it not only relies on conventional prediction errors to update parameters, but also incorporates the real stress and vibration response data of the blade under different wind speeds, temperatures, humidity and other working conditions into the calculation of the loss function.

[0069] Specifically, the stress distribution and vibration mode of the blades under various environmental conditions are simulated through offline finite element simulation data. Then, when training the neural network, the force data predicted by the network is compared with the benchmark values ​​under the corresponding working conditions in the simulation library, and the deviation is minimized in the form of a regression loss function. This can effectively prevent the network from overfitting without physical basis, and can also ensure that when the blades encounter extreme situations such as pollution or icing, the network can still give reasonable stress and vibration prediction results in a physical sense. In order to achieve this, it is necessary to cover a sufficiently rich range of simulation conditions in the training set, such as different wind speeds (from low wind to extremely high wind), different temperatures (including icing caused by low temperature and material softening caused by high temperature), different humidity (high salt fog and dust environment at sea), etc., to ensure that the network has sufficient generalization ability when facing real scenes.

[0070] During network training, each input potential feature vector corresponds to one or more simulation condition labels. If the predicted force distribution output by the network differs too much from the simulation label, the system will give a higher penalty in the regression loss. If the network performs poorly under certain extreme conditions, more simulation data under the conditions can be added for enhanced training. In this way, when the blade encounters pollution or icing scenarios similar to those in the simulation library in actual operation, the network can rely on the previously learned physical laws to output accurate force distribution predictions, providing a solid basis for subsequent evaluation and control.

[0071] In the application scenarios where special working condition identification is required, such as blade contamination and icing, the collaborative work of the above multimodal autoencoder and physical information neural network can effectively improve the monitoring accuracy and practicality of the system. The multimodal autoencoder is responsible for high-level fusion of various sensor features, while the physical information neural network maps the fused potential feature vector to a force prediction space through offline simulation data and physical constraints, and reasonably corrects the force changes under abnormal working conditions.

[0072] In this collaborative working mode, when ice appears on the blade surface, the image branch will capture the difference in ice edge and texture, the environmental monitoring branch will find that the temperature and humidity deviate from the normal range, and the vibration branch and fiber strain branch will also output abnormal dynamic features. Finally, under the cross-modal attention mechanism, the weights of these abnormal information are amplified, so that they are significantly reflected in the potential feature vector. Subsequently, the physical information neural network can determine whether the anomaly is caused by ice by comparing the low temperature and high humidity conditions, stress-strain relationship and blade modal characteristics in the simulation library. If it is an icing condition, the network will predict that the distribution of blade force will deviate significantly from the normal situation, such as additional loads in certain areas, or the overall force increase exceeds the safety limit. For pollution scenarios, if the surface dust or salt coverage causes the aerodynamic performance to decrease, the image features will show large dark spots or blurred areas, the environmental branch may show an indication of stable wind speed and poor air quality, and the vibration branch may show a slight energy increase in the characteristics of the medium and low frequency bands. After combining the simulation data, the network will give the corresponding force prediction and indicate the area of ​​concentrated force or the increase in overall force.

[0073] During the training phase, the system can add several simulation scenarios for different degrees of icing and contamination conditions, so that the network can gradually master the force distribution characteristics under various abnormal scenarios during learning, and quickly map the potential characteristics during actual operation. If the monitoring results show that the stress level at certain key locations continues to exceed the limit, the system will input the results into the next health assessment and control decision module to take active deicing, cleaning or load reduction measures to avoid further damage to the blades or a significant decline in the operating efficiency of the wind turbine. In this way, the combination of multimodal autoencoders and physical information neural networks is no longer just a passive monitoring of the blades, but a comprehensive system with early warning and accurate identification capabilities, which provides a highly reliable protection method for wind turbines in complex and changing environments, and also provides a scientific basis for the decision-making of operation and maintenance personnel.

[0074] The predicted blade force data is fused with the comprehensive sensor feature data to generate a blade health assessment vector, and hierarchical reinforcement learning is used to generate a control decision based on the health assessment vector. The control decision is used to adjust the blade state, and the control decision model is updated online through a closed-loop feedback mechanism.

[0075] By fusing the predicted blade force data with the comprehensive sensor feature data to generate a blade health assessment vector, and using hierarchical reinforcement learning to generate control decisions based on the assessment vector, the present invention can achieve dynamic state monitoring and precise control scheduling in the actual operation of long flexible blades. Overall, this process aims to integrate the optical fiber strain data, vibration raw data, environmental monitoring data, and image data collected by multi-source sensor nodes, and combine the comprehensive sensor feature data obtained through preliminary processing and feature extraction with the predicted blade force data output by the physical information neural network, and then fuse them into a blade health assessment vector in the multimodal deep fusion network.

[0076] This evaluation vector can not only characterize the operating characteristics of the blade under normal conditions, but also keenly capture abnormal performance under special conditions such as icing and pollution, providing key input for subsequent hierarchical reinforcement learning. Once the evaluation results show that the blade has potential risks or has entered the dangerous threshold range, the system will adjust the control strategy according to the evaluation vector and issue instructions such as load reduction, yaw adjustment or active deicing, so as to minimize the damage accumulation of the blade in extreme environments and ensure the overall wind turbine efficiency.

[0077] During this process, the closed-loop feedback mechanism is responsible for real-time sampling of the execution effect and comparing the actual response of the blade with the predicted data. If the deviation continues to expand, it will trigger the online adaptive update mechanism to dynamically correct the parameters of the hierarchical reinforcement learning. This self-correction mechanism can effectively improve the system's adaptability to complex environments, enabling it to maintain high monitoring accuracy and control effects under various extreme conditions such as icy weather, marine salt spray environments, or sand and dust pollution. Combining the core concepts of the aforementioned multi-source data fusion and physical information neural networks, the present invention has good scalability and stability in engineering applications, and is particularly suitable for long-term health management of large wind turbines at high altitudes, at sea, or in arid and dusty environments.

[0078] Preferably, the predicted blade force data and the comprehensive sensor feature data are input into a multimodal deep fusion network together. The multimodal deep fusion network uses convolutional layers, fully connected layers and long short-term memory networks to encode and extract features from each input data, thereby generating a unified blade health assessment vector, which reflects the blade health status and current risk level.

[0079] In the process of fusing the predicted blade force data with the integrated sensor feature data to generate the blade health assessment vector, the multimodal deep fusion network plays a vital role. The network usually uses multiple structural units such as convolutional layers, fully connected layers, and long short-term memory networks to encode and extract features from the input force data and integrated sensor feature data, respectively.

[0080] In application scenarios, working conditions such as pollution or icing often have cross-effects on different modal data: for example, local icing on the blade will not only change the texture in the image, but also cause asymmetric strain distribution in the optical fiber strain data, the vibration signal may increase energy in a specific frequency band, and the environmental monitoring data will also show abnormal combinations of temperature and humidity. The multimodal deep fusion network can capture this cross-modal coupling relationship, perform local perception of data with spatial or temporal structures such as images and vibrations through the convolution layer, integrate the feature vectors extracted by different branches into the same vector space through the fully connected layer, and then use the long short-term memory network to track the dynamic evolution of the time series. The advantage of this is that when the blade surface is frozen or polluted, abnormal signals will appear in various modal data at the same time, and the fusion network can extract these anomalies from the multimodal features, thereby forming significant feature items in the health assessment vector.

[0081] In order to make this process feasible in engineering, several intermediate layers can be set in the fusion network to align the timestamps and resolutions of each mode, and perform appropriate normalization or alignment processing based on the data differences of the blades at different wind speeds and different rotation angles. Through this global information integration, the present invention can detect the linkage changes between image texture and temperature and humidity in the early stages of icing, and accurately distinguish surface contamination features caused by dust, salt spray or oil in pollution scenarios. The blade health assessment vector obtained in this way will mark the risk level and abnormality type in detail, providing a clear data basis for the subsequent reinforcement learning decision module.

[0082] Preferably, the multimodal deep fusion network adopts an adaptive weighted algorithm when generating a blade health assessment vector, and dynamically weighted fuses the predicted blade force data output from the physical information neural network with the environmental monitoring data, optical fiber strain data and image feature data contained in the comprehensive sensor feature data to achieve an accurate assessment of the blade health status and risk level.

[0083] While generating the blade health assessment vector, in order to give full play to the advantages of each modal data when fusion, the present invention particularly adopts an adaptive weighting algorithm to dynamically weight the predicted blade force data output from the physical information neural network and the comprehensive sensor feature data. The algorithm adjusts the weight of each modality according to its contribution, confidence and historical performance to the monitoring results.

[0084] For example, in the case of blade icing, image features may best reflect the icing form and range, environmental monitoring data can provide background information of low temperature and high humidity, and optical fiber strain data can show stress concentration areas. If image features are detected to be highly significant during actual operation, and environmental data and force data also confirm the possibility of icing, then the adaptive weighting algorithm will assign higher weights to these modes, thereby highlighting the icing risk in the evaluation vector.

[0085] On the contrary, if the environmental data indicates that the temperature and humidity are normal, and the optical fiber strain and vibration data are relatively stable, then even if the image features have a small amount of false positives caused by noise, they will not be over-amplified by the system. In practical applications, for example: the algorithm will calculate the variance and cross-correlation indicators of each mode in real time in the middle layer of the fusion network, and dynamically update the weights based on the historical accuracy to ensure that it can adapt to changes in sensor status and fluctuations in the external environment in a timely manner during long-term monitoring. The implementation example shows that when the blades are slightly frozen and accompanied by mild sand and dust pollution, the image branch and the vibration branch are often the most capable of detecting anomalies. The environmental branch only provides auxiliary judgment when the temperature fluctuates greatly. The different feature distributions brought about by these two anomalies can be effectively distinguished through adaptive weighting, thereby outputting a more accurate health assessment vector.

[0086] Preferably, Figure 2 As shown, the hierarchical reinforcement learning includes low-level decision-making and high-level decision-making, wherein the low-level decision-making is used to respond to short-term abnormalities in blade status in real time and issue adjustment instructions for instantaneous vibration or drastic changes in strain, while the high-level decision-making combines the long-term health assessment vector with the predicted blade force data to formulate a global control strategy, and realizes the collaborative work between the low-level and high-level decisions through a feedback mechanism.

[0087] In the control decision link of the present invention, hierarchical reinforcement learning is used to generate and execute the adjustment command of the blade state. The hierarchical reinforcement learning consists of low-level decision-making and high-level decision-making. The low-level decision-making is responsible for responding quickly to short-term abnormalities of the blade, while the high-level decision-making formulates a global control strategy based on the long-term health assessment vector and the predicted blade force data.

[0088] When blade ice is detected, low-level decisions will quickly reduce the speed or turn on local heating devices in a short period of time according to the surge in vibration and strain characteristics to prevent the ice from spreading further; high-level decisions will decide whether to shut down for de-icing or implement other protective measures after evaluating the overall wind conditions. In pollution scenarios, low-level decisions will also instantly suppress vibration peaks, while high-level decisions may combine blade health assessment vectors to determine when to clean or overhaul to minimize power generation losses.

[0089] Such a hierarchical structure not only ensures the safety of the system within a short time scale, but also maintains the overall efficiency of the wind turbine within a long period. In order to better coordinate the two layers of decision-making, the present invention also introduces a feedback mechanism in hierarchical reinforcement learning, that is, the low-level and high-level decisions will exchange execution results and monitoring data with each other so as to make a comprehensive judgment in the next decision. For example, if the high-level instruction requires maintaining low-speed operation, but the low-level detects that the blade surface has been de-iced, it will feedback this state to the high-level, thereby prompting the high-level to adjust back to normal speed in the next stage to avoid losses caused by excessive conservatism.

[0090] Preferably, the closed-loop feedback mechanism collects feedback data in real time after executing the control decision, compares the feedback data with the predicted data to calculate the deviation, and optimizes the parameters of the hierarchical reinforcement learning through an online adaptive update mechanism to ensure real-time adjustment and long-term stability of the blade state.

[0091] After executing the control decision, the system will start the closed-loop feedback mechanism to collect feedback data in real time, and compare and analyze these feedback data with the predicted data to calculate the deviation and determine whether the system's execution effect meets expectations. If the deviation continues to increase or fluctuates greatly under certain key conditions, the online adaptive update mechanism will be triggered to modify the parameters of the hierarchical reinforcement learning, so that the decision-making strategy can dynamically adapt to the current icing, pollution or other sudden abnormal scenarios.

[0092] At the end of each monitoring cycle, the actual vibration energy, strain distribution, image changes, and differences between environmental data and predicted data are recorded, and then the strategy is iteratively optimized using the reward function of reinforcement learning. When local heating is adopted for the blades in the early stage of icing but the effect is not significant, the deviation analysis will show that there are still abnormal stress or vibration peaks, so the system will increase the weight of the heating intensity or change the angle adjustment strategy through online updates, and then try new action plans in the next cycle. This closed-loop feedback mechanism ensures that the system has the ability to self-learn and self-adjust to environmental changes, especially in extreme working conditions, it can find the optimal or suboptimal control plan through multiple iterations, which can avoid excessive load on the blades and maximize the use of wind energy.

[0093] In a pollution scenario, if the system finds that dust accumulation on the blade surface is aggravated under long-term dry and low wind speed conditions, and the effect of a single cleaning is limited, the system may adjust the maintenance cycle or make preventive shutdown cleaning recommendations through an online update mechanism to ensure that the health of the blades and the availability of the wind turbine are maintained at a high level. Combining the close linkage of adaptive weighted multimodal deep fusion networks, hierarchical reinforcement learning, and closed-loop feedback, the present invention provides a complete monitoring and control process for long and flexible blades in complex and changeable climatic conditions, and is both efficient and accurate in identifying and responding to special working conditions such as icing and pollution.

[0094] like Figure 3 As shown, a long flexible blade monitoring system based on multi-source data fusion is used to implement the long flexible blade monitoring method based on multi-source data fusion, and the system includes the following units:

[0095] Sensor node units, used to collect optical fiber strain data, vibration raw data, environmental monitoring data and image data on wind turbine blades;

[0096] The sensor node units are deployed at key locations of the wind turbine blades, usually selecting areas such as the blade root, middle section and blade tip, so as to fully cover the entire length of the blade. Each node integrates a fiber optic strain sensor, a MEMS accelerometer and an inertial measurement unit, an environmental monitoring sensor and an image sensor. Although the present invention also requires the arrangement of sensor nodes on the wind turbine blades to collect data, the arrangement and integration design of its sensor nodes have been optimized compared to the prior art. Traditional methods often require the installation of an additional single sensor on the blades, while the present invention uses an integrated multi-source sensor node that can be embedded in the manufacturing process or installed in a non-invasive manner, thereby greatly reducing the negative impact on the blade structure and aerodynamic performance. At the same time, by utilizing advanced data preprocessing and multimodal fusion technology, the present invention can simplify the calibration process and reduce reliance on the high precision of individual sensors, thereby reducing maintenance difficulty and detection costs. In terms of hardware design, the sensor node uses industrial-grade modules. Its optical fiber strain sensor uses high-precision fiber Bragg grating technology to capture the internal stress distribution and micro-deformation of the blade in real time; MEMS accelerometers and inertial measurement units are used to detect the vibration characteristics of the blades. The sampling frequency is generally set between 1 kHz and 5 kHz to meet the requirements for collecting high-frequency vibration data; environmental monitoring sensors are mainly responsible for collecting temperature, humidity and wind speed data, which are key to determining whether the blades are frozen or contaminated; image sensors use high-definition cameras or infrared thermal imagers to capture the surface status of the blades and obtain visual information of blade contamination, icing or damage in real time. During hardware integration, each sensor achieves millisecond-level synchronization through a unified clock synchronization device (for example, using GPS signals or IEEE1588 precision clocks) to ensure that the timestamps of data collected by different sensors are consistent, thereby facilitating subsequent data alignment and fusion. In addition, the power supply of the sensor node usually uses the power management unit inside the fan system, and the waterproof and dustproof design meets the long-term stable operation in harsh outdoor environments. In actual application scenarios, these sensor nodes can maintain high reliability of data collection under high-speed rotation of wind turbine blades and strong wind conditions, providing accurate basic data for subsequent processing.

[0097] A data preprocessing unit, configured to perform discrete wavelet transform on the collected data to extract the time-frequency characteristics of the vibration signal, remove noise from the vibration signal using a Kalman filter, process environmental monitoring data using a sliding average filter, and process image data using a pre-trained convolutional neural network and principal component analysis technology, thereby generating comprehensive sensor feature data;

[0098] As the front-end data processing module of the system, the data preprocessing unit usually uses an embedded processor, a digital signal processor or a field programmable gate array (FPGA) to realize high-speed data operations. The unit first converts the format and aligns the timing of the raw data collected from the sensor node. For the raw vibration data, the preprocessing unit uses discrete wavelet transform to perform multi-level decomposition of the signal using a preset wavelet basis to extract the time-frequency characteristics of the signal. The discrete wavelet transform can capture the local changes of the vibration signal at different scales, so that when the blades are frozen or contaminated, the abnormal frequency bands in the vibration signal can be accurately extracted. Afterwards, for the high-frequency noise that may exist in the vibration data, the preprocessing unit uses a Kalman filter to perform real-time smoothing of the vibration signal by establishing a state-space model, thereby eliminating errors caused by environmental interference or sensor noise. For environmental monitoring data, since data such as temperature, humidity and wind speed may experience instantaneous and drastic fluctuations during the collection process, in order to avoid such fluctuations affecting subsequent judgments, the preprocessing unit uses a sliding average filtering method to calculate the average value of continuous data within a set time window to obtain stable environmental parameters; for example, in the process of wind speed data collection, the window length can be set to 5 seconds, and a smooth curve is formed by calculating the average of all collected values ​​within 5 seconds. For image data, the preprocessing unit uses an embedded GPU or a dedicated acceleration chip to run a pre-trained convolutional neural network to extract the edge and texture features of the blade surface in the image, and then combines the principal component analysis technology to reduce the dimensionality of high-dimensional features, so that the final output image feature data can concentrate on reflecting the surface state of the blade. These preprocessing operations not only ensure the high signal-to-noise ratio of various types of data, but also lay a solid foundation for subsequent multimodal data fusion. In terms of hardware, the data preprocessing unit is usually integrated in the wind turbine edge computing platform, supporting real-time data stream processing and short-term caching, ensuring that the collected data is preprocessed within tens of milliseconds and transmitted to subsequent modules.

[0099] A multimodal autoencoder unit, for processing the comprehensive sensor feature data through each branch separately, and fusing them through a cross-modal attention mechanism to generate a potential feature vector;

[0100] The multimodal autoencoder unit mainly runs on the edge computing platform or server, and its core function is to perform deep feature extraction on the preprocessed integrated sensor feature data. The unit adopts a deep neural network architecture with multiple data processing branches, each of which independently encodes vibration data, fiber strain data, environmental monitoring data and image data. The vibration data branch extracts local time series features through a one-dimensional convolutional network and uses a long short-term memory network to capture the dynamic evolution of the signal; the fiber strain data branch extracts local strain features through a one-dimensional convolutional network, and then uses a fully connected network to integrate data from various regions; the environmental monitoring data branch uses a multi-layer perceptron to perform nonlinear mapping on the collected environmental parameters and extract time-varying features; the image data branch uses a pre-trained convolutional neural network to process image data, extract key edge and texture information and output a low-dimensional feature vector. Each branch realizes dynamic weighted fusion through a cross-modal attention mechanism in the middle layer, that is, the system adaptively adjusts the weights according to the contribution of the current modal data, so that abnormal or key features are strengthened in the potential feature vector after fusion. For example, when the blade surface shows obvious texture changes due to icing, the attention weight of the image branch will automatically increase, which significantly increases the sensitivity of the latent feature vector to the icing state. In terms of hardware, multimodal autoencoder units are often deployed on servers or edge devices with GPU acceleration, using deep learning frameworks for model training and real-time inference to ensure that latent feature vectors are generated within millisecond response times, providing high-dimensional data expression for subsequent force prediction.

[0101] A physical information neural network unit, used to map the potential feature vector into predicted blade force data by combining offline finite element simulation data and preset physical constraints;

[0102] The main task of the physical information neural network unit is to map the potential feature vector generated by the multimodal autoencoder into predicted blade force data. To this end, this unit combines offline finite element simulation data and preset physical constraints to introduce a physical constraint mechanism in the neural network training process. In terms of hardware, this unit usually runs on a server with high computing power, and its core lies in building a set of loss functions based on physical laws. Specifically, offline finite element simulation data is used to establish a dynamic model of the blade under different wind speeds, temperatures and humidity conditions, and simulate the stress distribution and vibration response of the blade under actual working conditions. During the training process, the physical information neural network matches the potential feature vector with the simulation data, calculates the difference between the predicted value and the simulated calibration value through the regression loss function, and adjusts the network parameters on this basis to ensure that the output predicted blade force data conforms to the actual force distribution. For example, in the simulation data, the nominal force value of a specific area of ​​the blade under a certain working condition is F 0 , if the network output is F, the regression loss function can be expressed as L = (FF 0)2, the network continuously reduces the L value through the gradient descent method. In this process, the physical constraints not only prevent the non-physical predictions that may occur in pure data-driven models, but also enable the model to maintain a high level of prediction accuracy when facing abnormal conditions such as icing or pollution. In the embodiment, when ice appears on the fan blades in a low temperature and high humidity environment, the simulation data will reflect the force changes caused by the additional load of the ice layer, and the physical information neural network adjusts the model parameters to achieve a high degree of consistency between the predicted data and the simulation data, thereby providing a basis for safety regulation.

[0103] A multimodal deep fusion unit, used to fuse the predicted blade force data with the comprehensive sensor feature data, extract features through convolutional layers, fully connected layers and long short-term memory network encoding, so as to generate a unified blade health assessment vector;

[0104] The multimodal deep fusion unit is responsible for fusing the predicted blade force data output by the physical information neural network with the comprehensive sensor feature data generated by preprocessing to generate a unified blade health assessment vector. In terms of hardware, this unit is usually deployed on an edge computing platform, using a deep neural network architecture, using convolutional layers to extract spatial features, using fully connected layers to integrate features, and using long short-term memory networks to capture temporal variation features. During the fusion process, the system dynamically weights each input data through an adaptive weighting algorithm to ensure that the abnormal features of each modal data under different environmental conditions can be appropriately amplified. For example, in a blade pollution scenario, the dust coverage in the image data and the abnormal wind speed index in the environmental monitoring data will receive a higher weight, so that the fused health assessment vector accurately reflects the adverse effects of pollution on blade health. In actual operation, GPUs can be used on edge devices to accelerate the real-time inference of deep fusion networks, so that health assessment vectors can be output within hundreds of milliseconds, providing real-time data support for the control decision module.

[0105] A hierarchical reinforcement learning decision unit generates a control decision based on the blade health assessment vector, wherein the low-level decision unit responds to short-term anomalies in real time, and the high-level decision unit formulates a global control strategy to adjust the blade state;

[0106] The hierarchical reinforcement learning decision unit is responsible for generating control decisions based on the blade health assessment vector in the system. Its hardware platform is generally an industrial-grade computing device with a built-in real-time control module. The decision process adopts a hierarchical reinforcement learning structure, which consists of low-level decision and high-level decision. The low-level decision module responds quickly to short-term abnormalities in the blade state. For example, when vibration or strain changes dramatically in a very short time, it immediately generates local adjustment instructions, such as reducing the speed or starting local heating. The high-level decision module formulates an overall control strategy based on the long-term health assessment vector and predicted blade force data, such as adjusting the blade pitch angle, reducing the fan output power, or issuing a shutdown command to avoid further damage. In the decision-making process, the hierarchical reinforcement learning module uses the historical state and action data collected in real time to continuously optimize the strategy through the deep reinforcement learning algorithm to ensure that the decision has high response speed and accuracy. The system is connected to the fan main control system through industrial Ethernet to realize real-time transmission and execution of decision instructions. In practical applications, the response time and decision accuracy of each layer of decision modules under different working conditions can be verified through simulation experiments, and the network parameters can be adjusted according to the feedback data to achieve online adaptive updates.

[0107] The closed-loop feedback unit is used to collect feedback data in real time after the control decision is executed, compare the feedback data with the predicted data to calculate the deviation, and adjust the parameters of the hierarchical reinforcement learning decision model through an online adaptive update mechanism.

[0108] As the feedback and adaptive adjustment mechanism of the whole system, the closed-loop feedback unit is usually integrated with the hierarchical reinforcement learning module in the same control platform, and is responsible for real-time collection of feedback data after executing control decisions. These feedback data include the latest observations of vibration, strain, image and environmental parameters, which are compared with the predicted data to calculate the deviation between the actual and expected. If the deviation exceeds the preset threshold, the closed-loop feedback unit will adjust the parameters of the hierarchical reinforcement learning decision module through an online adaptive update mechanism to optimize the decision strategy. To ensure the accuracy of the feedback data, the closed-loop feedback unit may use a high-precision analog-to-digital converter and a real-time data transmission interface to achieve millisecond-level data collection and processing. In an embodiment, when the system detects that the feedback data still shows abnormal vibration after the blade is locally heated, the closed-loop feedback unit will trigger parameter adjustment, reduce the response threshold of the low-level decision module, or adjust the weight distribution of the high-level decision module to the health assessment vector, so as to achieve more accurate control in the next cycle. This closed-loop feedback mechanism ensures that the system can maintain efficient and stable operation in the face of changing environmental conditions, and provides real-time status monitoring and decision-making basis for long-term operation and maintenance.

[0109] In summary, this system achieves comprehensive monitoring and precise control of long and flexible blades under special conditions such as pollution and icing through high-precision sensor nodes in hardware, real-time data preprocessing platform, autoencoder combining multimodal deep learning with physical constraints, and hierarchical reinforcement learning decision-making and closed-loop feedback mechanism. The system not only has real-time performance and accuracy, but also continuously improves its own performance through online adaptive updates, providing a scientific and reliable health monitoring and early warning method for wind turbine operation.

[0110] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware.

[0111] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A long flexible blade monitoring method based on multi-source data fusion, characterized in that: The following steps are involved: By arranging multi-source sensor nodes on the wind turbine blades, optical fiber strain data, vibration raw data, environmental monitoring data and image data are collected, and all data are pre-processed to generate comprehensive sensor feature data; Inputting the comprehensive sensor feature data into a multimodal autoencoder to extract potential feature vectors, and combining with offline finite element simulation data, mapping the potential feature vectors into predicted blade force data using a physical information neural network under preset physical constraints; The predicted blade force data is fused with the comprehensive sensor feature data to generate a blade health assessment vector, and hierarchical reinforcement learning is used to generate a control decision based on the health assessment vector. The control decision is used to adjust the blade state, and the control decision model is updated online through a closed-loop feedback mechanism.

2. The method according to claim 1, characterized in that The preprocessing step includes applying discrete wavelet transform to the original vibration data, performing multi-level decomposition using a preset wavelet basis to extract the time-frequency characteristics of the signal, removing the high-frequency noise in the vibration signal using a Kalman filter, and realizing signal smoothing processing using a state space model.

3. The method according to claim 1, characterized in that The preprocessing step also includes using a sliding average filter to smooth the temperature, humidity and wind speed data of the environmental monitoring data to eliminate sudden fluctuations, and using a pre-trained convolutional neural network to extract image edge and texture features from the image data, and using principal component analysis technology to perform dimensionality reduction processing to generate image feature data.

4. The method according to claim 1, characterized in that: The multimodal autoencoder is provided with a plurality of data processing branches, which are respectively used to process vibration data, optical fiber strain data, environmental monitoring data and image data, wherein the vibration data branch adopts a one-dimensional convolutional network combined with a long short-term memory network to extract and encode time series features of the vibration data; the optical fiber strain data branch adopts a one-dimensional convolutional network to extract local strain features and integrates features using a fully connected network; the environmental monitoring data branch adopts a multi-layer perceptron to process the temperature, humidity and wind speed data to extract time-varying features; the image data branch adopts a pre-trained convolutional neural network to extract edge and texture features in the image and output a low-dimensional feature vector; the feature vectors output by each branch are dynamically weighted fused in the middle layer through a cross-modal attention mechanism to generate a potential feature vector to comprehensively represent the blade state information.

5. The method according to claim 1, characterized in that The physical information neural network adopts physical constraints during the training process, establishes a blade dynamics model based on offline finite element simulation data to simulate the stress and vibration response of the blade under different wind speed, temperature and humidity conditions, and optimizes the neural network using a regression loss function to ensure that the output predicted blade force data is consistent with the actual force distribution.

6. The method according to claim 1, characterized in that The predicted blade force data and the comprehensive sensor feature data are input into a multimodal deep fusion network together. The multimodal deep fusion network uses a convolutional layer, a fully connected layer and a long short-term memory network to encode and extract features from each input data, thereby generating a unified blade health assessment vector. The assessment vector reflects the blade health status and current risk level.

7. The method according to claim 6, characterized in that The multimodal deep fusion network adopts an adaptive weighting algorithm when generating a blade health assessment vector, and dynamically weighted fuses the predicted blade force data output from the physical information neural network with the environmental monitoring data, optical fiber strain data and image feature data contained in the comprehensive sensor feature data to achieve an accurate assessment of the blade health status and risk level.

8. The method according to claim 1, characterized in that The hierarchical reinforcement learning includes low-level decision-making and high-level decision-making. The low-level decision-making is used to respond to short-term abnormalities in blade status in real time and issue adjustment instructions for instantaneous vibration or drastic changes in strain, while the high-level decision-making combines long-term health assessment vectors with predicted blade force data to formulate a global control strategy, and realizes collaborative work between low-level and high-level decisions through a feedback mechanism.

9. The method according to claim 1, characterized in that: The closed-loop feedback mechanism collects feedback data in real time after executing the control decision, compares the feedback data with the predicted data to calculate the deviation, and optimizes the parameters of the hierarchical reinforcement learning through an online adaptive update mechanism to ensure real-time adjustment and long-term stability of the blade state.

10. A long flexible blade monitoring system based on multi-source data fusion, used to implement the long flexible blade monitoring method based on multi-source data fusion according to any one of claims 1 to 9, characterized in that: The system includes the following units: Sensor node units, used to collect optical fiber strain data, vibration raw data, environmental monitoring data and image data on wind turbine blades; A data preprocessing unit, configured to perform discrete wavelet transform on the collected data to extract time-frequency characteristics of the vibration signal, remove noise from the vibration signal using a Kalman filter, process environmental monitoring data using a sliding average filter, and process image data using a pre-trained convolutional neural network and principal component analysis technology, thereby generating comprehensive sensor feature data; A multimodal autoencoder unit, for processing the comprehensive sensor feature data through each branch separately, and fusing them through a cross-modal attention mechanism to generate a potential feature vector; A physical information neural network unit, used to map the potential feature vector into predicted blade force data by combining offline finite element simulation data and preset physical constraints; A multimodal deep fusion unit, used to fuse the predicted blade force data with the comprehensive sensor feature data, extract features through a convolutional layer, a fully connected layer and a long short-term memory network encoding, thereby generating a unified blade health assessment vector; A hierarchical reinforcement learning decision unit generates a control decision based on the blade health assessment vector, wherein the low-level decision unit responds to short-term anomalies in real time, and the high-level decision unit formulates a global control strategy to adjust the blade state; The closed-loop feedback unit is used to collect feedback data in real time after the control decision is executed, compare the feedback data with the predicted data to calculate the deviation, and adjust the parameters of the hierarchical reinforcement learning decision model through an online adaptive update mechanism.

Images