Wind turbine gearbox fault early warning system based on multi-modal collaborative transformer network

Abstract

The application discloses a wind turbine gearbox fault early warning system based on a multi-modal collaborative Transformer network, relates to wind turbine early warning technology, and is proposed in view of the problem that the early warning effect is not ideal in the prior art. The system comprises a four-path feature extraction module, an adaptive fusion module, a feature refinement module, and a multi-scale early fault warning module; Spearman correlation analysis is used to select multi-modal features related to the average oil temperature of the gearbox; the system has the advantages that maintenance early warning is provided based on the early warning mechanism, and the limitation that the threshold value set by artificial is easily affected by external fluctuations is broken. In specific implementation, 14 days of early warning can be effectively provided without false positives, and the system is especially suitable for harsh marine environments with high temperature, high humidity, and variable wind speed.

Description

Technical Field

[0001] This invention relates to wind turbine early warning technology, and more particularly to a wind turbine gearbox fault early warning system based on a multimodal cooperative Transformer network. Background Technology

[0002] Offshore wind power, as a clean and renewable energy source, has experienced rapid development in recent years. As a key component of semi-direct-drive offshore wind turbines, the reliable operation of the gearbox directly affects the safety and stability of the wind turbine generator. Furthermore, the harsh marine environment with its high temperature, high humidity, and variable wind speeds, coupled with the difficulty and high cost of offshore maintenance, makes early fault warning for gearboxes crucial. However, most existing early fault warning methods for wind turbines [1-2] rely primarily on threshold-based strategies for making maintenance decisions. This approach not only leads to delayed early fault alarms but also easily triggers false alarms, making it difficult to provide accurate and timely early fault warnings.

[0003] Currently, various artificial intelligence algorithms are applied to early fault warning of mechanical equipment such as wind turbine generators. For example, Li et al. [3] used a backpropagation neural network prediction model to predict the power of the wind turbine based on the predicted wind speed and compared the predicted power with the normal power operating range to achieve fault warning. Zhu et al. [4] used an expert system to analyze SCADA (Supervisory Control and Data Acquisition) system data to provide early warning of potential faults in wind turbine generators. Shi et al. [5] used an abnormal data processing method that combines density-based clustering algorithm and normal power range estimation, as well as a feature weight measurement optimization scheme based on Bayesian Optimization Algorithm (BOA) and tree model, to improve the efficiency and accuracy of intuitive mapping of SCADA system monitoring data to fault features. Gu et al. [6] used an improved method for early warning and identification of key components of wind turbine generators based on Siamese neural network, combining fault identification classification and discrimination labels with the integrated learning algorithm measurement of equipment fault warning and identification. Yang et al. [7] used a fault early warning method for wind turbine pitch system based on kernel density estimation (KDE), and used Bernoulli binomial distribution hypothesis test to determine abnormal operating state and provide early warning. Li et al. [8] used a statistical method for early warning of static error of wind turbine yaw control system to solve the problem of inaccurate windward surface. Chen et al. [9] used a wind turbine abnormal state early warning method based on principal component analysis and radial basis function network, used correlation analysis to screen input parameters that are strongly correlated with the unit output power, and established a model under normal operating conditions for comparative analysis. In addition, convolutional neural network (CNN), recurrent neural network (RNN), long short-term memory network (LSTM) and temporal convolutional network (TCN) have been used for time series prediction of key features of key components [10-14]. These existing fault early warning methods still mainly rely on the statistical empirical threshold of the prediction results to determine whether fault early warning and maintenance are needed. However, external interference can cause these predictive models to have insufficient predictive performance and pose challenges to accurate predictions over long periods of time.In addition, the fault warning strategy suffers from high false alarm rate and insufficient reliability, making it difficult to apply in the actual fault warning of offshore wind turbine gearboxes.

[0004] In recent years, the Transformer model has attracted great attention from researchers in fields such as industrial defect monitoring and fault early warning [15-18]. The Transformer utilizes a self-attention mechanism, which enables it to consider all positions in the input sequence at the same time, thereby better capturing long-range dependencies. Yang et al.

[16] used online multimodal feature fusion to enhance fault identification capabilities. Yang et al.

[17] proposed an inverted transformer network, which uses spatiotemporal dimension features to enhance the sequential learning capability of the transformer. Shuvro et al.

[18] adopted an improved Transformer architecture, used time-series vehicle data for traffic flow prediction, generated time series sequences from the dataset to capture time dependencies, and effectively modeled the correlation between features and samples. However, these methods are mainly used only for performance prediction of key indicators and have not been well integrated with fault early warning strategies, thus making it difficult to achieve effective early fault warning.

[0005] [1] Li J, He D, "Strategies of Sustainable Development in China's Wind Power Industry," 2020.

[0006] [2] MS Bridgman, “Relating failure prognostics to systembenefits,” Proceedings, IEEE Aerospace Conference, Big Sky, MT, USA, 2002, pp. 7–7.

[0007] [3] Y. Li, Z. Ma, J. Feng, R. Zhang, N. Fu and D. Liang, "FaultWarning and Reliability Analysis of Wind Turbine Failure Based on Data-driven," 2024 8th International Conference on Green Energy and Applications(ICGEA), Singapore, Singapore, 2024, pp. 28-32.

[0008] [4] Z. Guangwei, C. Sifan, R. Na and F. Shaonan, "Fault diagnosis andwarning design of wind turbines based on expert system," 2021 IEEE 4thInternational Conference on Automation, Electronics and ElectricalEngineering (AUTEEE), Shenyang, China, 2021, pp. 755-758.

[0009] [5] Y. Shi, Y. Liu and X. Gao, "Study of Wind Turbine Fault Diagnosisand Early Warning Based on SCADA Data," in IEEE Access, vol. 9, pp. 124600-124615, 2021.

[0010] [6] C. Gu and S. Yin, "Improved Siamese Neural Betwork Based onFeature Fusion for Wind Turbine Fault Warning and Identification of KeyComponents," 2024 IEEE PES 16th Asia-Pacific Power and Energy EngineeringConference (APPEEC), Nanjing, China, 2024, pp. 1-5.

[0011] [7] X. Yang, M. Yang, X. Zeng, Y. Zhu and Y. Zhou, "Fault warning ofpitch system of wind turbine based on kernel density estimation," 8thRenewable Power Generation Conference (RPG 2019), Shanghai, China, 2019, pp.1-5, doi: 10.1049 / cp.2019.0579.

[0012] [8] M. Li and H. Zhang, "A Statistic-Based Static Error WarningMethod for Wind Turbine Yaw-Control System," 2024 4th InternationalConference on Energy Engineering and Power Systems (EEPS), Hangzhou, China,2024, pp. 1187-1190.

[0013] [9] S. Chen, Y. Ma, L. Ma, F. Qiao and H. Yang, "Early warning ofabnormal state of wind turbine based on principal component analysis and RBFneural network," 2021 6th Asia Conference on Power and Electrical Engineering(ACPEE), Chongqing, China, 2021, pp. 547-551.

[0014]

[10] G. Yang, Y. Zhong, L. Yang and R. Du, "Fault Detection ofHarmonic Drive Using Multiscale Convolutional Neural Network," in IEEETransactions on Instrumentation and Measurement, vol. 70, pp. 1-11, 2020, Artno. 3502411, doi: 10.1109 / TIM.2020.3024355.

[0015]

[11] G. Yang, H. Tao, R. Du and Y. Zhong, "Wear Prediction ofPetrochemical Granulator Gearbox Using Multiscale Temporal ConvolutionalNetwork via Online Oil Monitoring," 2023 IEEE International Instrumentationand Measurement Technology Conference (I2MTC), Kuala Lumpur, Malaysia, 2023,pp. 1-6, doi: 10.1109 / I2MTC53148.2023.10175962.

[0016]

[12] G. Yang, H. Tao, R. Du and Y. Zhong, "Wear Prediction ofPetrochemical Granulator Gearbox Using Multidimensional Transformer NetworkVia Online Oil Monitoring," 2024 IEEE International Instrumentation andMeasurement Technology Conference (I2MTC), Glasgow, United Kingdom, 2024, pp.1-6, doi: 10.1109 / I2MTC60896.2024.10560569.

[0017]

[13] G. Yang, Y. Zhong, L. Yang, H. Tao, J. Li and R. Du, "FaultDiagnosis of Harmonic Drive With Imbalanced Data Using Generative AdversarialNetwork," in IEEE Transactions on Instrumentation and Measurement, vol. 70,pp. 1-11, 2021, Art no. 3519911, doi: 10.1109 / TIM.2021.3089240.

[0018]

[14] G. Yang, H. Tao, R. Du and Y. Zhong, "CompoundFault Diagnosis ofHarmonic Drives Using Deep Capsule Graph Convolutional Network," in IEEETransactions on Industrial Electronics, doi: 10.1109 / TIE.2022.3176280.

[0019]

[15] G. Yang, H. Tao, K. Wu, R. Du and Y. Zhong, "Fault Diagnosis ofHarmonic Drives Using Multimodal Collaborative Meta Network With SeverelyMissing Modality," in IEEE Transactions on Industrial Informatics, doi:10.1109 / TII.2024.3396339.

[0020]

[16] G. Yang, H. Tao, T. Yu, R. Du and Y. Zhong, "Online FaultDiagnosis of Harmonic Drives Using Semisupervised Contrastive GraphGenerative Network via Multimodal Data," in IEEE Transactions on IndustrialElectronics, vol. 71, no. 3, pp. 3055-3063, March 2024.

[0021]

[17] G. Yang, H. Tao, S. He, W. Feng, R. Du and Y. Zhong, "MultimodalTime Series Forecasting for Online Oil Monitoring of Petrochemical PelletizerGearbox Using Multiscale Inverted Transform Network," in IEEE Internet ofThings Journal, doi: 10.1109 / JIOT.2024.3514081.

[0022]

[18] AA Shuvro, MS Khan, M. Rahman, F. Hussain, M. Moniruzzamanand MS Hossen, "Transformer Based Traffic Flow Forecasting in SDN-VANET," in IEEE Access, vol. 11, pp. 41816-41826, 2023, doi: 10.1109 / ACCESS.2023.3270889. Summary of the Invention

[0023] The purpose of this invention is to provide a wind turbine gearbox fault early warning system based on a multimodal cooperative Transformer network, so as to solve the problems existing in the prior art.

[0024] The wind turbine gearbox fault early warning system based on a multimodal collaborative Transformer network described in this invention includes a four-path feature extraction module, an adaptive fusion module, a feature refinement module, and a multi-scale early fault warning module; Spearman correlation analysis is used to select multimodal features related to the gearbox average oil temperature;

[0025] The four-path feature extraction module adopts a four-path parallel architecture to perform collaborative feature extraction on the input multimodal temporal features from four dimensions, comprehensively capturing the time dependence, local patterns, global correlations and original information preservation of the gearbox's operating state;

[0026] The adaptive fusion module is used to adaptively integrate the output of the four-path feature extraction module, dynamically adjust the contribution weight of each path according to the characteristics of the input data, and achieve the optimal combination of multi-path features through a learnable attention mechanism.

[0027] The feature refinement module is used to further optimize the fused features through a cross-attention mechanism, thereby achieving fine-grained selection and enhancement at the feature level; it automatically identifies and strengthens the feature dimensions most relevant to the gearbox fault early warning task, while suppressing redundant or noisy features, thus improving the quality of feature representation and discrimination ability.

[0028] The multi-scale early fault warning module performs multi-scale prediction based on refined features and establishes an adaptive warning mechanism based on nonparametric kernel density estimation. It extracts high-level abstract features through a multi-level fully connected network and constructs a data-driven warning threshold by combining statistical learning methods to achieve intelligent fault warning.

[0029] The wind turbine gearbox fault early warning system based on a multimodal collaborative Transformer network described in this invention has the advantage of providing maintenance early warnings based on an early warning mechanism, overcoming the limitations of manually set thresholds being susceptible to external fluctuations. In practical implementation, it effectively provides early warnings up to 14 days in advance with no false alarms, making it particularly suitable for harsh marine environments characterized by high temperature, high humidity, and variable wind speeds. Attached Figure Description

[0030] Figure 1 This is a schematic diagram of the structure of the wind turbine gearbox fault early warning system described in this invention.

[0031] Figure 2 This is a diagram showing the Spearman correlation coefficients of the top 16 variables most relevant to the average oil temperature characteristics of the gearbox.

[0032] Figure 3 yes Figure 2 The diagram shows a visualization of the correlation between the 16 variables.

[0033] Figure 4 These are time series plots of the top 6 variables most relevant to the average oil temperature characteristics of the gearbox.

[0034] Figure 5 This is a graph showing the loss value of the multimodal cooperative Transformer network described in this invention.

[0035] Figure 6 This is a curve showing the prediction results of the gearbox oil temperature of an offshore wind turbine using the multimodal collaborative Transformer network described in this invention.

[0036] Figure 7 This is a comparison chart of the prediction results of the multimodal cooperative Transformer network described in this invention and other networks in the prior art.

[0037] Figure 8 This is a graph showing the residual values ​​of the multimodal collaborative Transformer network described in this invention.

[0038] Figure 9 This is a schematic diagram of the residual distribution of the multimodal collaborative Transformer network described in this invention.

[0039] Figure 10 This is a graph showing the early fault warning results of the gearbox described in this invention. Detailed Implementation

[0040] The wind turbine gearbox fault early warning system based on a multimodal collaborative transformer network (MCTN) described in this invention is as follows: Figure 1 As shown, the system includes a four-path feature extraction module, an adaptive fusion module, a feature refinement module, and a multi-scale early fault warning module. Spearman correlation analysis is used to select multimodal features related to the gearbox's average oil temperature.

[0041] The SCADA system for offshore wind turbines collects data from various multimodal sensors, including speed sensors, temperature sensors, pressure sensors, pressure switches, level switches, level gauges, gearbox lubricating oil monitoring sensors, nacelle vibration sensors, and anemometers. The data primarily includes: differential pressure of the gearbox main pump filter, generator winding / bearing temperature, generator average speed, current, torque, ambient temperature, air density, atmospheric humidity, and grid power. The gearbox lubricating oil monitoring sensors also collect data on: water content, metal particles, 4μm cleanliness, 14μm cleanliness, 6μm cleanliness, oil dielectric constant, oil conductivity (%), and oil temperature. The SCADA system records a total of 348 monitored variables, each assigned an ordinal variable identifier for correlation analysis.

[0042] Without loss of generality, the gearbox average oil temperature, denoted as variable_72, is closely related to the "gearbox oil over-temperature" fault alarm. Spearman correlation analysis was used to select multimodal features related to the gearbox average oil temperature. The correlation analysis results identified 16 of the most relevant features, such as... Figure 2As shown, correlation analysis of 348 variables revealed 191 positive correlations and 74 negative correlations. The strongest positive correlation was with variable_140 (1.0000), and the strongest negative correlation was with variable_83 (-0.8750). The 13 variables with the strongest positive correlation, ranked from highest to lowest correlation, are: average oil temperature of the main oil pump in the gearbox (variable_140), secondary oil temperature of the gearbox (variable_138), average oil temperature of the offline filter pump in the gearbox (variable_141), average temperature of the gearbox hub-side bearing (variable_74), average temperature of the gearbox generator-side bearing (variable_75), average temperature of the lubricating oil cooler inlet (variable_76), average temperature of the main bearing in the gearbox (variable_73), average temperature of the generator nacelle-side bearing (variable_149), average temperature of the transformer (variable_270), minimum hydraulic system oil temperature (variable_218), average hydraulic system oil temperature (variable_219), maximum hydraulic system oil temperature (variable_217), and average nacelle azimuth angle (variable_243). Furthermore, the three variables with the strongest negative correlation, ranked from highest to lowest correlation, are: the average pressure at the first outlet of the third gearbox generator lubrication pump (variable_132), the average pressure at the second outlet of the third gearbox generator lubrication pump (variable_133), and the average pressure at the fourth port of the gearbox (variable_83). A scatter plot matrix was further constructed to visualize the relationships between these 16 most correlated features, such as... Figure 3 As shown. Based on the above analysis, the time series data of the top 6 variables with the highest correlation are compared, for example... Figure 4 As shown, existing methods using thresholds are prone to triggering false alarms, and it is difficult to achieve effective early fault warnings based on fault symptoms. Therefore, the wind turbine gearbox fault warning system proposed in this invention can effectively prevent false alarms.

[0043] The specific steps for selecting and preprocessing the multimodal features are as follows:

[0044] S1. Spearman correlation analysis;

[0045] Spearman correlation coefficient was used to screen multimodal features that are strongly correlated with the gearbox mean oil temperature:

[0046] ;

[0047] R x , R Y These are the ordered columns of variables X and Y, respectively.

[0048] S2. Mini-Max Normalization;

[0049] The multimodal features selected through Spearman correlation analysis were normalized to eliminate differences in units and numerical ranges among different features, thereby improving the stability and convergence speed of model training. A min-max normalization method was used to linearly transform the original feature data to the [0,1] interval.

[0050] The normalization formula is as follows:

[0051] ;

[0052] x is the original feature value, representing the actual measured value of any monitored variable collected from the SCADA system at a specific time point; x min The minimum value of the feature represents the minimum observed value of the monitored variable within the training set time period; x max is the maximum value of the feature, representing the maximum observed value of the monitored variable within the training set time period; x* is the normalized feature value, representing the standardized value in the interval [0,1] obtained after linear transformation of the original feature value.

[0053] This normalization process normalizes all input features to the same numerical range, effectively avoiding model training instability caused by differences in feature scales, accelerating the convergence speed of gradient descent, and improving the fairness and accuracy of weight calculation for each path in the multi-path feature extraction module.

[0054] The four-path feature extraction module adopts a four-path parallel architecture to perform collaborative feature extraction on the input multimodal features from four dimensions, comprehensively capturing the time dependence, local patterns, global correlations and original information preservation of the gearbox's operating state.

[0055] The first path is a spatiotemporal path based on LSTM and Temporal Attention; this path is specifically designed to capture long-term dependencies and the contribution of important time steps in multimodal features. Temporal features are extracted using bidirectional LSTM, and then the importance of features at different time steps is adaptively weighted using a temporal attention mechanism.

[0056] The formula for calculating the hidden state of LSTM is:

[0057] ;

[0058] x t It is the input feature vector at time step t, with dimension d. in h t-1 It is the hidden state of the previous time step;

[0059] h tIt is the output of the hidden state at the current time step, with dimension d. hidden .

[0060] The formula for calculating the time attention weight is:

[0061] ;

[0062] The importance scoring function is defined as follows:

[0063] ;

[0064] h t Let be the LSTM hidden state vector at time step t; u is the learnable attention parameter vector; u* is the transpose of vector u; u*h t are vectors u and h t The importance score is obtained after the dot product operation; It is the normalized attention weight at the t-th time step.

[0065] The formula for weighted feature output is:

[0066] ;

[0067] Z lstm It is the final output feature vector of the spatiotemporal path, with dimension d. hidden .

[0068] The second path is the CNN-based local pattern path; this path focuses on extracting local short-term patterns and periodic features from time-series data in multimodal features, and captures local correlations at different time scales through multi-layer one-dimensional convolutional kernels.

[0069] The formula for convolution feature extraction is:

[0070] ;

[0071] X is the input time-series feature matrix with dimension 1. ; C is the convolution kernel weight parameter; C is the convolution output feature map with dimension 1. ;

[0072] The global feature aggregation formula is:

[0073] ;

[0074] Z cnn It is the final output feature vector of the local mode path, with dimension d. filters .

[0075] The third path is the global self-attention path; this path establishes global dependencies between all time steps through a self-attention mechanism, capturing long-distance temporal correlations and feature interactions.

[0076] The formula for calculating self-attention is:

[0077] ;

[0078] Q is the query matrix, K is the key matrix, and V is the value matrix, all obtained from the input X through linear transformation; K* is the transpose of K; d k It is the dimension of the key matrix, used to scale the dot product result; It is an attention-weighted feature representation.

[0079] The formula for global feature output is:

[0080] ;

[0081] Z attn It is the final output feature vector of the global self-attention path, with dimension d. model .

[0082] The fourth path is the residual connection path; this path preserves important information from the original input, ensures that gradients propagate stably in the network, prevents information loss, and enhances the model's generalization ability.

[0083] The formula for a fully connected mapping is:

[0084] ;

[0085] The flattened input feature vector has dimensions T×d. in ; This is the weight matrix of the fully connected layer; For bias terms; This is the final output feature vector of the residual connection path, with dimension d. res .

[0086] To ensure that subsequent modules can effectively integrate the features of each path, all path outputs are uniformly mapped to the same dimension through a fully connected layer. The mapping formula is as follows:

[0087] ;

[0088] It is a unified weight matrix for each path dimension; It is a bias term; These are path output feature vectors with unified dimensions, all having dimension d. uniform .

[0089] The four-path architecture works collaboratively through different feature extraction perspectives: the temporal attention path captures important time points, the local pattern path extracts short-term features, the global self-attention path establishes long-range dependencies, and the residual path preserves the original information, together forming a comprehensive and robust feature representation system.

[0090] The adaptive fusion module adaptively integrates the outputs of the four-path feature extraction module, dynamically adjusting the contribution weights of each path based on the characteristics of the input data. This module utilizes a learnable attention mechanism to achieve the optimal combination of multi-path features, overcoming the limitations of traditional fixed-weight fusion and enhancing the model's adaptability to different operating conditions.

[0091] The formula for calculating path importance weight is:

[0092] ;

[0093] It is the feature output vector of the i-th path (i=1,2,3,4 correspond to LSTM, CNN, Attention, and Residual paths respectively); v i It is the learnable weight vector of the i-th path; It is a vector v i The transpose of z is used with the eigenvector z. i Perform dot product operation; It is the normalized importance weight of the i-th path, and satisfies .

[0094] The formula for weighted feature fusion is:

[0095] ;

[0096] These are the importance weights of each path; These are the feature output vectors of each path; It is the fused feature vector, with dimensions consistent with the output dimensions of each path.

[0097] This module can automatically adjust the contribution ratio of the four paths based on the specific characteristics of the input samples. For data with strong periodicity, the CNN path may receive higher weights; for data that requires long-range dependency modeling, the attention path weights will increase, achieving true adaptive multimodal feature fusion.

[0098] The feature refinement module further optimizes the fused features through a cross-attention mechanism, achieving fine-grained selection and enhancement at the feature level. This module can automatically identify and enhance the feature dimensions most relevant to the gearbox fault early warning task, while suppressing redundant or noisy features, thereby improving the quality of feature representation and discriminative ability.

[0099] The formula for generating the feature selection mask is:

[0100] ;

[0101] It is the feature vector output by the dynamic fusion module; It is the feature selection weight matrix; It is a bias term; It is the sigmoid activation function, which compresses the output to the (0,1) interval; It is a feature selection mask vector, where each element represents the importance of the corresponding feature dimension.

[0102] The feature refinement formula is:

[0103] ;

[0104] It is element-wise multiplication; It is a feature selection mask vector, with dimensions of ... same; It is a refined feature vector, used to enhance important features and suppress redundant features.

[0105] By selecting features at a fine-grained level, this module can adaptively highlight sensitive features related to early gearbox failures, such as vibration characteristics at specific frequencies and temperature change patterns, while reducing interference from environmental noise and irrelevant features, thus significantly improving the model's early warning accuracy.

[0106] The multi-scale early fault warning module performs multi-scale prediction based on refined features and establishes an adaptive warning mechanism based on nonparametric kernel density estimation. High-level abstract features are extracted through a multi-layer fully connected network, and data-driven warning thresholds are constructed using statistical learning methods, achieving intelligent fault warning without human intervention.

[0107] The calculation formula for multi-scale prediction networks is:

[0108] ;

[0109] W1 is the feature vector output by the feature refinement module; W2, W3 are the weight matrices of each fully connected layer, with the dimensions decreasing progressively; b1, b2, b3 are the bias terms of each layer. It is a modified linear unit activation function; This is the final predicted output value.

[0110] The loss function uses mean squared error:

[0111] ;

[0112] It is the true value of the i-th sample; It is the predicted value of the i-th sample; It is the total number of training samples; It is the mean squared error loss value.

[0113] Nonparametric kernel density estimation includes the following calculations:

[0114] Residual sequence calculation: ;

[0115] Kernel density estimation: ;

[0116] Gaussian kernel function: ;

[0117] Early warning threshold determined: ;

[0118] It is the prediction residual of the i-th sample; It is the bandwidth parameter for kernel density estimation; It is a Gaussian kernel function; It is an estimate of the probability density function of the residual; It is the mean of the residuals during the training phase; It is the standard deviation of the residuals during the training phase; It is the upper limit threshold for early warning; It is the lower threshold for early warning.

[0119] This module achieves high-precision prediction through multi-scale neural networks and establishes an adaptive early warning mechanism by combining data-driven kernel density estimation methods. It can dynamically adjust the early warning threshold according to the actual operating status of the equipment, which significantly reduces the false alarm rate compared with the fixed threshold method and achieves accurate fault early warning up to 14 days in advance.

[0120] To verify the predictive performance of the MCTN model, a loss function was used to evaluate its predictive performance. The loss curves for the training and validation sets are shown below. Figure 5 As shown, the MCTN model converges in approximately 5 epochs, demonstrating excellent model performance and rapid convergence. Furthermore, to more intuitively illustrate the predictive effectiveness of the proposed model, the actual and predicted values ​​of the gearbox oil temperature during the testing phase are plotted, as shown below. Figure 6 As shown. From Figure 6 As can be seen, the MCTN model can accurately predict the future trend of gearbox oil temperature in the test set, including fluctuations in temperature data.

[0121] Several comparative examples are introduced to compare the performance of different prediction methods in existing technologies: To compare the time series prediction performance of different intelligent methods, Recurrent Neural Networks (RNNs), Convolutional Neural Networks (CNNs), Long Short-Term Memory Networks (LSTMs), Temporal Convolutional Networks (TCNs), and Transformer models are used for comparative analysis. The RNN method uses a two-layer stacked SimpleRNN structure for sequence processing and feature extraction. The CNN method uses a two-layer convolution and pooling architecture to extract local temporal features through convolutional kernels. The LSTM method uses a two-layer stacked LSTM architecture specifically designed to capture long sequence dependencies. The TCN method utilizes an dilated convolutional residual network with multi-scale dilation rates to expand the receptive field. The Transformer method is based on a self-attention mechanism architecture, fusing positional encoding and a feedforward network. The MCTN model proposed in this invention integrates a four-path collaborative fusion network, combining LSTM, CNN, attention, and residual paths. All models use uniform parameter configurations to ensure fair performance comparison; the input sequence length is set to 48, the prediction time limit is set to 1, the batch size is set to 4, the learning rate is set to 0.001, the number of training epochs is set to 100, the early termination patience value is set to 15, and the number of basic units is fixed at 64. Figure 7 The results show the comparison between the predicted and actual values ​​of these methods on the test set.

[0122] The root mean square error (RMSE), mean absolute error (MAE), and coefficient of determination (R²) were further used as comprehensive indicators to evaluate the predictive performance of the model. The prediction results of various comparison methods are shown in the table below:

[0123]

[0124] Smaller RMSE and MAE values ​​indicate better model prediction performance, while a larger R² value reflects stronger predictive ability. As shown in the table, the MCTN model proposed in this invention achieves the minimum RMSE and MAE values ​​while obtaining the maximum R² value, demonstrating the best prediction performance among all compared methods.

[0125] Regarding the early fault prediction results of nonparametric kernel density estimation: After obtaining an optimized model capable of accurately predicting future key characteristics, the prediction residuals are analyzed using nonparametric kernel density estimation. This allows for the acquisition of data distribution of wind turbine gearbox oil temperature residuals. Based on the Three Sigma rule, a reliable and safe operating range for key characteristics is established, such as... Figure 8 and Figure 9 As shown, when the residual consistently exceeds this range over a continuous period, an early failure can be identified. This mechanism provides maintenance early warnings, overcoming the limitation of manually set temperature thresholds being susceptible to external fluctuations.

[0126] In the test dataset, the residual statistics of the MCTN model are as follows: mean = -0.391593, standard deviation = 0.835867, minimum = -3.062435, maximum = 4.238422, skewness = 1.450490, kurtosis = 6.257344. The residual mean is -0.391593, and the residual standard deviation is 0.835867. The upper limit of three sigma is 2.116007, and the lower limit is -2.899192. Furthermore, the trained optimal model was applied to the latest data from the 20 days prior to the failure (this data was not used in any previous model training or testing). Figure 10 As shown, the warning results indicate that if the residual continuously exceeds the defined three sigma range, the state is judged as abnormal.

[0127] The monitoring methods currently used by technicians are highly prone to false alarms. For example... Figure 5 As shown, traditional threshold-based methods failed to accurately identify early fault characteristics before March 29, 2022. However, as... Figure 10 As shown, the proposed method provided a clear fault warning as early as March 15, 2022, 14 days earlier than the technician's inspection on March 29, and did not trigger any false alarms. This confirms the effectiveness of the present invention in early fault warning of semi-direct drive offshore wind turbine gearboxes.

[0128] In summary, this invention proposes a wind turbine gearbox fault early warning system based on a multimodal cooperative Transformer network (MCTN). Experimental results show that MCTN not only achieves excellent performance in multimodal data prediction but also enables accurate and reliable early fault warning.

[0129] This invention has at least the following technical advantages:

[0130] 1) The proposed MCTN model designs a multi-path time series prediction collaborative attention mechanism, which effectively improves prediction performance through multimodal attention collaboration and dynamic path fusion.

[0131] 2) The proposed MCTN model combines multimodal collaborative prediction with nonparametric kernel density estimation that directly reflects the inherent characteristics of the data, eliminating the dependence on threshold settings and enabling it to be directly and effectively applied to early fault warning.

[0132] 3) By utilizing multimodal collaborative Transform networks, the degradation trend of key features of offshore wind power systems can be accurately predicted, thus achieving precise and reliable early fault warning for offshore wind power systems.

[0133] For those skilled in the art, various other corresponding changes and modifications can be made based on the technical solutions and concepts described above, and all such changes and modifications should fall within the protection scope of the claims of this invention.

Claims

1. A wind turbine gearbox fault early warning system based on a multimodal cooperative Transformer network, characterized in that, It includes a four-path feature extraction module, an adaptive fusion module, a feature refinement module, and a multi-scale early fault warning module; Spearman correlation analysis is used to select multimodal features related to the gearbox average oil temperature; The four-path feature extraction module adopts a four-path parallel architecture to perform collaborative feature extraction on the input multimodal temporal features from four dimensions, comprehensively capturing the time dependence, local patterns, global correlations and original information preservation of the gearbox's operating state; The adaptive fusion module is used to adaptively integrate the output of the four-path feature extraction module, dynamically adjust the contribution weight of each path according to the characteristics of the input data, and achieve the optimal combination of multi-path features through a learnable attention mechanism. The feature refinement module is used to further optimize the fused features through a cross-attention mechanism, thereby achieving fine-grained selection and enhancement at the feature level; it automatically identifies and strengthens the feature dimensions most relevant to the gearbox fault early warning task, while suppressing redundant or noisy features, thus improving the quality of feature representation and discrimination ability. The multi-scale early fault warning module performs multi-scale prediction based on refined features and establishes an adaptive warning mechanism based on nonparametric kernel density estimation. High-level abstract features are extracted through multi-layer fully connected networks, and data-driven early warning thresholds are constructed by combining statistical learning methods to achieve intelligent fault early warning. In the multi-scale early fault warning module, the multi-scale prediction network calculation formula is as follows: ； W1 is the feature vector output by the feature refinement module; W2, W3 are the weight matrices of each fully connected layer, with the dimensions decreasing progressively; b1, b2, b3 are the bias terms of each layer. It is a modified linear unit activation function; This is the final predicted output value; The loss function uses mean squared error: ； It is the true value of the i-th sample; It is the predicted value of the i-th sample; It is the total number of training samples; This is the mean squared error loss value; Nonparametric kernel density estimation includes the following calculations: Residual sequence calculation: ; Kernel density estimation: ; Gaussian kernel function: ; Early warning threshold determined: ; It is the prediction residual of the i-th sample; It is the bandwidth parameter for kernel density estimation; It is a Gaussian kernel function; It is an estimate of the probability density function of the residual; It is the mean of the residuals during the training phase; It is the standard deviation of the residuals during the training phase; It is the upper limit threshold for early warning; It is the lower threshold for early warning.

2. The wind turbine gearbox fault early warning system based on a multimodal cooperative Transformer network according to claim 1, characterized in that, The specific steps for selecting and preprocessing the multimodal features are as follows: S1. Spearman correlation analysis; Spearman correlation coefficient was used to screen multimodal features that are strongly correlated with the gearbox mean oil temperature: ； R x , R Y are rank sequences of variables X and Y, respectively; S2. Mini-Max Normalization; The multimodal features selected through Spearman correlation analysis are normalized to eliminate the differences in units and numerical ranges between different features, thereby improving the stability and convergence speed of model training; the min-max normalization method is used to linearly transform the original feature data to the [0,1] interval. The normalization formula is as follows: ； x is the original feature value, representing the actual measured value of any monitored variable collected from the SCADA system at a specific time point; x min The minimum value of the feature represents the minimum observed value of the monitored variable within the training set time period; x max The maximum value of the feature represents the maximum observed value of the monitored variable within the training set time period; x The normalized eigenvalue represents the standardized value in the interval [0,1] obtained after linear transformation of the original eigenvalue.

3. The wind turbine gearbox fault early warning system based on a multimodal cooperative Transformer network according to claim 2, characterized in that, In the four-path feature extraction module: The first path is a spatiotemporal path based on LSTM and Temporal Attention; it is used to capture the long-term dependencies of temporal data and the contribution of important time steps in multimodal features; temporal features are extracted by bidirectional LSTM, and then the importance of features at different time steps is adaptively weighted by combining the temporal attention mechanism. The formula for calculating the hidden state of LSTM is: ； x t It is the input feature vector at time step t, with dimension d. in h t-1 It is the hidden state of the previous time step; h t It is the output of the hidden state at the current time step, with dimension d. hidden ; The formula for calculating the time attention weight is: ； The importance scoring function is defined as follows: ； h t is the LSTM hidden state vector at time step t; u is the learnable attention parameter vector; u It is the transpose of vector u; u h t are vectors u and h t The importance score is obtained after the dot product operation; It is the normalized attention weight at the t-th time step; The formula for weighted feature output is: ； Z lstm It is the final output feature vector of the spatiotemporal path, with dimension d. hidden ; The second path is a CNN-based local pattern path; it is used to extract local short-term patterns and periodic features from time-series data in multimodal features, and captures local correlations at different time scales through multi-layer one-dimensional convolutional kernels. The formula for convolution feature extraction is: ； X is the input time-series feature matrix with dimension 1. ; C is the convolution kernel weight parameter; C is the convolution output feature map with dimension 1. ; The global feature aggregation formula is: ； Z cnn It is the final output feature vector of the local mode path, with dimension d. filters ; The third path is the global self-attention path; it is used to establish global dependencies between all time steps through the self-attention mechanism, capturing long-distance temporal correlations and feature interactions. The formula for calculating self-attention is: ； Q is the query matrix, K is the key matrix, and V is the value matrix, all obtained from the input X through a linear transformation; K d is the transpose of K; k It is the dimension of the key matrix, used to scale the dot product result; It is an attention-weighted feature representation; The formula for global feature output is: ； Z attn It is the final output feature vector of the global self-attention path, with dimension d. model ; The fourth path is the residual connection path; it is used to preserve important information of the original input, ensure that the gradient propagates stably in the network, prevent information loss, and enhance the generalization ability of the model. The fully connected mapping formula is: ； The flattened input feature vector has dimensions T×d. in ; This is the weight matrix of the fully connected layer; For bias terms; This is the final output feature vector of the residual connection path, with dimension d. res ; To ensure that subsequent modules can effectively integrate the features of each path, all path outputs are uniformly mapped to the same dimension through a fully connected layer. The mapping formula is as follows: ； It is a unified weight matrix for each path dimension; It is a bias term; These are path output feature vectors with unified dimensions, all having dimension d. uniform .

4. The wind turbine gearbox fault early warning system based on a multimodal cooperative Transformer network according to claim 3, characterized in that, In the adaptive fusion module, the formula for calculating path importance weights is: ； It is the feature output vector of the i-th path; v i It is the learnable weight vector of the i-th path; It is a vector v i The transpose of z is used with the eigenvector z. i Perform dot product operation; It is the normalized importance weight of the i-th path, and satisfies ; The formula for weighted feature fusion is: ； These are the importance weights of each path; These are the feature output vectors of each path; It is the fused feature vector, with dimensions consistent with the output dimensions of each path.

5. The wind turbine gearbox fault early warning system based on a multimodal cooperative Transformer network according to claim 4, characterized in that, In the feature refinement module, the feature selection mask generation formula is: ； It is the feature vector output by the dynamic fusion module; It is the feature selection weight matrix; It is a bias term; It is the sigmoid activation function, which compresses the output to the (0,1) interval; It is a feature selection mask vector, where each element represents the importance of the corresponding feature dimension; The feature refinement formula is: ； It is element-wise multiplication; It is a feature selection mask vector, with dimensions of ... same; It is a refined feature vector, used to enhance important features and suppress redundant features.

Images