Fault early warning and life prediction method and system for wind generating set

Abstract

The invention relates to the technical field of state monitoring of wind generating sets, and discloses a fault early warning and service life prediction method and system for a wind generating set, and the method comprises the steps: obtaining first state data, second state data and image data of a target wind generating set, and forming multi-dimensional data; fusing the multi-dimensional data by using a multi-modal fusion model to obtain multi-modal data fusion features of the target wind generating set; and performing fault early warning and / or life prediction on the target wind generating set based on the multi-modal data fusion features. By integrating the multi-modal data, the problem that fault features are difficult to comprehensively capture by a single data source is solved, fault early warning and service life prediction are performed by utilizing the multi-modal data fusion features, the false report and missing report rate of faults is reduced, accurate quantitative prediction of the remaining service life of the wind generating set is realized in combination with the data driving model, and the prediction efficiency is improved. By improving the accuracy of fault early warning and life prediction, the wind generating set is effectively operated and maintained in advance.

Description

Technical Field

[0001] The present invention relates to the technical field of wind turbine generator set status monitoring, and in particular to a method and system for fault early warning and life prediction of a wind turbine generator set. Background Art

[0002] In existing technologies, most gearbox fault warning systems rely solely on Supervisory Control and Data Acquisition (SCADA) data (such as temperature, pressure, and speed), using statistical thresholds or simple machine learning models for anomaly detection, which struggles to fully capture fault characteristics. Some studies have attempted to incorporate vibration data for fault diagnosis. For example, the paper "Vibration Signal Feature Extraction and Fault Identification for Wind Turbine Gearboxes" uses short-time Fourier transforms to analyze the vibration spectrum. However, this approach fails to consider the impact of speed fluctuations under variable operating conditions on signal analysis, limiting the model's generalization under complex operating conditions and making it prone to false positives and negatives. Existing life prediction technologies are often based on physical models (such as the Paris crack growth law) or empirical formulas, lacking integration with real-time monitoring data. For example, patent CN114481161B proposes a remaining life prediction method based on vibration signals, but fails to integrate multimodal data, limiting prediction accuracy due to the information integrity of a single data source. Summary of the Invention

[0003] In view of this, the present invention provides a fault warning and life prediction method and system for a wind turbine generator set to solve the problems of difficulty in fully capturing fault characteristics, high false alarm and missed alarm rates, and inability to quantify the remaining service life.

[0004] In a first aspect, the present invention provides a method for fault warning and life prediction of a wind turbine generator set, the method comprising: Acquire first state data of the target wind turbine generator set using a data acquisition and monitoring system, acquire second state data of the target wind turbine generator set using a state monitoring system, acquire image data of the target wind turbine generator set using an image acquisition device, and pre-process the first state data, the second state data, and the image data to obtain multidimensional data; The multi-modal fusion model is used to fuse the multi-dimensional data to obtain the multi-modal data fusion features of the target wind turbine generator set; Fault warning and / or life prediction of target wind turbines are performed based on multimodal data fusion features.

[0005] The fault warning and life prediction method for wind turbines provided by the present invention solves the problem that a single data source is difficult to fully capture fault characteristics by integrating multimodal data. It uses the fusion characteristics of multimodal data to perform fault warning and life prediction, reduces the false alarm and missed alarm rate of faults, and combines data-driven models to achieve accurate quantitative prediction of the remaining service life of wind turbines. By improving the accuracy of fault warning and life prediction, effective operation and maintenance of wind turbines can be carried out in advance, thereby increasing the service life of wind turbines.

[0006] In an optional embodiment, preprocessing the first state data, the second state data, and the image data is performed separately to obtain multidimensional data, including: The first state data is sequentially cleaned, standardized, filtered, and smoothed to obtain first-dimensional data; Cleaning and resampling the second state data, and extracting fault-sensitive characteristic values from the resampled vibration signal as second-dimensional data; The image data includes infrared images and visible light images. The infrared images are converted into temperature distribution matrices, and the local temperature gradient is calculated. Areas where the local temperature gradient exceeds a preset gradient threshold are marked as abnormal. The visible light image including the oil stain area is segmented to obtain a mask image. The visible light image including the coupling is matched with feature points to obtain the coupling position offset.

[0007] The fault warning and life prediction method for wind turbines provided by the present invention constructs a multi-dimensional feature space by integrating and processing first state data, second state features and image data. Different preprocessing methods are used for different types of data to retain key features, reduce noise interference, improve data quality, cover the mechanical, thermodynamic and appearance states of gearbox operation, more comprehensively obtain the operating characteristics of the wind turbine, and improve the accuracy of subsequent judgments on the operating status of the wind turbine.

[0008] In an optional embodiment, a multimodal fusion model is used to fuse multidimensional data to obtain multimodal data fusion features of a target wind turbine generator set, including: Based on the timestamp of the first state data, the second state data and the image data are subjected to time synchronization processing to obtain multi-dimensional synchronized data; The preset cross-modal fusion model is used to fuse the multi-dimensional synchronized data to obtain the multi-modal data fusion features.

[0009] The fault warning and life prediction method for wind turbines provided by the present invention eliminates the timing deviation caused by asynchronous acquisition through time synchronization, avoids feature misassociation caused by time dislocation, and improves data reliability. By fusing multi-dimensional synchronous data through a preset cross-modal fusion model, it can capture complex associations that are difficult to reflect in a single modality, break through the limitations of single-modal information, and comprehensively reflect the multi-dimensional status of the equipment through fused features, thereby enhancing the model's sensitivity to early hidden dangers.

[0010] In an optional embodiment, a preset cross-modal fusion model is used to fuse the multi-dimensional synchronized data to obtain a multi-modal data fusion feature, including: Perform spatial mapping on each multi-dimensional synchronization data to obtain multi-dimensional synchronization alignment data in the same latent space; Calculate the similarity between the multi-dimensional synchronous alignment data and the feature data in the historical working condition database, and determine the corresponding working condition and attention weight based on the similarity calculation results; Based on the multi-dimensional synchronous alignment data and the corresponding attention weights, weighted calculation is performed to obtain multi-dimensional fusion features; The multi-dimensional fusion features are standardized to obtain multi-modal data fusion features.

[0011] In an optional embodiment, the process of constructing the historical operating condition database includes: Obtain historical multi-dimensional synchronous alignment data, cluster the historical multi-dimensional synchronous alignment data, and generate a working condition label library; By using feature importance analysis, the feature weights corresponding to each multi-dimensional synchronous alignment data under different working condition labels are determined. The historical multi-dimensional synchronous alignment data and the corresponding working condition labels and feature weights constitute the historical working condition database.

[0012] The fault warning and life prediction method for wind turbines provided by the present invention embeds them into the same latent space through spatial mapping to achieve alignment, calculates the similarity with historical operating condition feature data to determine the operating condition and attention weights, performs weighted calculation and standardization based on the weights, dynamically focuses on sensitive features of the current operating condition, suppresses redundant information, comprehensively captures multi-dimensional coupling relationships, and improves the feature representation capability of the equipment status, thereby enhancing the accuracy of fault diagnosis and life prediction and adaptability to variable operating conditions, providing more comprehensive and reliable feature support for the intelligent operation and maintenance of wind turbines.

[0013] In an optional embodiment, performing fault warning and / or life prediction on a target wind turbine generator set based on multimodal data fusion features includes: Acquire multimodal data fusion features in real time, and use the preset fault warning model to process the multimodal data fusion features to obtain the probability distribution of multiple preset fault types; Determine the warning level based on the probability distribution of each preset fault type and the preset fault warning threshold, and issue a warning prompt based on the warning level; Acquire multiple historical multimodal data fusion features within a preset time period to form a historical fusion feature sequence; The historical fusion feature sequence is input into the preset life prediction model to obtain the predicted remaining life of the target wind turbine generator set.

[0014] The fault warning and life prediction method for wind turbines provided by the present invention integrates multi-dimensional information such as vibration, temperature, and images to achieve precise status monitoring and health management of wind turbines. It processes fusion features in real time, outputs probability distributions for multiple fault types, and dynamically adjusts warning levels based on preset thresholds. This allows for rapid identification of early-stage hidden dangers and graded responses to avoid fault escalation. It also captures equipment performance degradation trends through historical feature sequences and predicts remaining life. Multimodal fusion effectively compensates for the limitations of a single data source, improves diagnostic robustness under complex operating conditions, and reduces false alarm rates. A dynamic warning mechanism shortens the fault response cycle, and a predictive maintenance strategy reduces unplanned downtime losses, extending equipment service life and achieving an optimal balance between operation and maintenance costs and power generation benefits.

[0015] In an optional embodiment, the preset fault warning model is a bidirectional long short-term memory network + attention mechanism + fully connected layer network structure, the preset life prediction model is a physical information constraint model based on the long short-term memory network, and the method further includes: Calculating a first prediction variance of a fault warning result and a second prediction variance of a life prediction result; if the first prediction variance or the second prediction variance exceeds a preset variance threshold, updating feature weights in a historical operating condition database using multi-dimensional synchronous data acquired in real time; The multimodal data fusion features are updated based on the updated feature weights and multi-dimensional synchronous data. The updated multimodal data fusion features are used to dynamically optimize the preset fault warning model and the preset life prediction model and update the model parameters.

[0016] The fault warning and life prediction method for wind turbines provided by the present invention triggers a model optimization process when the variance of the prediction results is too large, updates the model parameters to adapt to new failure modes and changes caused by equipment aging, and utilizes rich data resources and powerful computing power to continuously optimize model performance and improve the adaptability and accuracy of the system.

[0017] In a second aspect, the present invention provides a fault warning and life prediction system for a wind turbine generator set, the system comprising: a multidimensional data acquisition module, configured to acquire first state data of a target wind turbine generator set using a data acquisition and monitoring system, acquire second state data of the target wind turbine generator set using a state monitoring system, acquire image data of the target wind turbine generator set using an image acquisition device, and pre-process the first state data, the second state data, and the image data to obtain multidimensional data; A multimodal data fusion module is used to fuse multidimensional data using a multimodal fusion model to obtain multimodal data fusion features of the target wind turbine generator set; The early warning and prediction module is used to provide fault early warning and / or life prediction for the target wind turbine generator set based on multimodal data fusion features.

[0018] In a third aspect, the present invention provides a computer device comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0019] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the method of the first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0021] Figure 1 is a flow chart of a method for fault warning and life prediction of a wind turbine generator set according to an embodiment of the present invention; Figure 2 is a flow chart of another method for fault warning and life prediction of a wind turbine generator set according to an embodiment of the present invention; Figure 3 is a structural block diagram of a fault warning and life prediction system for a wind turbine generator set according to an embodiment of the present invention; Figure 4 Schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0022] To make the purpose, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative efforts shall fall within the scope of protection of the present invention.

[0023] An embodiment of the present invention provides a method for fault warning and life prediction of a wind turbine generator set, which integrates multimodal data and uses fused data to perform fault warning and life prediction, thereby achieving the effect of improving the accuracy of fault warning and life prediction.

[0024] According to an embodiment of the present invention, an embodiment of a method for fault warning and life prediction of a wind turbine generator set is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0025] In this embodiment, a method for fault warning and life prediction of a wind turbine generator set is provided, which can be used in the above-mentioned computer system. Figure 1 FIG. 1 is a flow chart of a method for fault warning and life prediction of a wind turbine generator set according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps: Step S101: Use a data acquisition and monitoring system to obtain first state data of a target wind turbine generator set, use a state monitoring system to obtain second state data of a target wind turbine generator set, and use an image acquisition device to obtain image data of a target wind turbine generator set. Preprocess the first state data, the second state data, and the image data respectively to obtain multidimensional data.

[0026] Specifically, the first state data collected by the Supervisory Control And Data Acquisition (SCADA) system includes but is not limited to: gearbox input / output shaft speed, lubricating oil temperature (PT100 sensor ±0.5°C), oil pressure (piezoresistive sensor 0-10 Bar), gearbox vibration intensity (mm / s), ambient temperature, generator power, wind wheel speed, etc. The sampling frequency of the first state data is 1Hz, and the timestamp is aligned to the second level.

[0027] When using a Condition Monitoring System (CMS) to collect data, sensors must be deployed first. For example, a triaxial accelerometer (frequency response 0.5Hz-10kHz) can be installed on the input shaft to monitor gear meshing impact; an ICP accelerometer can be installed radially on the planetary carrier bearing to capture the characteristic frequency of the planetary gear fault (the formula is: ,in, is the characteristic frequency, is the shaft rotation frequency, is the number of planetary gear teeth). The CMS data acquisition frequency is 25.6 kHz (satisfying the Nyquist theorem and covering the high-frequency components of gear meshing), and one file is stored every 10 seconds (including timestamp synchronization information).

[0028] For image data, according to the characteristics to be monitored, select the appropriate image acquisition equipment, such as using an infrared thermal imager to detect the temperature distribution of the gearbox housing (accuracy ±2°C, spatial resolution 640×480), and using a visible light camera to shoot the external mechanical structure of the gearbox (ROI focusing on the coupling and oil pipe interface) to monitor whether the mechanical structure is deformed. It should be noted that image acquisition can be done in real time, or the trigger conditions for collecting image data can be set according to actual needs. For example, when the oil temperature in the first state data is >70°C or the vibration RMS in the second state data is >4mm / s, start continuous shooting (30fps, lasting 10 seconds); when SCADA monitors that the temperature of a certain point on the gearbox housing exceeds the set threshold (for example, it exceeds the average value of the normal operating temperature of the area plus 3 times the standard deviation), start the infrared thermal imager and continuously shoot the abnormal temperature area with a shooting parameter of 30fps for 10 seconds to further analyze the cause of the abnormal local temperature increase; when SCADA When the oil level is detected to be below a set threshold (e.g., below 20% of the full scale of the oil level sensor), the visible light camera is activated to continuously capture the gearbox's external mechanical structure (focusing on potential leaks, such as oil pipe connections) at 30fps for 10 seconds. When the SCADA system detects that the gearbox bearing temperature exceeds a set threshold (e.g., exceeding the average normal operating temperature plus three standard deviations, or the temperature suddenly rises at a rate exceeding a certain level, such as more than 5°C per minute), the infrared thermal imager is activated to continuously capture the bearing area at 30fps for 10 seconds to identify problems such as bearing wear or poor lubrication. This is for example only and is not intended to be limiting.

[0029] The collected original data of the first state data, the second state data, and the image data may contain invalid data or data loss. To ensure the reliability of the multidimensional data, it is necessary to preprocess the data, delete the invalid data, and retain the valid data as the multidimensional data.

[0030] Step S102 : using a multimodal fusion model to fuse the multidimensional data to obtain multimodal data fusion features of the target wind turbine generator set.

[0031] Specifically, multidimensional data is fed into a multimodal fusion model for fusion. Using a pre-trained Transformer cross-modal encoder, the multidimensional data is mapped to a shared latent space. An attention mechanism is then used to calculate the weight coefficients for each dimension of the data (for example, a weight coefficient of 0.45 for vibration data, 0.35 for temperature data, and 0.2 for current data). Finally, batch normalization (BN) is performed on the fused features, outputting a comprehensive feature vector encompassing mechanical state, thermal performance, and electrical characteristics. This is used for subsequent fault warning and lifespan prediction. This effectively integrates the multidimensional information of the wind turbine and improves the ability to represent features under complex operating conditions.

[0032] Step S103 : performing fault warning and / or life prediction on the target wind turbine generator set based on the multimodal data fusion feature.

[0033] Specifically, the multimodal data fusion features corresponding to different fault types and warning levels are constructed into a fault feature library. When performing fault warnings, the current fault type and warning level are determined by comparing and analyzing the wind turbine's current multimodal data fusion features with the fusion features in the fault feature library. A life prediction database is also formed based on the remaining life corresponding to different multimodal data fusion features. When performing life prediction, the current remaining life is determined by comparing and analyzing the wind turbine's current multimodal data fusion features with the fusion features in the life prediction database. To ensure the accuracy of fault warnings and life predictions, the fault feature library and life prediction database need to be regularly updated to ensure data reliability.

[0034] It should be noted that fault warning primarily monitors the operating status of the gearbox in real time, identifying the presence, type, and severity of faults. Its core focus is timeliness, identifying potential fault signs. Life prediction, based on fault warning, further quantifies and predicts the remaining useful life of the gearbox, focusing on predicting how long the gearbox can operate normally before or in the early stages of a fault. Its core focus is foresight. Fault warning results provide timely reference information for life prediction, helping the life prediction model more accurately assess the health and remaining life of the gearbox.

[0035] In scenarios where equipment lifespan prediction is highly demanding but real-time fault warnings are less important, lifespan prediction can be performed independently. For example, for non-critical equipment, the primary focus is on its remaining useful life to rationally plan maintenance, without overemphasizing real-time fault monitoring. This is an example, but not limited to this. In scenarios where real-time performance is extremely important and immediate fault response is required, fault warnings can be performed independently. For example, in older wind farms, sensor configuration or data transmission limitations may prevent the support of complex lifespan prediction models, yet timely fault detection is essential to prevent equipment damage. This is an example, but not limited to this.

[0036] It should be noted that this embodiment utilizes edge-cloud collaborative computing for data processing. The edge primarily performs real-time data processing (such as vibration signal feature extraction and image anomaly detection) to reduce data transmission volume and latency. Specifically, this includes: Real-time processing of vibration signals, using a short-time Fourier transform combined with a residual network feature extraction method to quickly capture fault characteristics in vibration signals, reducing data transmission volume and latency to meet real-time requirements; image anomaly detection, deploying a lightweight YOLOv8 model and accelerating it with TensorRT, enabling real-time monitoring of gearbox appearance anomalies and timely identification of potential fault hazards. The cloud performs more complex data fusion and model inference to achieve more accurate fault diagnosis and life prediction. Specifically, this includes receiving SCADA data and feature data extracted by the edge, performing multimodal fusion and model inference, integrating global information, and performing comprehensive fault diagnosis and life prediction. Furthermore, the cloud stores historical data and regularly updates model parameters, leveraging abundant data resources and powerful computing power to continuously optimize model performance and enhance system adaptability and accuracy.

[0037] The fault warning and life prediction method for wind turbines provided in this embodiment solves the problem that a single data source is unable to fully capture fault characteristics by integrating multimodal data. It uses the fusion characteristics of multimodal data to perform fault warning and life prediction, reduces the false alarm and missed alarm rate of faults, and combines data-driven models to achieve accurate quantitative prediction of the remaining service life of wind turbines. By improving the accuracy of fault warning and life prediction, effective operation and maintenance of wind turbines can be carried out in advance, thereby increasing the service life of wind turbines.

[0038] In this embodiment, a method for fault warning and life prediction of a wind turbine generator set is provided, which can be used in the above-mentioned computer system. Figure 2 FIG. 1 is a flow chart of a method for fault warning and life prediction of a wind turbine generator set according to an embodiment of the present invention. Figure 2 As shown, the process includes the following steps: In step S201, the first state data of the target wind turbine generator set is obtained by using a data acquisition and monitoring system, the second state data of the target wind turbine generator set is obtained by using a state monitoring system, and the image data of the target wind turbine generator set is obtained by using an image acquisition device. The first state data, the second state data, and the image data are preprocessed respectively to obtain multidimensional data.

[0039] Specifically, the above step S201 includes: Step S2011 , cleaning, standardizing, filtering, and smoothing the first state data in sequence to obtain first dimensional data.

[0040] Specifically, time series prediction (LSTM imputation with a 60-second input window length) is used to estimate missing values. When the oil temperature exceeds 120°C for three consecutive sampling points, it is considered a sensor failure, triggering a manual calibration process. If the oil temperature at a single point exceeds 100°C but does not persist, it is marked as suspicious data and corrected through adjacent time interpolation. SCADA data is normalized to a specific range (such as 0 to 1 or -1 to 1) to eliminate dimensionality effects and make different parameters comparable. A low-pass filter (such as a Butterworth filter) is applied to remove high-frequency noise while retaining low-frequency signal components. Moving average or exponential smoothing is used to smooth the data to reduce the impact of short-term fluctuations. After performing the above preprocessing on the first-state data, the first-dimensional data is obtained.

[0041] Step S2012 : cleaning and resampling the second state data, and extracting fault-sensitive characteristic values from the resampled vibration signal as second-dimensional data.

[0042] Specifically, the data cleaning process includes: calculating the peak-to-peak value of each frame signal, and discarding those that exceed 3 times the historical median (the historical median is determined based on the vibration baseline data during normal equipment operation); using a 5th-order Savitzky-Golay filter (window length 11 points) to eliminate low-frequency drift and retain valid signal components above 10 Hz.

[0043] The resampling process includes: resampling the vibration signal in the angular domain according to the real-time speed provided by SCADA, converting the time domain signal into the angular domain signal, eliminating the signal expansion and contraction caused by speed changes, especially under variable speed operating conditions, making the vibration characteristics easier to extract and analyze. The conversion formula is: ,in, is the integral variable, representing the instantaneous time point in the integration process; dτ: represents a small time increment; x(t): the original time-domain vibration signal, representing the vibration amplitude that changes with time; t: the time variable, representing the time point of signal sampling; x(θ): the signal after angular domain resampling, representing the vibration amplitude that changes with angle; θ: the angle variable, representing the rotation angle, which is related to the speed of the rotating machinery; : The real-time speed of the gearbox input shaft, usually expressed in revolutions per minute (RPM) or revolutions per second (Hz), reflecting the rotation speed of the mechanical shaft.

[0044] Fault-sensitive characteristic indicators can be extracted from the vibration signal after angular domain resampling, including but not limited to: the sensitive characteristic indicator corresponding to tooth surface wear is the modulation sideband energy ratio (SER); the sensitive characteristic indicator corresponding to bearing spalling is the envelope kurtosis (Envelope Kurtosis); the sensitive characteristic indicator corresponding to shaft misalignment is the 1x frequency amplitude ratio (1X Energy Ratio), which is used as an example only and is not limited to this.

[0045] Feature extraction methods include: time domain signal processing: calculating statistical features in the time domain, such as peak value, root mean square value (RMS), etc.; frequency domain signal processing: converting the time domain signal to the frequency domain through fast Fourier transform (FFT) and calculating the spectrum characteristics; time-frequency analysis: using methods such as short-time Fourier transform (STFT) or wavelet transform to analyze the time-frequency characteristics of the signal; envelope spectrum analysis: performing envelope detection on the signal and extracting envelope spectrum features for detecting impact faults; combining multiple extraction methods to analyze the vibration signal from different angles and extract sensitive features that can reflect different fault types. The extraction process of each fault sensitive feature indicator specifically includes: (1) Modulation Sideband Energy Ratio (SER): Frequency domain signal processing and envelope spectrum analysis are used together. First, the time domain signal is converted to the frequency domain using FFT to obtain a spectrum diagram. Then, the carrier frequency and its modulation sideband are identified, and the ratio of the modulation sideband energy to the total energy is calculated, which is the modulation sideband energy ratio (SER). At the same time, envelope spectrum analysis can help detect impact faults and further confirm the existence and characteristics of the modulation sideband.

[0046] (2) Kurtosis of the envelope spectrum corresponding to bearing spalling: Envelope spectrum analysis performs envelope detection on the original vibration signal to extract envelope spectrum features. The envelope spectrum can highlight the characteristics of impact faults, such as the periodic impact signal generated by bearing spalling. The kurtosis value of the envelope spectrum is calculated to obtain the kurtosis of the envelope spectrum corresponding to bearing spalling. This feature reflects the strength of the signal's impact component and can be used to detect bearing spalling faults.

[0047] (3) The proportion of the 1-octave amplitude corresponding to shaft misalignment: Time domain signal processing and frequency domain signal processing are used in combination. In the time domain, the statistical characteristics of the signal, such as the root mean square (RMS), are calculated to evaluate the overall energy level of the signal. In the frequency domain, the signal is converted to the frequency domain through FFT to identify the amplitude of the 1-octave component. The proportion of the 1-octave amplitude to the total energy is calculated, which is the proportion of the 1-octave amplitude corresponding to shaft misalignment. This feature reflects the enhancement of specific frequency components caused by shaft misalignment faults.

[0048] Step S2013: The image data includes an infrared image and a visible light image. The infrared image is converted into a temperature distribution matrix, and the local temperature gradient is calculated. Areas where the local temperature gradient exceeds a preset gradient threshold are marked as abnormal. The visible light image including the oil stain area is segmented to obtain a mask image. The visible light image including the coupling is matched with feature points to obtain the coupling position offset.

[0049] Specifically, the infrared image is converted into a temperature distribution matrix T(x,y) with a resolution of 0.1°C per pixel. The temperature distribution matrix represents the temperature information of each pixel in the infrared image, where (x,y) represents the pixel with horizontal coordinate x and vertical coordinate y in the infrared image. Each element t(x,y) in the temperature distribution matrix represents the temperature value of the corresponding pixel. This two-dimensional matrix can intuitively reflect the temperature distribution on the gearbox housing surface. The local temperature gradient is calculated (Sobel operator), and areas with a gradient greater than 5°C / cm² are marked as abnormal. This is only an example and is not limited to this.

[0050] For visible light images, the oily area is segmented based on the fully convolutional network model U-Net, and a mask image is output. Scale-Invariant Feature Transform (SIFT) feature point matching is used to calculate the coupling position offset between adjacent frames (the pixel to millimeter conversion requires calibration of the camera's internal and external parameters). This process is a mature existing technology and will not be repeated here.

[0051] The fault warning and life prediction method for wind turbines provided in this embodiment constructs a multi-dimensional feature space by integrating and processing first state data, second state features and image data. Different preprocessing methods are used for different types of data to retain key features, reduce noise interference, and improve data quality. It covers the mechanical, thermodynamic and appearance states of the gearbox operation, more comprehensively obtains the operating characteristics of the wind turbine, and improves the accuracy of subsequent judgments on the operating status of the wind turbine.

[0052] Step S202 : Using a multimodal fusion model to fuse the multidimensional data, and obtain multimodal data fusion features of the target wind turbine generator set.

[0053] Specifically, the above step S202 includes: Step S2021 : Based on the timestamp of the first state data, the second state data and the image data are subjected to time synchronization processing to obtain multi-dimensional synchronized data.

[0054] Specifically, since multidimensional data involves multiple data types and formats, it needs to be consistent before data fusion to ensure that the multidimensional data meets the fusion conditions. Based on the timestamp of the SCADA data, the vibration data is downsampled (10kHz→1Hz), and the image data is aligned by seconds to ensure the consistency of the multimodal data in the time dimension.

[0055] For vibration data, the time domain signal is converted into a frequency domain signal through fast Fourier transform (FFT) to obtain a high-dimensional vibration spectrum of the vibration signal. Specific fault-sensitive features are extracted from the spectrum, such as the modulation sideband energy ratio (SER) of tooth surface wear, the envelope spectrum kurtosis of bearing spalling, and the proportion of the 1-time frequency amplitude of shaft misalignment. The principal component analysis algorithm is used to reduce the dimensionality of the high-dimensional vibration spectrum, retaining 95% of the variance, thereby reducing the data dimension and retaining key feature information, thereby improving data processing efficiency.

[0056] Step S2022: Using a preset cross-modal fusion model, the multi-dimensional synchronized data is fused to obtain multi-modal data fusion features.

[0057] Specifically, based on the Transformer's multi-head attention mechanism, dynamic weight allocation of feature information from different modalities is achieved, fully exploring the correlation and complementarity between modal data, and improving the expressive power and diagnostic performance of fused features.

[0058] The multi-dimensional synchronized data includes: SCADA feature vectors (dimension 20), which contain key macro parameters of gearbox operation; ResNet-50 features of vibration spectrograms (dimension 1024), which extract deep feature representations of vibration signals; and YOLOv8 features of image regions of interest (ROI) (dimension 512), which provide semantic feature information of image data.

[0059] The fault warning and life prediction method for wind turbines provided in this embodiment eliminates the timing deviation caused by asynchronous acquisition through time synchronization, avoids feature misassociation caused by time dislocation, and improves data reliability. By fusing multi-dimensional synchronous data through a preset cross-modal fusion model, it can capture complex associations that are difficult to reflect in a single modality, break through the limitations of single-modal information, and comprehensively reflect the multi-dimensional status of the equipment through fused features, thereby enhancing the model's sensitivity to early hidden dangers.

[0060] In some optional implementations, the above step S2022 includes: Step a1: spatially map each multi-dimensional synchronous data to obtain multi-dimensional synchronous aligned data in the same latent space.

[0061] Specifically, assuming that feature data of different modalities have been obtained, such as SCADA feature vectors (dimension 20), ResNet-50 features of vibration spectrum graphs (dimension 1024), YOLOv8 features of image regions of interest (ROI) (dimension 512), etc., these feature data have different dimensions and belong to different feature spaces. It is necessary to use a specific mapping function to convert feature data of different dimensions into the same latent space. For example, a fully connected neural network layer or a learnable linear transformation matrix can be used to achieve this mapping. For SCADA features, a fully connected layer can be used to map them from 20 dimensions to a 256-dimensional latent space; for vibration spectrum graph features and image features, their respective fully connected layers are also used to map them to a 256-dimensional latent space. After mapping, the features of each modality are converted into representations under the same latent space, assuming that 、 、 , mapping each modal feature to the same latent space (dimension 256):

[0062] in, Represents the representation of SCADA features after being mapped to the same latent space, Represents the representation of the vibration spectrum after the features are mapped to the same latent space, Represents the representation of image features after mapping to the same latent space. The three transformed feature vectors, each with a dimension of 256, represent the new representation in the latent space. represents the original SCADA feature vector (dimension 20), ResNet-50 features (dimension 1024) representing the original vibration spectrogram, YOLOv8 features (dimension 512) representing the region of interest (ROI) of the original image, 、 、 It is a learnable weight matrix used to transform features of different dimensions into the same latent space to achieve feature alignment and preliminary fusion.

[0063] Step a2: Calculate the similarity between the multi-dimensional synchronous alignment data and the feature data in the historical working condition database, and determine the corresponding working condition and attention weight based on the calculation result of the similarity.

[0064] Specifically, the Query-Key-Value mechanism is used to calculate the attention weight. The currently input multi-dimensional synchronized alignment data is used as the query vector, and the feature data in the historical working condition database is used as the key matrix and value matrix. By calculating the similarity between the query vector and the key matrix, the attention weight corresponding to each modal feature is obtained. For example, using the dot product similarity calculation, the formula is: , where Q is the query vector, Represents the similarity score of the i-th modal feature, and the formula represents the calculation of the key matrix between the query vector Q and the i-th modality The dot product between is the bond matrix of the ith mode, which comes from the characteristic data in the historical operating condition database.

[0065] Then, the obtained similarity scores are normalized and the softmax function is used to obtain the attention weight αi of each modality, which is: in, Represents the similarity score of the j-th modal feature, which can be obtained by calculating the key matrix between the query vector Q and the j-th modality The similarity between them is: The attention weight represents the relative importance of each modal feature under the current input data. The larger the weight, the more valuable the modal feature is to the current fault diagnosis or life prediction task.

[0066] Calculate cross-modal attention weights (Query-Key-Value mechanism):

[0067]

[0068] in, As a query vector, it represents the feature or feature combination that needs to be paid attention to at present. It can be a mapping of SCADA features or other modal features; As a key matrix, it represents the feature set that matches the query vector Q and is used to calculate the attention score; As a value matrix, it represents the eigenvalues to be weighted and summed. By calculating the similarity between the query vector and the key matrix, the attention weight corresponding to the synchronized alignment data of each dimension can be calculated.

[0069] Step a3: Perform weighted calculation based on the multi-dimensional synchronous alignment data and the corresponding attention weights to obtain multi-dimensional fusion features.

[0070] Specifically, by performing weighted summation on the value matrix, dynamic fusion of different modal features is achieved to form a comprehensive feature representation. According to the characteristics of the current input data, the contribution weight of each modal feature is automatically adjusted to highlight the most valuable feature information for fault diagnosis while suppressing relatively less important information.

[0071] Based on Dynamic Time Warping (DTW) or K-means clustering algorithm, the current working conditions and historical data are matched in real time, and the adaptability under changing working conditions is improved by dynamically adjusting the weights of model input features.

[0072] Step a4: standardize the multi-dimensional fusion features to obtain multimodal data fusion features.

[0073] Specifically, the following formula is used to standardize the multidimensional fusion features:

[0074] in, Represents the normalized multimodal data fusion features, represents the SCADA (Supervisory Control and Data Acquisition System) feature vector, represents the output of the attention mechanism. The attention mechanism dynamically adjusts the weights of different features to highlight the most valuable information for the current task (such as fault diagnosis). Layer Normalization (LayerNorm) standardizes the fused features, stabilizing the model training process and improving the model's generalization and predictive performance. Suppose we have the following feature data: SCADA feature HSCADA: [0.1, 0.2, 0.3, …, 0.5] (the specific dimensions depend on the actual data); the output of the attention mechanism, Attention(Q, K, V): [0.05, 0.1, 0.15, …, 0.25]; after addition, the resulting feature vector is: HSCADA + Attention(Q, K, V) = [0.15, 0.3, 0.45, …, 0.75].

[0075] Then, the above feature data is normalized: z=LayerNorm([0.15,0.3,0.45,…,0.75]), and finally the standardized multimodal data fusion feature z is obtained, and the value of each element will be adjusted to have zero mean and unit variance.

[0076] In some optional implementations, the process of constructing the historical operating condition database includes: Obtain historical multi-dimensional synchronous alignment data, cluster the historical multi-dimensional synchronous alignment data, and generate a working condition label library.

[0077] Specifically, cluster analysis is performed using historical multi-dimensional synchronous alignment data to generate a working condition label library for each cluster. Based on the characteristic data under the cluster, the sensitive characteristic data under the fault condition is summarized to determine the importance of each feature to the occurrence of the fault.

[0078] In wind turbine gearbox fault diagnosis, different fault types will present unique vibration, temperature, and image characteristics. The following lists the data characteristics corresponding to several typical fault conditions: (1) Tooth surface wear fault condition: Spectral analysis of the vibration signal shows that obvious modulation sidebands appear near the meshing frequency and its harmonics of the gears, which is usually caused by micro-relative motion caused by tooth surface wear; SCADA data shows that the lubricating oil temperature of the gearbox is slightly higher than that during normal operation (about 5-10 degrees Celsius), and the lubricating oil pressure fluctuates slightly; infrared thermal imaging shows that the temperature distribution in the relevant areas of the gearbox is uneven, and there is a phenomenon of local temperature increase, but it has not yet reached the threshold of thermal failure. Sensitive feature data include modulation sideband energy ratio (SER), lubricating oil temperature, and local temperature increase in infrared thermal imaging. Their importance weights can be determined based on feature importance analysis. For example, the weight of modulation sideband energy ratio is 0.5, the weight of lubricating oil temperature is 0.3, and the weight of local temperature in infrared thermal imaging is 0.2.

[0079] (2) Bearing spalling fault condition: Envelope spectrum analysis of the vibration signal showed a significant periodic impact component at the characteristic frequency of the bearing, which is a typical characteristic of bearing spalling. SCADA data showed that the vibration intensity of the gearbox increased significantly, exceeding the vibration level during normal operation. Visible light images showed a slight increase in the position offset of the coupling, which may be due to slight misalignment caused by bearing spalling. Sensitive feature data include envelope spectrum kurtosis, vibration intensity, and coupling position offset. The feature weights can be set to 0.6 for envelope spectrum kurtosis, 0.3 for vibration intensity, and 0.1 for coupling position offset.

[0080] (3) Shaft misalignment fault condition: Vibration signal analysis shows a significant amplitude increase at the 1-octave frequency of the rotating shaft, which is a typical characteristic of shaft misalignment. SCADA data shows that there is a slight asymmetry in the input / output shaft speed of the gearbox, and the torque fluctuation is large. The visible light image shows that there is obvious misalignment at the connection of the coupling. The position offset can be quantified by feature point matching. Sensitive feature data includes the 1-octave amplitude ratio, speed asymmetry, and coupling position offset. The feature weight can be set to 0.5 for the 1-octave amplitude ratio, 0.3 for speed asymmetry, and 0.2 for the coupling position offset.

[0081] By using feature importance analysis, the feature weights corresponding to each multi-dimensional synchronous alignment data under different working condition labels are determined. The historical multi-dimensional synchronous alignment data and the corresponding working condition labels and feature weights constitute the historical working condition database.

[0082] For each operating condition label (such as normal operation, tooth surface wear, bearing spalling, etc.), a large amount of historical multi-dimensional synchronous alignment data is collected. This data contains multiple features (such as vibration signal features, temperature features, image features, etc.). Feature importance analysis algorithms (such as random forest-based feature importance scores, SHAP values, etc.) are applied to calculate the weight of each feature under that operating condition. The weight value reflects the contribution of the feature to that operating condition. Ultimately, the historical multi-dimensional synchronous alignment data, the corresponding operating condition labels, and the weights of each feature are integrated together to form a historical operating condition database. This provides key information for subsequent fault diagnosis and life prediction models, helping the models better understand which features are most important under different operating conditions, thereby improving the accuracy and reliability of the models.

[0083] Suppose you are building a fault warning system for a wind turbine gearbox and performing feature importance analysis on the operating condition label "tooth wear." The collected historical multi-dimensional synchronized alignment data includes the following features: vibration signal feature - modulation sideband energy ratio (SER); temperature feature - lubricating oil temperature average; image feature - the proportion of localized temperature elevation areas in infrared thermal images; and other SCADA features - oil pressure average.

[0084] A feature importance analysis was performed on the above features, and the following weights were obtained: the weight of the modulation sideband energy ratio (SER) is 0.5, indicating that under the condition of tooth surface wear, this vibration signal feature has the strongest indicative effect on faults and is a key indicator for judging tooth surface wear; the weight of the mean lubricating oil temperature is 0.3, showing that temperature changes have a certain indicative effect on tooth surface wear, but the influence is slightly weaker than that of vibration signals; the weight of the proportion of local temperature increase areas in infrared thermal images is 0.15, indicating that image features may not be as sensitive as vibration and temperature features in the early stages of tooth surface wear, but still have certain reference value; the weight of the mean oil pressure is 0.05, indicating that under the condition of tooth surface wear, the change in oil pressure has a relatively weak indicative effect on faults and may not be the main sensitive feature.

[0085] The above features and their weights will be stored in the historical operating condition database to provide support for subsequent data analysis, model training and fault diagnosis.

[0086] The fault warning and life prediction method for wind turbines provided in this embodiment embeds them into the same latent space through spatial mapping to achieve alignment, calculates the similarity with historical operating condition feature data to determine the operating condition and attention weights, performs weighted calculation and standardization based on the weights, dynamically focuses on sensitive features of the current operating condition, suppresses redundant information, comprehensively captures multi-dimensional coupling relationships, and improves the feature representation capability of the equipment status, thereby enhancing the accuracy of fault diagnosis and life prediction and adaptability to variable operating conditions, and providing more comprehensive and reliable feature support for the intelligent operation and maintenance of wind turbines.

[0087] Step S203 : performing fault warning and / or life prediction on the target wind turbine generator set based on the multimodal data fusion feature.

[0088] Specifically, the above step S203 includes: Step S2031 : acquiring multimodal data fusion features in real time, and processing the multimodal data fusion features using a preset fault warning model to obtain probability distributions of multiple preset fault types.

[0089] Specifically, the preset fault warning model is a bidirectional long short-term memory network (LSTM) + attention mechanism + fully connected layer network structure. Among them, the bidirectional LSTM layer is a bidirectional structure with 128 units, which can effectively capture the hidden information of time series data in both positive and negative directions, and improve the perception of fault development trends; the attention layer is used to calculate the time step weight, automatically focusing on abnormal time periods containing fault characteristics, highlighting key information, and suppressing irrelevant noise interference; the output layer is a Softmax classification layer, which maps the fused features to the preset fault categories (normal / wear / broken tooth / bearing fault) to achieve accurate identification of fault types.

[0090] The loss function uses Focal Loss to solve the problem of category imbalance. Its expression is:

[0091] in, is the category weight, which is used to adjust the proportion of different categories in the loss calculation. γ=2 is the adjustment factor, which is used to suppress the contribution of easy-to-classify samples to the loss, so that the model pays more attention to difficult-to-classify samples and improves the overall classification performance. Indicates the probability of the model predicting a certain fault type. If the model is very confident in the prediction of a certain sample (i.e. If the value of the sample is close to 1, the contribution of the sample to the loss is reduced due to the design of Focal Loss. This leverages the advantages of bidirectional LSTM in capturing bidirectional dependencies in time series data, the attention mechanism in focusing on key information, and the fully connected layer in classification and decision-making to achieve accurate early warning of gearbox failures.

[0092] By using the trained preset fault warning model, the mapping relationship between different fault characteristics and fault types can be learned based on historical data. When new multimodal fusion data features are input into the model, the model will output the possible fault types and corresponding probability distributions of the current wind turbine according to the learned mapping relationship.

[0093] Step S2032: Determine the warning level according to the probability distribution of each preset fault type and the preset fault warning threshold, and issue a warning prompt according to the warning level.

[0094] Specifically, for example, if the probability distribution is: normal operation: 60%, tooth wear: 25%, bearing spalling: 10%, and shaft misalignment: 5%, and the preset fault warning threshold is set to 0.7, then under this probability distribution, the probability of all fault types is lower than the preset fault warning threshold, and no warning is issued. Based on historical fault data and normal operation data, a machine learning algorithm (isolation forest) is used to identify abnormal data points and expert experience is used to set the preset fault warning threshold. Furthermore, the receiver operating characteristic curve (ROC) can be used to optimize the threshold to ensure the accuracy and timeliness of warnings.

[0095] The warning levels are divided into the following categories: low-level warning: the probability of failure is between 0.5-0.7, which prompts attention and recommends further monitoring; high-level warning: the probability of failure is between 0.7-0.9, which prompts that maintenance may be required in the near future and recommends arranging a maintenance plan; high-level warning: the probability of failure is greater than 0.9, which prompts immediate shutdown and inspection to avoid the expansion of the failure.

[0096] Step S2033: Acquire multiple historical multimodal data fusion features within a preset time period to form a historical fusion feature sequence.

[0097] Specifically, multiple continuous historical multimodal data fusion features within a preset time period (for example, 30 days) are obtained to form a historical fusion feature sequence, which contains the comprehensive operating status information of the wind turbine generator set in the past period of time, providing rich historical data support for life prediction.

[0098] Step S2034: input the historical fusion feature sequence into a preset life prediction model to obtain the predicted remaining life of the target wind turbine generator set.

[0099] Specifically, the preset life prediction model is a physical information constraint model based on a long short-term memory network. This model combines a data-driven LSTM model with a physical model of gearbox degradation, leveraging prior physical knowledge and data information to improve the accuracy and reliability of life prediction. The LSTM layer (64 units) extracts the temporal characteristics of the input sequence and captures hidden patterns and trends in fault evolution. The physical constraint layer introduces the gearbox degradation equation (Paris's crack growth law) as a regularization term:

[0100] in, is the crack length, which indicates the size of the crack in the gearbox; N is the number of cycles, which indicates the number of cyclic loading experienced by the gearbox; ΔK is the stress intensity factor amplitude, which describes the stress field intensity at the crack tip; λ is the weight coefficient, which is used to balance the proportion of the physical constraint term and the data fitting term in the loss function; C represents the crack growth rate constant, which is related to material properties and indicates the distance a crack propagates per unit cycle under a given stress intensity factor (SIF) amplitude. A larger C value indicates faster crack propagation under the same SIF amplitude. m represents the crack growth exponent, which represents the relationship between the crack growth rate and the SIF amplitude. It typically ranges from 2 to 4 and reflects the material's sensitivity to changes in the SIF. A larger m value indicates a more sensitive crack growth rate to changes in the SIF amplitude. By introducing physical constraints and incorporating prior physical knowledge into the model training process, the prediction results are more consistent with actual physical laws.

[0101] The trained preset life prediction model can directly output the predicted remaining useful life (RUL, unit: day) and confidence interval based on the historical fusion feature sequence. Monte Carlo Dropout is used to simulate uncertainty and obtain the model's prediction variance threshold, providing a reliable basis for operation and maintenance decision-making and risk assessment. The Monte Carlo Dropout formula is: , where μ is the predicted mean, T is the number of predictions or trials, represents the result of the t-th prediction, σ 2 The smaller the variance, the more stable the prediction result of the model and the lower the uncertainty.

[0102] The fault warning and life prediction method for wind turbines provided in this embodiment integrates multi-dimensional information such as vibration, temperature, and images to achieve precise status monitoring and health management of wind turbines. It processes fusion features in real time, outputs probability distributions for multiple fault types, and dynamically adjusts warning levels based on preset thresholds. This allows for rapid identification of early hidden dangers and graded responses to avoid fault escalation. It also captures equipment performance degradation trends through historical feature sequences and predicts remaining life. Multimodal fusion effectively compensates for the limitations of a single data source, improves diagnostic robustness under complex working conditions, and reduces false alarm rates. A dynamic warning mechanism shortens the fault response cycle, and a predictive maintenance strategy reduces unplanned downtime losses, extends equipment service life, and achieves an optimal balance between operation and maintenance costs and power generation benefits.

[0103] In some optional implementations, the preset fault warning model is a bidirectional long short-term memory network + attention mechanism + fully connected layer network structure, the preset life prediction model is a physical information constraint model based on the long short-term memory network, and the method further includes: Step b1, calculate the first prediction variance of the fault warning result and the second prediction variance of the life prediction result. If the first prediction variance or the second prediction variance exceeds the preset variance threshold, the feature weight in the historical operating condition database is updated using the multi-dimensional synchronous data obtained in real time.

[0104] Specifically, this embodiment utilizes a collaborative mechanism of active learning and incremental learning to achieve continuous model optimization. Assuming a preset variance threshold of 10% calculated through Monte Carlo Dropout, when the model's predicted variance exceeds 10%, the system automatically marks the sample as high uncertainty. Multimodal feature vectors (dimensions 256+512+20), such as vibration waveforms and infrared thermal images, are packaged and pushed to a cloud-based expert system via the edge. Experts use an interactive annotation platform to annotate fault modes (supporting preset or custom types) and assign a confidence score of 0-10 to the annotation results.

[0105] Step b2: updating the multimodal data fusion features based on the updated feature weights and multidimensional synchronous data, and using the updated multimodal data fusion features to dynamically optimize the preset fault warning model and the preset life prediction model, respectively, and update the model parameters.

[0106] Specifically, the labeled data is filtered for quality (score ≥ 7) and redundancy eliminated (based on cosine similarity) before entering the incremental learning queue. Incremental learning uses the elastic weight consolidation algorithm (EWC), whose core formula is:

[0107] Among them: L_new is the cross entropy loss of new samples, is the Fisher information matrix of the old parameters (calculated using old data to reflect the importance of the parameters), s_i is the expert annotation confidence (0≤s_i≤1), λ∈[0.1,1.0] is the elasticity coefficient, β∈[0.5, 2.0] is the expert weight, and η uses cosine annealing scheduling. .

[0108] To prevent catastrophic forgetting, the system maintains a knowledge protection zone, imposes stronger regularization on parameters related to physical constraints (such as the Paris crack growth law), and transfers the old model output as soft labels to the new model through progressive knowledge distillation.

[0109] The edge and cloud perform collaborative optimization through model fingerprints (key parameter hash values) and abnormal event logs. The cloud periodically calculates model differences (KL divergence). When the model health index: ( ) is lower than the threshold, automatically triggering update strategies of different strengths: initial stage (0-2 years): λ=0.1, β=0.5, focusing on knowledge accumulation; mid-term (2-5 years): λ=0.5, β=1.0, balancing new and old knowledge; late stage (>5 years): λ=1.0, β=2.0, strengthening the protection of old knowledge, for example only, but not limited to this.

[0110] The fault warning and life prediction method for wind turbines provided in the embodiment of the present invention triggers a model optimization process when the variance of the prediction result is too large, updates the model parameters to adapt to new failure modes and changes brought about by equipment aging, and utilizes rich data resources and powerful computing power to continuously optimize model performance and improve the adaptability and accuracy of the system.

[0111] This embodiment also provides a fault warning and life prediction system for a wind turbine generator set. This system is used to implement the above-mentioned embodiments and preferred embodiments, and details already described will not be repeated. As used below, the term "module" may refer to a combination of software and / or hardware that implements a predetermined function. Although the systems described in the following embodiments are preferably implemented in software, implementation using hardware, or a combination of software and hardware, is also possible and contemplated.

[0112] This embodiment provides a fault warning and life prediction system for a wind turbine generator set. Figure 3 Shown, including: The multi-dimensional data acquisition module 301 is used to obtain the first state data of the target wind turbine generator set using the data acquisition and monitoring system, obtain the second state data of the target wind turbine generator set using the state monitoring system, and obtain the image data of the target wind turbine generator set using the image acquisition device, and pre-process the first state data, the second state data, and the image data respectively to obtain multi-dimensional data.

[0113] The multimodal data fusion module 302 is used to fuse the multidimensional data using a multimodal fusion model to obtain the multimodal data fusion features of the target wind turbine generator set.

[0114] The early warning and prediction module 303 is used to perform fault early warning and / or life prediction on the target wind turbine generator set based on the multimodal data fusion feature.

[0115] In some optional implementations, the multidimensional data acquisition module 301 includes: The first data processing unit is used to perform cleaning, standardization, filtering, and smoothing on the first state data in sequence to obtain first dimensional data.

[0116] The second data processing unit is used to clean and resample the second state data, and extract the fault-sensitive characteristic value from the resampled vibration signal as the second dimensional data.

[0117] The third data processing unit is used for image data including infrared images and visible light images, converting infrared images into temperature distribution matrices, calculating local temperature gradients, marking areas where local temperature gradients exceed a preset gradient threshold as abnormalities, performing oil stain segmentation on visible light images including oil stain areas to obtain mask images, and performing feature point matching on visible light images including couplings to obtain coupling position offsets.

[0118] In some optional implementations, the multimodal data fusion module 302 includes: The time synchronization unit is used to perform time synchronization processing on the second state data and the image data based on the timestamp of the first state data to obtain multi-dimensional synchronized data.

[0119] The feature fusion unit is used to fuse the multi-dimensional synchronous data using a preset cross-modal fusion model to obtain multi-modal data fusion features.

[0120] In some optional implementations, the early warning prediction module 303 includes: The real-time data acquisition unit is used to acquire multimodal data fusion features in real time, and use a preset fault warning model to process the multimodal data fusion features to obtain the probability distribution of multiple preset fault types.

[0121] The fault prediction and warning unit is used to determine the warning level according to the probability distribution of each preset fault type and the preset fault warning threshold, and issue a warning prompt according to the warning level.

[0122] The feature sequence generation unit is used to obtain multiple historical multimodal data fusion features within a preset time period to form a historical fusion feature sequence.

[0123] The life prediction unit is used to input the historical fusion feature sequence into the preset life prediction model to obtain the predicted remaining life of the target wind turbine generator set.

[0124] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0125] The fault warning and life prediction system of the wind turbine generator set in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0126] The embodiment of the present invention also provides a computer device having the above Figure 3 The fault warning and life prediction system of the wind turbine generator set shown.

[0127] See also Figure 4 , Figure 4 is a structural diagram of a computer device provided by an optional embodiment of the present invention, such as Figure 4 As shown, the computer device includes: one or more processors 10, memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components utilize different buses to communicate with each other and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in the memory or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Equally, multiple computer devices can be connected, and each device provides part of the necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 4 A processor 10 is taken as an example.

[0128] The processor 10 may be a central processing unit, a network processor, or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit, a programmable logic device, or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic, or any combination thereof.

[0129] The memory 20 stores instructions that can be executed by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0130] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and application programs required for at least one function; the data storage area may store data created based on the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely located relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0131] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0132] The computer device further includes a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0133] The embodiment of the present invention also provides a computer-readable storage medium. The above-mentioned method according to the embodiment of the present invention can be implemented in hardware, firmware, or implemented as a computer code that can be recorded in a storage medium, or implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and downloaded through a network and will be stored in a local storage medium, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state drive, etc.; further, the storage medium can also include a combination of the above-mentioned types of memory. It can be understood that a computer, a processor, a microprocessor controller or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor or hardware, the method shown in the above embodiment is implemented.

[0134] Although the embodiments of the present invention have been described with reference to the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention. Such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A method for fault warning and life prediction of a wind turbine generator set, characterized in that: The method comprises: Acquire first state data of a target wind turbine generator set using a data acquisition and monitoring system, acquire second state data of the target wind turbine generator set using a state monitoring system, acquire image data of the target wind turbine generator set using an image acquisition device, and pre-process the first state data, the second state data, and the image data to obtain multidimensional data; fusing the multidimensional data using a multimodal fusion model to obtain multimodal data fusion features of the target wind turbine generator set; Based on the multimodal data fusion features, fault warning and / or life prediction are performed on the target wind turbine generator set.

2. The method according to claim 1, characterized in that Preprocessing the first state data, the second state data, and the image data respectively to obtain multidimensional data includes: performing cleaning, standardization, filtering, and smoothing processing on the first state data in sequence to obtain first-dimensional data; Cleaning and resampling the second state data, and extracting fault-sensitive characteristic values from the resampled vibration signal as second-dimensional data; The image data includes infrared images and visible light images. The infrared images are converted into a temperature distribution matrix, and the local temperature gradient is calculated. Areas where the local temperature gradient exceeds a preset gradient threshold are marked as abnormal. The visible light image including the oil stain area is segmented to obtain a mask image. The visible light image including the coupling is matched with feature points to obtain the coupling position offset.

3. The method according to claim 1, characterized in that The multi-dimensional data is fused using a multi-modal fusion model to obtain multi-modal data fusion features of the target wind turbine generator set, including: Based on the timestamp of the first state data, the second state data and the image data are subjected to time synchronization processing to obtain multi-dimensional synchronized data; The multi-dimensional synchronized data is fused using a preset cross-modal fusion model to obtain multi-modal data fusion features.

4. The method according to claim 3, characterized in that The multi-dimensional synchronized data is fused using a preset cross-modal fusion model to obtain multi-modal data fusion features, including: Perform spatial mapping on each multi-dimensional synchronization data to obtain multi-dimensional synchronization alignment data in the same latent space; Calculate the similarity between the multi-dimensional synchronous alignment data and the feature data in the historical working condition database, and determine the corresponding working condition and attention weight based on the similarity calculation results; Based on the multi-dimensional synchronous alignment data and the corresponding attention weights, weighted calculation is performed to obtain multi-dimensional fusion features; The multi-dimensional fusion features are standardized to obtain multi-modal data fusion features.

5. The method according to claim 4, characterized in that The construction process of the historical operating condition database includes: Acquire historical multi-dimensional synchronous alignment data, cluster the historical multi-dimensional synchronous alignment data, and generate a working condition label library; By using feature importance analysis, the feature weights corresponding to each multi-dimensional synchronous alignment data under different working condition labels are determined. The historical multi-dimensional synchronous alignment data and the corresponding working condition labels and feature weights constitute a historical working condition database.

6. The method according to claim 5, characterized in that Performing fault warning and / or life prediction on a target wind turbine generator set based on the multimodal data fusion feature includes: Acquire multimodal data fusion features in real time, and process the multimodal data fusion features using a preset fault warning model to obtain probability distributions of multiple preset fault types; Determine the warning level based on the probability distribution of each preset fault type and the preset fault warning threshold, and issue a warning prompt based on the warning level; Acquire multiple historical multimodal data fusion features within a preset time period to form a historical fusion feature sequence; The historical fusion feature sequence is input into a preset life prediction model to obtain the predicted remaining life of the target wind turbine generator set.

7. The method according to claim 6, characterized in that The preset fault warning model is a bidirectional long short-term memory network + attention mechanism + fully connected layer network structure, and the preset life prediction model is a physical information constraint model based on the long short-term memory network. The method further includes: Calculating a first prediction variance of a fault warning result and a second prediction variance of a life prediction result; if the first prediction variance or the second prediction variance exceeds a preset variance threshold, updating feature weights in a historical operating condition database using multi-dimensional synchronous data acquired in real time; The multimodal data fusion features are updated based on the updated feature weights and multidimensional synchronous data, and the updated multimodal data fusion features are used to dynamically optimize the preset fault warning model and the preset life prediction model, and update the model parameters.

8. A fault warning and life prediction system for a wind turbine generator set, characterized in that: The system comprises: a multidimensional data acquisition module, configured to acquire first state data of a target wind turbine generator set using a data acquisition and monitoring system, acquire second state data of the target wind turbine generator set using a state monitoring system, acquire image data of the target wind turbine generator set using an image acquisition device, and pre-process the first state data, second state data, and image data to obtain multidimensional data; A multimodal data fusion module is used to fuse the multidimensional data using a multimodal fusion model to obtain multimodal data fusion features of the target wind turbine generator set; The early warning and prediction module is used to perform fault early warning and / or life prediction on the target wind turbine generator set based on the multimodal data fusion characteristics.

9. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the method according to any one of claims 1 to 7 by executing the computer instructions.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the method according to any one of claims 1 to 7.

Images