Water-turbine generator set intelligent monitoring system and method based on LSTM-Transform multi-modal fusion

Abstract

The invention discloses an intelligent monitoring system and method for a water-turbine generator set based on LSTM-Transform multi-modal fusion, and provides an innovative deep learning solution aiming at the defects of the existing monitoring technology in the aspects of multi-source data fusion, early fault recognition and cross-spatio-temporal scale feature extraction. According to the system, multi-modal monitoring data such as vibration signals, infrared thermal imaging and oil analysis are integrated, time sequence dynamic features are captured by using an LSTM network, cross-modal feature fusion is realized in combination with a Transform global attention mechanism, and a comprehensive unit health state evaluation model is constructed. According to the method, the early abnormal symptoms of unit operation can be effectively recognized, the accuracy and timeliness of fault early warning are remarkably improved, meanwhile, different working conditions are adapted through a self-adaptive learning mechanism, and reliable technical support is provided for intelligent operation and maintenance of the water-turbine generator set. The system has the characteristics of flexible deployment and strong expansibility, and can be widely applied to state monitoring and maintenance decision-making of various hydroelectric power generation equipment.

Description

A Smart Monitoring System and Method for Hydropower Generator Units Based on LSTM-Transformer Multimodal Fusion Technical Field

[0001] This invention belongs to the field of intelligent monitoring technology for hydropower equipment, and in particular relates to an intelligent monitoring system and method for hydro-generator units based on LSTM-Transformer multimodal fusion. Background Technology

[0002] As the core equipment of hydropower generation, the operating status of hydro-turbine generator units directly affects grid security and power generation efficiency. Currently, mainstream monitoring methods mainly rely on the analysis of single physical quantities such as vibration and temperature, which has obvious technical limitations: 1. Insufficient early fault detection capability: Traditional vibration monitoring is slow to respond to progressive faults such as minor wear and local overheating, often triggering early warnings only in the middle and late stages of the fault development, missing the best maintenance opportunity; 2. Lack of multi-parameter collaborative analysis: Mechanical vibration, thermodynamic state, lubrication wear and other indicators are analyzed independently, lacking cross-modal correlation models, making it difficult to comprehensively assess the health status of the unit; 3. Data heterogeneity challenge: High-frequency vibration signals, infrared thermal imaging, and oil analysis have large differences in spatiotemporal scales, making it difficult for existing algorithms to effectively integrate them; 4. Limited level of intelligence: Relying on manual experience to set thresholds, it cannot adapt to different operating conditions such as variable load and start-up and shutdown stages, resulting in high false alarm and false negative rates. Therefore, it is necessary to propose an intelligent monitoring method for hydro-turbine generator units based on LSTM-Transformer multimodal fusion to solve the above problems. Summary of the Invention

[0003] The technical problem to be solved by this invention is to provide an intelligent monitoring system and method for hydro-generator units based on LSTM-Transformer multimodal fusion. It aims to solve the problems of insufficient fusion of multi-source heterogeneous data, low early fault identification rate, and difficulty in feature extraction across time and space scales in existing hydro-generator unit condition monitoring technologies. By combining the temporal modeling capability of LSTM and the global feature extraction advantage of Transformer, it achieves efficient fusion and deep feature mining of multimodal monitoring data such as vibration, temperature, and oil, which significantly improves the accuracy of early fault warning and the comprehensiveness of condition assessment.

[0004] To address the aforementioned technical problems, the technical solution adopted in this invention is: an intelligent monitoring method for hydro-generator units based on LSTM-Transformer multimodal fusion, comprising the following steps: synchronously acquiring vibration signals, infrared thermal imaging data, and oil analysis data; performing spatiotemporal alignment processing on multi-source heterogeneous data through a three-level data alignment mechanism; extracting the temporal dynamic features of vibration signals using an LSTM network; extracting the spatial distribution features of infrared thermal imaging using a CNN network; realizing cross-modal correlation analysis of vibration features, infrared features, and oil chemical features through a Transformer fusion layer; and outputting a unit health status score and fault early warning signal.

[0005] Preferably, the three-level data alignment mechanism includes: time alignment: establishing an interpolation mapping function between the high-frequency sampling timestamp of the vibration signal (≥10kHz) and the low-frequency acquisition period of the infrared image (0.1-1Hz); spatial alignment: mapping the installation coordinates of the vibration sensor to the pixel coordinate system of the infrared image based on the three-dimensional CAD model of the unit; and semantic alignment: constructing an empirical correlation matrix between the oil metal content index and the vibration spectrum characteristics.

[0006] Preferably, the specific implementation of the Transformer fusion layer includes: setting up three parallel attention heads to process respectively: the correlation between vibration spectrum features and thermal gradient distribution as attention head 1; the correlation between vibration envelope features and oil viscosity change rate as attention head 2; and the joint projection of the three-modal features in the latent space as attention head 3.

[0007] Preferably, a modal balance loss function is introduced: ;in, Vibration signal reconstruction loss (using weighted mean square error): ; For frequency band energy weighting, satisfying The weighting of high-frequency components above 3kHz is increased by 20%-30%. Infrared feature classification loss (using focal loss): ; Hot spot category The predicted probability, For focusing parameters (preferred) ); Loss prediction for oil performance indicators (using log-cosine similarity): ; A small constant to prevent numerical overflow; Cross-modal correlation loss (based on HSIC criterion): ; Here is the Gram matrix of each modal feature. This represents the Hadamard product.

[0008] Preferably, the parameter constraints of the modal balance loss function are: modal weight coefficients. , Regularization coefficient Preferred .

[0009] Preferably, the processing of the oil analysis data includes: calculating the chemical-physical coupling coefficient (CPC). ;in, Indicates the first The iron content (ppm) of the secondary oil liquid was determined by ferrography. Indicates the vibration signal at the 1st The spectral energy norm of the frequency band The normalized weight coefficients satisfy... ; This represents the value of the i-th oil ferrography analysis. Let represent the vibration energy of the j-th frequency band, and α and β be weighting coefficients.

[0010] Preferably, the sliding window index (SWI) is constructed to reflect the historical degradation trend: ; ;in, The value is the length of the sliding window, which is 5 ≤ ​​m ≤ 15. For time decay weight, The attenuation factor is 0.1≤γ≤0.3.

[0011] Preferably, the infrared thermal imaging processing includes: locating key component areas such as bearings and windings using a target detection network; and generating region attention masks. ;in, For the Sigmoid function, coordinates Temperature value at that location, The average temperature of the component area. For temperature standard deviation, For adjustable bias, -0.5≤b≤0.5; apply a suppression coefficient of 0.2-0.5 to non-critical region features.

[0012] Preferably, an intelligent monitoring system for hydro-generator units based on LSTM-Transformer multimodal fusion is provided to implement the aforementioned intelligent monitoring method for hydro-generator units based on LSTM-Transformer multimodal fusion. The system includes: a multi-source data acquisition module containing a vibration accelerometer, an infrared thermal imager, and an online oil sensor; an edge computing unit deploying an LSTM vibration feature extraction model to perform data preprocessing; a cloud analysis platform running the Transformer multimodal fusion model; and a visualization terminal displaying a three-dimensional health status map and fault tracing path.

[0013] Preferably, the data transmission between the edge computing unit and the cloud platform adopts the following methods: vibration data, transmitting 128-dimensional feature vectors compressed by LSTM; infrared images, transmitting only the ROI area of ​​key components, occupying ≤30% of the original image area; and oil data, transmitting the SWI index and CPC coefficient.

[0014] The beneficial effects of this invention are as follows: 1. Enhanced intelligent monitoring capabilities: Through the innovative LSTM-Transformer fusion architecture, deep feature extraction and intelligent analysis of multi-source heterogeneous monitoring data of the unit are realized, which significantly improves the accuracy of early fault identification and the comprehensiveness of status assessment, and solves the problems of slow response and insufficient feature extraction in traditional methods.

[0015] 2. Multimodal collaborative analysis and optimization: The proposed cross-modal fusion mechanism effectively integrates multi-dimensional monitoring information such as vibration, temperature, and oil, overcoming the technical bottleneck caused by differences in the spatiotemporal scales of different data, and providing a more accurate comprehensive evaluation basis for the health status of the unit.

[0016] 3. Adaptive Operation and Maintenance System Innovation: The system has the ability to learn dynamically and continuously optimize, and can adapt to the monitoring needs under different operating conditions. It realizes the transformation of the operation and maintenance mode from passive inspection to proactive prevention, and provides reliable technical support for the intelligent management of hydropower stations. Attached Figure Description

[0017] Figure 1 is a flowchart of the overall system architecture of the present invention; Figure 2 is a flowchart of the multimodal data preprocessing of the present invention; Figure 3 is a flowchart of the LSTM-Transformer hybrid model training of the present invention; Figure 4 is a flowchart of the online monitoring and early warning of the present invention. Detailed Implementation

[0018] Example 1: As shown in Figures 1-4, an intelligent monitoring method for hydro-generator units based on LSTM-Transformer multimodal fusion includes the following steps: synchronously acquiring vibration signals, infrared thermal imaging data, and oil analysis data; performing spatiotemporal alignment processing on multi-source heterogeneous data through a three-level data alignment mechanism; extracting the temporal dynamic features of vibration signals using an LSTM network; extracting the spatial distribution features of infrared thermal imaging using a CNN network; realizing cross-modal correlation analysis of vibration features, infrared features, and oil chemical features through a Transformer fusion layer; and outputting a unit health status score and fault warning signal.

[0019] Preferably, the three-level data alignment mechanism includes: time alignment: establishing an interpolation mapping function between the high-frequency sampling timestamp of the vibration signal (≥10kHz) and the low-frequency acquisition period of the infrared image (0.1-1Hz); spatial alignment: mapping the installation coordinates of the vibration sensor to the pixel coordinate system of the infrared image based on the three-dimensional CAD model of the unit; and semantic alignment: constructing an empirical correlation matrix between the oil metal content index and the vibration spectrum characteristics.

[0020] Preferably, the specific implementation of the Transformer fusion layer includes: setting up three parallel attention heads to process respectively: the correlation between vibration spectrum features and thermal gradient distribution as attention head 1; the correlation between vibration envelope features and oil viscosity change rate as attention head 2; and the joint projection of the three-modal features in the latent space as attention head 3.

[0021] Preferably, a modal balance loss function is introduced: ;in, Vibration signal reconstruction loss (using weighted mean square error): ; For frequency band energy weighting, satisfying The weighting of high-frequency components above 3kHz is increased by 20%-30%. Infrared feature classification loss (using focal loss): ; Hot spot category The predicted probability, For focusing parameters (preferred) ); Loss prediction for oil performance indicators (using log-cosine similarity): ; A small constant to prevent numerical overflow; Cross-modal correlation loss (based on HSIC criterion): ; Here is the Gram matrix of each modal feature. This represents the Hadamard product.

[0022] Preferably, the parameter constraints of the modal balance loss function are: modal weight coefficients. , Regularization coefficient Preferred .

[0023] Preferably, the processing of the oil analysis data includes: calculating the chemical-physical coupling coefficient (CPC). ;in, Indicates the first The iron content (ppm) of the secondary oil liquid was determined by ferrography. Indicates the vibration signal at the 1st The spectral energy norm of the frequency band The normalized weight coefficients satisfy... ; This represents the value of the i-th oil ferrography analysis. Let represent the vibration energy of the j-th frequency band, and α and β be weighting coefficients.

[0024] Preferably, the sliding window index (SWI) is constructed to reflect the historical degradation trend: ; ;in, The value is the length of the sliding window, which is 5 ≤ ​​m ≤ 15. For time decay weight, The attenuation factor is 0.1≤γ≤0.3.

[0025] Preferably, the infrared thermal imaging processing includes: locating key component areas such as bearings and windings using a target detection network; and generating region attention masks. ;in, For the Sigmoid function, coordinates Temperature value at that location, The average temperature of the component area. For temperature standard deviation, For adjustable bias, -0.5≤b≤0.5; apply a suppression coefficient of 0.2-0.5 to non-critical region features.

[0026] Preferably, an intelligent monitoring system for hydro-generator units based on LSTM-Transformer multimodal fusion is provided to implement the aforementioned intelligent monitoring method for hydro-generator units based on LSTM-Transformer multimodal fusion. The system includes: a multi-source data acquisition module containing a vibration accelerometer, an infrared thermal imager, and an online oil sensor; an edge computing unit deploying an LSTM vibration feature extraction model to perform data preprocessing; a cloud analysis platform running the Transformer multimodal fusion model; and a visualization terminal displaying a three-dimensional health status map and fault tracing path.

[0027] Preferably, the data transmission between the edge computing unit and the cloud platform adopts the following methods: vibration data, transmitting 128-dimensional feature vectors compressed by LSTM; infrared images, transmitting only the ROI area of ​​key components, occupying ≤30% of the original image area; and oil data, transmitting the SWI index and CPC coefficient.

[0028] Example 2: This example provides a specific application method, including the following process: I. System Architecture Implementation: 1. Hardware Deployment: Vibration acceleration sensors (sampling frequency 10kHz), infrared thermal imagers (resolution 640×512, frame rate 30Hz), and online oil monitoring instruments (ferrography analysis cycle 1 hour) are installed at key parts of the unit; industrial-grade GPU servers are used for edge computing nodes, and data preprocessing modules are deployed; an LSTM-Transformer fusion analysis platform is deployed in the cloud. 2. Data Processing Flow: Vibration signal: Energy features in the 0-5kHz frequency band are extracted using 5-layer wavelet packet decomposition; Infrared image: ROI of key areas such as bearings and windings is located based on the improved YOLOv5 model; Oil data: Sliding window index (SWI) is constructed to characterize the trend of iron content change.

[0029] II. Model Training Implementation: 1. Network Structure Parameters: LSTM Branch: Two-layer structure, 128 hidden units, dropout rate 0.2; Transformer Branch: 4-layer encoder, 8 attention heads, feedforward network dimension 512; Fusion Layer: Dynamic weight coefficient λ initial value 0.6, adaptive adjustment range ±0.1. 2. Training Strategy: Adam optimizer used, initial learning rate 0.001; batch size 64, early stopping mechanism (patience=15); loss function combination: weighted MSE (vibration) + focus loss (infrared) + HSIC correlation constraint.

[0030] III. Implementation of Online Monitoring: 1. Real-time Analysis Process: Edge nodes upload 128-dimensional feature vectors every 5 seconds; the cloud generates a health status score (0-100) every 10 minutes; when the score is below 70, a three-level early warning mechanism is triggered. 2. Typical Fault Detection: Bearing wear: Vibration energy rises at 3-5kHz + iron content increase > 5ppm / day; Winding overheating: Local temperature rise > 15K + sudden change in high-frequency components of vibration characteristics; Oil deterioration: SWI index exceeds the threshold 3 times consecutively + abnormal viscosity change rate. 3. Typical Fault Handling Process: Three-level early warning mechanism: (1). Primary early warning (health 80-70): Generate monitoring report; (2). Intermediate early warning (70-60): Trigger review process; (3). Advanced early warning (<60): Initiate emergency response.

[0031] Example: Upper guide bearing wear treatment (1). D-30 days: The system detected a 5dB increase in sideband energy at 3.2kHz; (2). D-15 days: Ferrography showed that the particle concentration exceeded the threshold (>15ppm); (3). D-7 days: Infrared thermography found a local temperature rise of 8K; (4). D-1 day: The overall failure probability was determined to be >85%, and a shutdown for maintenance was arranged.

[0032] In use, this invention combines the temporal modeling capabilities of LSTM with the global feature extraction advantages of Transformer to achieve efficient fusion and deep feature mining of multimodal monitoring data such as vibration, temperature, and oil levels. The system can effectively capture subtle abnormal changes in unit operation, identifying potential equipment degradation trends before conventional monitoring methods detect obvious fault signs. This provides a sufficient time window for preventative maintenance and helps improve the overall economic efficiency of operation and maintenance.

Claims

1. A smart monitoring method for hydro-generator units based on LSTM-Transformer multimodal fusion, characterized in that, Includes the following steps: Simultaneously acquire vibration signals, infrared thermal imaging data, and oil analysis data; A three-level data alignment mechanism is used to perform spatiotemporal alignment processing on multi-source heterogeneous data; an LSTM network is used to extract the temporal dynamic features of vibration signals; and a CNN network is used to extract the spatial distribution features of infrared thermal imaging. The Transformer fusion layer enables cross-modal correlation analysis of vibration characteristics, infrared characteristics, and oil chemical characteristics; it outputs unit health status scores and fault warning signals.

2. The intelligent monitoring method for hydro-generator units based on LSTM-Transformer multimodal fusion according to claim 1, characterized in that, The three-level data alignment mechanism includes: time alignment: establishing an interpolation mapping function between the high-frequency sampling timestamp of the vibration signal and the low-frequency acquisition period of the infrared image; spatial alignment: mapping the installation coordinates of the vibration sensor to the pixel coordinate system of the infrared image based on the three-dimensional CAD model of the unit; and semantic alignment: constructing an empirical correlation matrix between the oil metal content index and the vibration spectrum characteristics.

3. The intelligent monitoring method for hydro-generator units based on LSTM-Transformer multimodal fusion according to claim 1, characterized in that, The specific implementation of the Transformer fusion layer includes setting up three parallel attention heads to process: the correlation between vibration spectrum features and thermal gradient distribution as attention head 1; the correlation between vibration envelope features and oil viscosity change rate as attention head 2; and the joint projection of the three-modal features in the latent space as attention head 3.

4. The intelligent monitoring method for hydro-generator units based on LSTM-Transformer multimodal fusion according to claim 3, characterized in that, Introducing the modal balance loss function: ;in, For vibration signal reconstruction loss: ； For frequency band energy weighting, satisfying ； For infrared feature classification loss: ； Hot spot category The predicted probability, For focusing parameters; Predicting losses based on oil performance indicators: ； A small constant to prevent numerical overflow; For cross-modal correlation loss: ； Here is the Gram matrix of each modal feature. This represents the Hadamard product.

5. The intelligent monitoring method for hydro-generator units based on LSTM-Transformer multimodal fusion according to claim 4, characterized in that, The parameter constraints of the modal balance loss function are: modal weight coefficients. , Regularization coefficient 。 6. The intelligent monitoring method for hydro-generator units based on LSTM-Transformer multimodal fusion according to claim 1, characterized in that, The processing of the oil analysis data includes: calculating the chemical-physical coupling coefficient (CPC). ;in, Indicates the first The iron content (ppm) of the secondary oil liquid was determined by ferrography. Indicates the vibration signal at the 1st The spectral energy norm of the frequency band The normalized weight coefficients satisfy... ； This represents the value of the i-th oil ferrography analysis. Let represent the vibration energy of the j-th frequency band, and α and β be weighting coefficients.

7. The intelligent monitoring method for hydro-generator units based on LSTM-Transformer multimodal fusion according to claim 1, characterized in that, Constructing a sliding window index (SWI) to reflect historical degradation trends: ； ;in, The length of the sliding window. For time decay weight, This is the attenuation factor.

8. The intelligent monitoring method for hydro-generator units based on LSTM-Transformer multimodal fusion according to claim 1, characterized in that, The infrared thermal imaging processing includes: locating key component areas such as bearings and windings using a target detection network; and generating region attention masks. ;in, For the Sigmoid function, coordinates Temperature value at that location, The average temperature of the component area. For temperature standard deviation, The bias is adjustable; a suppression coefficient of 0.2-0.5 is applied to non-critical region features.

9. A smart monitoring system for hydro-generator units based on LSTM-Transformer multimodal fusion, used to implement the smart monitoring method for hydro-generator units based on LSTM-Transformer multimodal fusion as described in any one of claims 1-8, characterized in that, The system includes: a multi-source data acquisition module containing a vibration accelerometer, an infrared thermal imager, and an online oil sensor; an edge computing unit that deploys an LSTM vibration feature extraction model to perform data preprocessing; a cloud analysis platform that runs a Transformer multimodal fusion model; and a visualization terminal that displays a 3D health status map and fault tracing path.

10. The intelligent monitoring system for hydro-generator units based on LSTM-Transformer multimodal fusion according to claim 9, characterized in that, The data transmission between the edge computing unit and the cloud platform adopts the following methods: vibration data, transmitting 128-dimensional feature vectors compressed by LSTM; infrared images, transmitting only the ROI areas of key components; and oil data, transmitting the SWI index and CPC coefficient.

Images