Hydroelectric equipment multi-modal data analysis early warning method

Abstract

The invention belongs to the field of hydroelectric equipment monitoring and early warning, and particularly provides a hydroelectric equipment multi-modal data analysis and early warning method, which comprises the following steps: acquiring related data of hydroelectric equipment, performing data preprocessing, and generating a first hydroelectric equipment data set; performing feature extraction and feature fusion on the first hydroelectric equipment data set to obtain a high-level fusion feature; performing anomaly recognition on the fusion features, and correcting abnormal data points to obtain corrected high-quality fusion features; key index data in equipment operation are extracted, a hydroelectric equipment operation state model is constructed, an equipment special operation trend is identified in real time, and an equipment health state score is calculated; and monitoring the state of the equipment in real time, obtaining data in real time, comparing the data with a predicted value, and judging that the equipment is abnormal when an absolute error is greater than a preset error threshold value. The method achieves the analysis of the multi-dimensional data of the hydroelectric equipment, and can achieve the fault prediction of the state of the equipment.

Description

Technical Field

[0001] This invention belongs to the field of hydropower equipment monitoring and early warning, specifically, it relates to a method for multimodal data analysis and early warning of hydropower equipment. Background Technology

[0002] In the management and operation of hydropower equipment, traditional analysis methods and scenarios are relatively limited, making it difficult to integrate video surveillance, time-series data, management data, and other data. The lack of automated and intelligent analysis tools further complicates real-time monitoring of equipment status and fault early warning. Therefore, there is an urgent need to design a self-service, specialized analysis method for hydropower equipment that can integrate multimodal data from various equipment.

[0003] CN114154814A proposes an adaptive learning-based trend analysis and early warning system and method for hydropower equipment. Through real-time data analysis and fault mining, it automatically diagnoses faults in hydropower equipment, ensuring the safe and stable operation of hydropower plants, saving manpower and resources, and improving equipment management efficiency. It features self-learning and adaptive capabilities, continuously optimizing analysis results based on the latest data, aiming to improve the monitoring and fault early warning capabilities of hydropower equipment. However, this technical solution lacks sufficient data fusion capabilities, struggling to effectively integrate multi-source data such as video surveillance and equipment operating status, leading to information silos. This limitation results in a lack of comprehensive understanding of the overall operating status of the equipment. Furthermore, although online monitoring and trend analysis are introduced, truly intelligent management methods are lacking; the automation level of real-time monitoring and fault early warning remains low, failing to fully utilize advanced data analysis technologies.

[0004] CN119004301A proposes a fault analysis method for hydropower equipment based on an industrial internet platform. Through operating condition analysis, time trend analysis, frequency trend analysis, and correlation analysis, it utilizes big data and intelligent algorithms to deeply mine equipment status data and identify potential faults. By integrating real-time data, using intelligent analysis and visualization tools, it helps maintenance teams quickly understand the causes of faults, formulate preventative maintenance strategies, optimize equipment management, and promote the intelligent development of the hydropower industry. However, in practical applications, this technical solution still faces challenges in data integration, especially when effectively integrating real-time monitoring data with historical data and video surveillance information, often leading to one-sided analysis results. Secondly, the lack of sufficient automation and intelligent means, relying on manual intervention for fault diagnosis, limits the timeliness and accuracy of the analysis and fails to fully leverage the advantages of the industrial internet.

[0005] Currently, the challenges in managing and operating hydropower equipment lie in fragmented data collection, outdated monitoring methods, and over-reliance on manual inspections. Sensor data quality varies significantly, and information silos are prevalent, preventing real-time fusion and intelligent analysis of multi-source data. This leads to delayed fault warnings and insufficient data for maintenance decisions. Furthermore, a lack of inter-system coordination negatively impacts overall safety and operational efficiency. Summary of the Invention

[0006] The technical problem to be solved by the present invention is to provide a multimodal data analysis and early warning method for hydropower equipment, which analyzes multi-dimensional data of hydropower equipment and can predict equipment failure status.

[0007] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is: a multimodal data analysis and early warning method for hydropower equipment, comprising the following steps: Step 1: Multi-source data acquisition and preprocessing: Acquire video surveillance data, equipment operation status data, physical coordinate data, and historical operation data of the hydropower equipment, and perform data preprocessing to generate the first hydropower equipment dataset; Step 2, Key Feature Extraction: Feature extraction and feature fusion are performed on the first hydropower equipment dataset generated in Step 1. Feature extraction includes video feature extraction from video data and temporal feature extraction from data sequences. Then, the video features and temporal features are fused to obtain high-level fused features. Step 3, Abnormal Data Identification and Correction: Abnormal data points are identified in the fusion features generated in Step 2. The abnormal data points are then corrected to obtain high-quality fusion features after correction. Step 4: Specific Analysis: Based on the corrected high-quality fusion features, extract key indicator data during equipment operation. These key indicator data correspond to different specific equipment analysis scenarios. A multivariate feature analysis and adaptive learning algorithm are used to construct a hydropower equipment operation status model to identify specific equipment operation trends in real time. Based on the key indicator data, the equipment health status score is calculated, and an equipment health decay factor is introduced to dynamically correct the monitoring threshold of specific equipment operation status. Step 5: Fault Prediction: Based on Steps 1-4, calculate the health status score for each historical moment, and establish a mapping relationship between operating conditions and health status scores to form a benchmark database; when performing fault prediction, calculate the current health status score based on real-time acquired equipment data. The current equipment operating environment parameters and working condition information are matched with the benchmark database to find the health status score corresponding to the historical operating scenario most similar to the current working condition, which is then used as the predicted health status score. , and The comparison is performed, and an anomaly is determined when the absolute error exceeds a preset error threshold.

[0008] In the preferred embodiment, step one, data preprocessing includes spatiotemporal alignment, which comprises time alignment and spatial alignment, and the operation is as follows: 1) Time alignment For each data sampling time Find two adjacent video frames in the video data, whose timestamps are respectively and ,satisfy: ; in, Earlier The latest video frame time, Later Latest video frame time; according to and The video data at each moment is used to calculate the value using an interpolation algorithm. The video data value corresponding to the given moment; 2) Spatial alignment Establish a unified spatial coordinate system: Construct a global three-dimensional coordinate system for the equipment based on its structural information and installation calibration data. Define the following in the global coordinate system: , and Three axes determine the physical location of each component of the equipment; Video data preprocessing and feature extraction: Image enhancement and preprocessing are performed on the acquired video data to identify and extract key areas or feature points of the device, and obtain the corresponding two-dimensional coordinates in the image. ,in The x-axis is... The vertical axis is used as the coordinate. Data spatial calibration: The physical location corresponding to the acquired data is determined as three-dimensional coordinates in the global coordinate system. ; Constructing a 3D-to-2D spatial mapping model: A pinhole camera model is used to establish the mapping relationship between global 3D coordinate points and 2D coordinate points in the video image. The basic mapping formula is as follows: ; in, This is a scaling factor used to normalize mapping relationships; Let be the homogeneous coordinate vector of the target point in the image; The camera intrinsic parameter matrix is ​​in the form of , With the camera in a horizontal position, For the camera to be in the vertical direction, The x-coordinate of the optical center in the image, The ordinate of the optical center in the image; The extrinsic parameter matrix of the camera is a 3×3 rotation matrix. and a 3×1 translation vector Composition, used to describe the rotation and translation relationship of the camera relative to the global coordinate system; The homogeneous coordinates of the target point in the global coordinate system; Spatial alignment and data fusion: Utilizing the mapping relationships in the constructed 3D to 2D spatial mapping model, the position of data in the global coordinate system is determined. Transform to the corresponding position in the video image coordinate system .

[0009] In the preferred embodiment, in step two, a convolutional neural network is used to process the input video data. Low-level feature extraction is performed to obtain low-level video features. Temporal features are extracted from the data sequence S using a Long Short-Term Memory (LSTM) network. Using attention mechanisms to and Perform feature fusion to generate unified high-level fused features. .

[0010] In the preferred embodiment, in step three, high-level fusion features are obtained. mean Standard deviation Maximum value and minimum value ; through the maximum value and minimum value To address the characteristics of high-level integration Perform normalization processing; for any data point i High-level fusion features When the conditions are met If so, the data point is determined to be abnormal data. ,in This is a preset constant.

[0011] In the preferred embodiment, the operation of correcting abnormal data points in step three is as follows: For the identified abnormal data A linear regression model was established by combining historical normal data, and the time variable was analyzed. The regression formula for fitting the data is as follows: ; in, and These are the regression coefficients obtained by fitting historical data. These are regression predictions; Intelligent compensation is applied to outlier data based on regression prediction results, using the following compensation formula: ; in, This is the compensation coefficient, used to control the correction amplitude; These are the corrected data values.

[0012] In the preferred embodiment, in step four, the key indicator data extracted during equipment operation includes at least temperature data, vibration data, pressure data, and energy consumption data. The extracted key indicators correspond to the specific operating status of the equipment. Temperature data corresponds to equipment temperature analysis; vibration data corresponds to start-up and shutdown analysis, gas system and stability analysis; pressure data corresponds to speed regulating oil system and main transformer oil chromatography analysis; and energy consumption data corresponds to water collection and drainage system and oil level analysis.

[0013] In a preferred embodiment, step four, calculating the equipment health status score based on the key indicator data, includes: The data for each key indicator were normalized to obtain normalized values. , , and ; Preset weights are determined based on historical data and expert experience. , , and ; The weighted summation method is used to calculate the equipment health status score, and the formula is as follows: ; in, Rate the health status of the equipment. Equipment health degradation factor; The steps for obtaining the equipment health degradation factor include: Collecting the cumulative running time of the equipment t In addition to historical fault records and maintenance records; Fit the health degradation curve of the equipment and determine the equipment degradation constant. ; The health degradation factor of equipment is obtained using the exponential decay model, and its calculation formula is as follows: ; in, The cumulative operating time of the equipment. This is the decay constant obtained by fitting historical data. In a preferred embodiment, the step of introducing an equipment health degradation factor to dynamically correct the equipment-specific operating status monitoring threshold includes: The initial monitoring threshold was obtained based on long-term equipment operation data. ; Introducing equipment health degradation factor The monitoring threshold is then corrected using the following formula: ; in, This is the dynamically adjusted monitoring threshold.

[0014] In the preferred embodiment, step five, fault prediction, specifically includes the following steps: Get the current health status score in real time and the predicted value of health status score Compare and calculate the absolute error ; when Greater than the preset error threshold When this occurs, it is determined to be abnormal, and the degree of abnormality is calculated. The calculation formula is as follows: ; Based on the degree of abnormality The system will automatically trigger an early warning signal based on the current operating status.

[0015] The preferred solution also includes step six, fault cause analysis: Based on equipment operating data, health assessment results, and historical maintenance records, a comprehensive knowledge graph specific to the equipment is constructed. Using the fault trigger point as the starting node, a depth-first search or breadth-first search algorithm is employed in the knowledge graph to construct the fault propagation path. Each node in the fault propagation path is assigned a weight, calculated using the following formula: ; in, For the first Node weight, This represents the degree of association between a node and its predecessor node. The severity of the fault, and These are empirical weighting coefficients; Cumulative calculation of the total weight of the fault propagation path ,according to The fault propagation path with the highest weight is sorted to determine the main fault cause node.

[0016] The present invention provides a multimodal data analysis and early warning method for hydropower equipment, which has the following beneficial effects: 1. Through multi-source data acquisition, spatiotemporal alignment and multimodal data fusion technologies, accurate integration of equipment video, physical coordinate data, operating status data and historical data has been achieved.

[0017] This study utilizes interpolation algorithms and a pinhole camera model to correct for both time and space, overcoming the challenge of temporal and spatial matching between low video sampling rates and high-frequency physical coordinate data. Simultaneously, low-level and temporal features of video and physical coordinate data are extracted using convolutional neural networks and long short-term memory networks, and then weighted and fused through an attention mechanism to obtain a unified and high-dimensional data feature representation. This multi-dimensional data fusion method significantly improves data accuracy, laying a solid foundation for subsequent fault diagnosis and equipment health assessment, and is significantly superior to existing single-data source analysis methods.

[0018] 2. The hydropower equipment operation status model enables quantitative assessment and dynamic monitoring of equipment health status. The system first normalizes key indicators such as temperature, vibration, pressure, and energy consumption, and then assigns weights based on historical data and expert experience. A weighted summation is used to calculate the equipment health status score. Simultaneously, using cumulative equipment operating time and fault maintenance records, an exponential decay model is employed to obtain a health decay factor. This factor is then used to dynamically adjust the initial monitoring threshold, ensuring timely and accurate early warning information. This dynamic monitoring method, based on a combination of real-time data and historical trends, effectively reduces false alarm rates and improves the accuracy of fault prediction and equipment operational safety.

[0019] 3. By utilizing cross-system root cause reasoning technology based on a device-specific full-domain knowledge graph, the system achieves deep fusion of multi-source information and automatic construction of fault propagation paths. The system first acquires equipment structure, environmental data, maintenance records, and operational status data. Through semantic annotation and relation extraction, it constructs a full-domain knowledge graph. Then, starting with the fault trigger point, it uses depth-first or breadth-first search algorithms to progressively expand the fault propagation path along relationships such as "cause" and "impact," assigning weights to each node to accurately identify the primary causes of the fault. Attached Figure Description

[0020] The present invention will be further described below with reference to the accompanying drawings and embodiments: Figure 1 This is a flowchart of the present invention; Figure 2 Flowchart for key feature extraction; Figure 3 This is a flowchart for fault prediction. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the invention.

[0022] like Figure 1 As shown, a multimodal data analysis and early warning method for hydropower equipment includes the following steps: Step 1: Multi-source data acquisition and preprocessing: Acquire video monitoring data, equipment operation status data, physical coordinate data, and historical operation data of the hydropower equipment, and perform data preprocessing to generate the first hydropower equipment dataset.

[0023] Data preprocessing includes spatiotemporal alignment, which comprises temporal alignment and spatial alignment, and the operations are as follows: 1) Time alignment For each data sampling time Find two adjacent video frames in the video data, whose timestamps are respectively and ,satisfy: ; in, Earlier The latest video frame time, Later Latest video frame time; according to and The video data at each moment is used to calculate the value using an interpolation algorithm. The video data value corresponding to the given moment.

[0024] Because the video frame sampling rate is low, directly taking or The data can cause timing discrepancies. Therefore, an interpolation algorithm is used to calculate the... The video data value corresponding to the given time. Assume the vibration amplitude indices extracted from the video frames are as follows: and Then in time Corresponding estimated vibration amplitude It can be calculated using the following formula: .

[0025] 2) Spatial alignment Establish a unified spatial coordinate system: Construct a global three-dimensional coordinate system for the hydroelectric equipment based on its structural information and installation calibration data. Define the following in the global coordinate system: , and Three axes determine the physical location of each component of the equipment.

[0026] Video data preprocessing and feature extraction: Image enhancement and preprocessing are performed on the acquired video data. Image processing techniques are used to identify and extract key areas or feature points of the device, and the corresponding two-dimensional coordinates are obtained from the image. ,in The x-axis is... The vertical axis is denoted as y.

[0027] Data spatial calibration: The physical location corresponding to the acquired data is determined as three-dimensional coordinates in the global coordinate system. .

[0028] Constructing a 3D-to-2D spatial mapping model: A pinhole camera model is used to establish the mapping relationship between global 3D coordinate points and 2D coordinate points in the video image. The basic mapping formula is as follows: ; in, This is a scaling factor used to normalize mapping relationships; Let be the homogeneous coordinate vector of the target point in the image; The camera intrinsic parameter matrix is ​​in the form of , With the camera in a horizontal position, For the camera to be in the vertical direction, The x-coordinate of the optical center in the image, The ordinate of the optical center in the image; The extrinsic parameter matrix of the camera is a 3×3 rotation matrix. and a 3×1 translation vector Composition, used to describe the rotation and translation relationship of the camera relative to the global coordinate system; The coordinates of the target point are homogeneous in the global coordinate system.

[0029] Spatial alignment and data fusion: Utilizing the mapping relationships in the constructed 3D to 2D spatial mapping model, the position of data in the global coordinate system is determined. Transform to the corresponding position in the video image coordinate system .

[0030] Step 2, Key Feature Extraction: such as Figure 2 As shown, feature extraction and feature fusion are performed on the first hydropower equipment dataset generated in step one. Feature extraction includes video feature extraction from video data and temporal feature extraction from data sequences. Then, the video features and temporal features are fused to obtain high-level fused features.

[0031] Using convolutional neural networks to process input video data Low-level feature extraction is performed to obtain low-level video features. The calculation formula is: ; in, For video data, These are the low-level video features extracted.

[0032] Temporal features are extracted from the data sequence S using a Long Short-Term Memory (LSTM) network. The calculation formula is as follows: ; in, The original data sequence, The extracted temporal features are used to obtain the features.

[0033] Using attention mechanism and Perform feature fusion to generate unified high-level fused features. .

[0034] The calculation formula is: ; in, This is the attention mechanism process used to dynamically adjust the importance weights of each feature.

[0035] Step 3: Anomaly identification and correction: Anomalies are identified in the fusion features generated in Step 2 to obtain anomaly data points. The anomaly data points are then corrected to obtain high-quality fusion features after correction, in order to ensure data accuracy and consistency.

[0036] Obtaining high-level fusion features mean Standard deviation Maximum value and minimum value ; through the maximum value and minimum value To address the characteristics of high-level integration Perform normalization processing; for any data point i High-level fusion features When the conditions are met If so, the data point is determined to be abnormal data. ,in This is a preset constant.

[0037] For the identified abnormal data A linear regression model was established by combining historical normal data, and the time variable was analyzed. The regression formula for fitting the data is as follows: ; in, and These are the regression coefficients obtained by fitting historical data. These are regression predicted values.

[0038] Intelligent compensation is applied to outlier data based on regression prediction results, using the following compensation formula: ; in, This is the compensation coefficient, used to control the correction amplitude; These are the corrected data values.

[0039] By constructing a multimodal data fusion model, high-level fused features are obtained through feature extraction and feature fusion of multimodal data. ; and through data quality assessment and anomaly detection algorithms, high-level fusion features were evaluated. Corrections are performed to ensure the accuracy and consistency of the high-level fusion feature data; the extracted key indicators are selected for corresponding special analysis scenarios, including equipment temperature, start-up and shutdown analysis, water supply and drainage system, oil level, speed regulating oil system, gas system, main transformer oil chromatography analysis, and stability analysis.

[0040] Step 4, Specific Analysis: such as Figure 3 As shown, based on the corrected high-quality fusion features, key indicator data in equipment operation are extracted, and the key indicator data correspond to different equipment-specific analysis scenarios.

[0041] The key performance indicators extracted during equipment operation include at least temperature data, vibration data, pressure data, and energy consumption data. The extracted key indicators correspond to the specific operating status of the equipment. Temperature data corresponds to equipment temperature analysis; vibration data corresponds to start-up and shutdown analysis, gas system and stability analysis; pressure data corresponds to speed regulating oil system and main transformer oil chromatography analysis; and energy consumption data corresponds to water supply and drainage system and oil level analysis.

[0042] Based on the key indicator data, the equipment health status score is calculated, and an equipment health decay factor is introduced to dynamically correct the equipment-specific operation status monitoring threshold.

[0043] The equipment health status score is calculated based on the aforementioned key indicator data, including: The data for each key indicator were normalized to obtain normalized values. , , and ; Preset weights are determined based on historical data and expert experience. , , and .

[0044] Based on the actual situation of hydropower equipment operation and management, the process of determining the pre-set weights can be implemented according to the following steps: In the historical data analysis stage, collect at least 3-5 years of historical equipment operation data and fault records. Statistically analyze the frequency and severity of abnormal performance of four indicators—temperature, vibration, pressure, and energy consumption—in historical fault events. Calculate the correlation strength between each indicator and the actual occurrence of faults using methods such as Pearson correlation coefficient or mutual information, forming a quantitative ranking of indicator importance. In the expert experience acquisition stage, organize 5-10 senior experts in hydropower equipment operation, maintenance, and design to conduct multiple rounds of questionnaire surveys using the Delphi method. Experts will score and evaluate the importance of the four indicators based on their practical experience and explain the scoring basis. Consistency of expert opinions will be achieved through multiple iterations. In the weight comprehensive determination stage, construct a hierarchical structure of indicator importance using the analytic hierarchy process (AHP). Use historical data statistics as objective weights and expert evaluation results as subjective weights. Determine the final weight allocation through a weighted average or linear combination method. For example, final weight = 0.6 × objective weight + 0.4 × subjective weight. In the weight verification and optimization phase, the determined weights are used to perform backtesting on historical data to verify the fault prediction accuracy and false alarm rate. If the effect is not good, the combination ratio of objective weights and subjective weights is adjusted, and the verification is repeated until a satisfactory prediction effect is achieved. Finally, the preset weight values ​​of the four indicators of temperature, vibration, pressure and energy consumption are determined.

[0045] The weighted summation method is used to calculate the equipment health status score, and the formula is as follows: ; in, Rate the health status of the equipment. This is the equipment health degradation factor.

[0046] The steps for obtaining the equipment health degradation factor include: Collecting the cumulative running time of the equipment t In addition to historical fault records and maintenance records; Fit the health degradation curve of the equipment and determine the equipment degradation constant. ; The health degradation factor of equipment is obtained using the exponential decay model, and its calculation formula is as follows: ; in, The cumulative operating time of the equipment. This is the decay constant obtained by fitting historical data.

[0047] The value ranges from 0 to 1, with a larger value indicating a higher degree of degradation in the device's health.

[0048] A device health degradation factor is introduced to dynamically correct the threshold for monitoring the specific operating status of the equipment, including: The initial monitoring threshold was obtained based on long-term equipment operation data. ; Introducing equipment health degradation factor The monitoring threshold is then corrected using the following formula: ; in, This is the dynamically adjusted monitoring threshold.

[0049] Step 5, Fault Prediction: Based on Steps 1 to 4, calculate the health status score at each historical moment, establish the mapping relationship between operating conditions and health status scores to form a benchmark database.

[0050] Specifically, the process begins by collecting historical multi-source data, including video surveillance data, equipment operating status data, physical coordinate data, and corresponding historical operating data under different operating conditions. This historical data is preprocessed using the same workflow as real-time processing, involving steps such as spatiotemporal alignment, convolutional neural network feature extraction, long short-term memory network temporal analysis, and attention mechanism fusion to obtain a high-level fused feature sequence for historical time periods. From these fused features, four key indicators—temperature, vibration, pressure, and energy consumption—are extracted, and the actual health status score for each historical moment is calculated, forming a historical sequence of health status scores with timestamps. Simultaneously, the model records the equipment operating environment parameters corresponding to each historical moment, such as load level, water level, ambient temperature, and operating mode, establishing a benchmark database mapping the operating conditions to the health status scores.

[0051] When performing fault prediction, a health status score is calculated based on real-time acquired equipment data. The system acquires current equipment operating environment parameters and condition information, including key operating condition data such as current load level, water level, environmental conditions, and operating mode. This current operating condition information is then matched against a benchmark database to identify the health status score corresponding to the historically most similar operating scenario, which is used as the predicted health status score. , and The system compares the results, and when the absolute error exceeds the preset error threshold, it is considered abnormal. When an abnormality is detected, the degree of abnormality is calculated and an early warning is triggered.

[0052] Fault prediction specifically includes the following steps: Get the current health status score in real time and the predicted value of health status score Compare and calculate the absolute error ; when Greater than the preset error threshold When this occurs, it is determined to be abnormal, and the degree of abnormality is calculated. The calculation formula is as follows: ; Based on the degree of abnormality The system will automatically trigger an early warning signal based on the current operating status.

[0053] The primary function of anomaly severity calculation is to quantify the degree to which equipment status deviates from normal operating range, providing a basis for graded response in the early warning system. It determines whether an alert needs to be triggered and the level of urgency. Minor anomalies may only require monitoring, moderate anomalies trigger a warning at the attention level, while high anomalies trigger an emergency warning and maintenance recommendations. Furthermore, anomaly severity data provides training samples for the self-learning mechanism. By recording actual operating data and early warning judgment data during the warning period, a new dataset is created for dynamic model optimization and false alarm rate reduction.

[0054] At the same time, the actual operational data during the early warning period and the early warning judgment data will be recorded to form a new dataset. .

[0055] An incremental learning algorithm is used to dynamically update the hydropower equipment operation status model through self-learning, which improves the accuracy of the fault prediction model and reduces the false alarm rate in the next cycle.

[0056] Step 6, Fault Cause Analysis: Construct a comprehensive knowledge graph based on equipment specific to the equipment, based on equipment operation data, health assessment results, and historical maintenance records.

[0057] Acquire specific operational data, health assessment results, and historical specific maintenance records for hydropower equipment; simultaneously acquire specific structural information, environmental data, and specific maintenance records for the equipment; preprocess and semantically annotate the data to extract key entity information and attributes for specific purposes, providing the original data foundation for constructing a specific knowledge graph.

[0058] The steps for constructing a device-specific full-domain knowledge graph include: Define the entity types involved in the knowledge graph, including but not limited to equipment-specific components, fault types, maintenance events, environmental factors, and operating status; Define the types of relationships between entities; The relation extraction algorithm is used to perform entity recognition and relation extraction on the preprocessed data, generating nodes and edges; The nodes and their interrelationships are organized into a graph structure to form a device-specific full-domain knowledge graph, in which each node is accompanied by attribute information.

[0059] Starting with the fault trigger point as the initial node, a depth-first search or breadth-first search algorithm is used in the knowledge graph to progressively expand along the "cause" or "impact" relationship, constructing a fault propagation path. Each node in the fault propagation path is assigned a weight, calculated using the following formula: ; in, For the first Node weight, This represents the degree of association between a node and its predecessor node. The severity of the fault, and These are empirical weighting coefficients; Cumulative calculation of the total weight of the fault propagation path ,according to The fault propagation path with the highest weight is sorted to determine the main fault cause node.

[0060] After determining the fault propagation path with the highest weight, the identification of the main fault cause node is achieved by analyzing the weight distribution of each node on that path. In the fault propagation path with the highest weight, the node with the largest weight value is identified as the main fault cause. Simultaneously, the node's position in the fault propagation chain is considered; high-weight nodes located at the beginning of the propagation path or at key propagation nodes are usually more likely to be the root cause. Combining the severity and correlation of the faults of the nodes, the core node with the greatest impact on the entire fault event is comprehensively determined, which is the main fault cause node.

[0061] In practical use, it can also generate operation and maintenance suggestion reports. Based on equipment operation data, health assessment results, and historical maintenance records, it constructs a device-specific full-domain knowledge graph. Using a knowledge graph-driven cross-system root cause reasoning method, it automatically constructs fault propagation paths, identifies major fault causes, and automatically generates operation and maintenance suggestions, including optimal maintenance timing, possible fault causes, and specific repair solutions. Specifically, based on the aforementioned major fault cause nodes, combined with the historical maintenance records of each node in the device-specific full-domain knowledge graph, equipment health status scores, and based on equipment operation data and health degradation trends, it generates threshold correction suggestions for newly detected fault modes. Finally, it automatically generates operation and maintenance suggestions, including but not limited to optimal maintenance timing, possible fault causes, and repair solutions, based on the above analysis results.

[0062] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for multimodal data analysis and early warning of hydropower equipment, characterized in that, Includes the following steps: Step 1: Multi-source data acquisition and preprocessing: Acquire video surveillance data, equipment operation status data, physical coordinate data, and historical operation data of the hydropower equipment, and perform data preprocessing to generate the first hydropower equipment dataset; Step 2, Key Feature Extraction: Feature extraction and feature fusion are performed on the first hydropower equipment dataset generated in Step 1. Feature extraction includes video feature extraction from video data and temporal feature extraction from data sequences. Then, the video features and temporal features are fused to obtain high-level fused features. Step 3, Abnormal Data Identification and Correction: Abnormal data points are identified in the fusion features generated in Step 2. The abnormal data points are then corrected to obtain high-quality fusion features after correction. Step 4: Specific Analysis: Based on the corrected high-quality fusion features, extract key indicator data during equipment operation. These key indicator data correspond to different specific equipment analysis scenarios. Based on the key indicator data, the equipment health status score is calculated, and the equipment health decay factor is introduced to dynamically correct the equipment-specific operation status monitoring threshold. Step 5: Fault Prediction: Based on Steps 1-4, calculate the health status score for each historical moment, and establish a mapping relationship between operating conditions and health status scores to form a benchmark database; when performing fault prediction, calculate the current health status score based on real-time acquired equipment data. The current equipment operating environment parameters and working condition information are matched with the benchmark database to find the health status score corresponding to the historical operating scenario most similar to the current working condition, which is then used as the predicted health status score. , and The comparison is performed, and an anomaly is determined when the absolute error exceeds a preset error threshold.

2. The method for multimodal data analysis and early warning of hydropower equipment according to claim 1, characterized in that, In step one, data preprocessing includes spatiotemporal alignment, which includes time alignment and spatial alignment, and the operation is as follows: 1) Time alignment For each data sampling time Find two adjacent video frames in the video data, whose timestamps are respectively and ,satisfy: ； in, Earlier The latest video frame time, Later Latest video frame time; according to and The video data at each moment is used to calculate the value using an interpolation algorithm. The video data value corresponding to the given moment; 2) Spatial alignment Establish a unified spatial coordinate system: Construct a global three-dimensional coordinate system for the equipment based on its structural information and installation calibration data. Define the following in the global coordinate system: , and Three axes determine the physical location of each component of the equipment; Video data preprocessing and feature extraction: Image enhancement and preprocessing are performed on the acquired video data to identify and extract key areas or feature points of the device, and obtain the corresponding two-dimensional coordinates in the image. ,in The x-axis is... The vertical axis is used as the coordinate. Data spatial calibration: The physical location corresponding to the acquired data is determined as three-dimensional coordinates in the global coordinate system. ; Constructing a 3D-to-2D spatial mapping model: A pinhole camera model is used to establish the mapping relationship between global 3D coordinate points and 2D coordinate points in the video image. The basic mapping formula is as follows: ； in, This is a scaling factor used to normalize mapping relationships; Let be the homogeneous coordinate vector of the target point in the image; The camera intrinsic parameter matrix is ​​in the form of , With the camera in a horizontal position, For the camera to be in the vertical direction, The x-coordinate of the optical center in the image, The ordinate of the optical center in the image; The extrinsic parameter matrix of the camera is a 3×3 rotation matrix. and a 3×1 translation vector Composition, used to describe the rotation and translation relationship of the camera relative to the global coordinate system; The homogeneous coordinates of the target point in the global coordinate system; Spatial alignment and data fusion: Utilizing the mapping relationships in the constructed 3D to 2D spatial mapping model, the position of data in the global coordinate system is determined. Transform to the corresponding position in the video image coordinate system .

3. The method for multimodal data analysis and early warning of hydropower equipment according to claim 1, characterized in that, In step two, a convolutional neural network is used to process the input video data. Low-level feature extraction is performed to obtain low-level video features. Temporal features are extracted from the data sequence S using a Long Short-Term Memory (LSTM) network. Using attention mechanisms to and Perform feature fusion to generate unified high-level fused features. .

4. The method for multimodal data analysis and early warning of hydropower equipment according to claim 1, characterized in that, In step three, high-level fusion features are obtained. mean Standard deviation Maximum value and minimum value ; through the maximum value and minimum value To address the characteristics of high-level integration Perform normalization processing; for any data point i High-level fusion features When the conditions are met If so, the data point is determined to be abnormal data. ,in This is a preset constant.

5. The method for multimodal data analysis and early warning of hydropower equipment according to claim 4, characterized in that, In step three, the operation of correcting abnormal data points is as follows: For the identified abnormal data A linear regression model was established by combining historical normal data, and the time variable was analyzed. The regression formula for fitting the data is as follows: ； in, and These are the regression coefficients obtained by fitting historical data. These are regression predictions; Intelligent compensation is applied to outlier data based on regression prediction results, using the following compensation formula: ； in, This is the compensation coefficient, used to control the correction amplitude; These are the corrected data values.

6. The method for multimodal data analysis and early warning of hydropower equipment according to claim 1, characterized in that, In step four, the key indicator data extracted during equipment operation includes at least temperature data, vibration data, pressure data, and energy consumption data. The extracted key indicators correspond to the specific operating status of the equipment. Temperature data corresponds to equipment temperature analysis; vibration data corresponds to start-up and shutdown analysis, gas system and stability analysis; pressure data corresponds to speed regulating oil system and main transformer oil chromatography analysis; and energy consumption data corresponds to water collection and drainage system and oil level analysis.

7. The method for multimodal data analysis and early warning of hydropower equipment according to claim 6, characterized in that, In step four, the equipment health status score is calculated based on the key indicator data, including: The data for each key indicator were normalized to obtain normalized values. , , and ; Preset weights are determined based on historical data and expert experience. , , and ; The weighted summation method is used to calculate the equipment health status score, and the formula is as follows: ； in, Rate the health status of the equipment. Equipment health degradation factor; The steps for obtaining the equipment health degradation factor include: Collecting the cumulative running time of the equipment t In addition to historical fault records and maintenance records; Fit the health degradation curve of the equipment and determine the equipment degradation constant. ; The health degradation factor of equipment is obtained using the exponential decay model, and its calculation formula is as follows: ； in, The cumulative operating time of the equipment. This is the decay constant obtained by fitting historical data.

8. The method for multimodal data analysis and early warning of hydropower equipment according to claim 7, characterized in that, The introduction of an equipment health degradation factor to dynamically correct the equipment-specific operating status monitoring threshold includes: The initial monitoring threshold was obtained based on long-term equipment operation data. ; Introducing equipment health degradation factor The monitoring threshold is then corrected using the following formula: ； in, This is the dynamically adjusted monitoring threshold.

9. The method for multimodal data analysis and early warning of hydropower equipment according to claim 1, characterized in that, Step five, fault prediction, specifically includes the following steps: Get the current health status score in real time and the predicted value of health status score Compare and calculate the absolute error ; when Greater than the preset error threshold When this occurs, it is determined to be abnormal, and the degree of abnormality is calculated. The calculation formula is as follows: ； Based on the degree of abnormality The system will automatically trigger an early warning signal based on the current operating status.

10. The method for multimodal data analysis and early warning of hydropower equipment according to claim 1, characterized in that, The process also includes step six, fault cause analysis: Based on equipment operating data, health assessment results, and historical maintenance records, a comprehensive knowledge graph for the specific equipment is constructed. Using the fault trigger point as the starting node, a depth-first search or breadth-first search algorithm is employed within the knowledge graph to construct the fault propagation path. Each node in the fault propagation path is assigned a weight, calculated using the following formula: ； in, For the first Node weight, This represents the degree of association between a node and its predecessor node. The severity of the fault, and These are empirical weighting coefficients; Cumulative calculation of the total weight of the fault propagation path ,according to The fault propagation path with the highest weight is sorted to determine the main fault cause node.

Images