A hydrological data governance method and system based on machine learning

Abstract

The application discloses a kind of based on machine learning's hydrological data governance method and system, S1. Generation has unified space-time resolution hydrological monitoring data set;S2. Construction fuzzy entropy matrix of hydrological monitoring data;S3. Generation is represented in directed acyclic graph form causal model of hydrological monitoring data;S4. Real-time identification of abnormal hydrological monitoring data by dynamically adjusting anomaly detection threshold;S5. Generation is traced to abnormal candidate path, forms abnormal hydrological monitoring data causal chain;S6. Fuzzy entropy optimization processing is carried out to abnormal candidate path, according to the fuzzy entropy value of each causal chain in hydrological variable and its influence weight in causal chain, key abnormal causal path is screened out, and then the positioning of the root cause of abnormal hydrological monitoring data is positioned.The application provides a more efficient, intelligent technical solution for water resources management, disaster prevention and warning and environmental monitoring.

Description

Technical Field

[0001] This invention relates to the field of hydrological technology, and in particular to a method and system for hydrological data management based on machine learning. Background Technology

[0002] With the continuous development of hydrological monitoring technology, various hydrological sensors have been widely used in water resource management, disaster prevention and mitigation, and ecological environment monitoring to achieve real-time acquisition of key hydrological variables. However, due to the complexity of the natural environment, the limitations of sensor accuracy, and interference during data transmission, hydrological monitoring data often contains noise, missing values, or abnormal data. Abnormal data may be caused by equipment failure, data acquisition errors, or extreme weather events, posing a great challenge to the accuracy and reliability of hydrological monitoring data.

[0003] Currently, methods for detecting hydrological data anomalies mainly rely on time series analysis, statistical methods, and machine learning techniques. For example, threshold-based methods can quickly detect abnormal data that exceeds the normal range, but these methods depend on fixed thresholds set empirically and are difficult to adapt to dynamically changing hydrological environments. Time series prediction methods use historical data to build trend models, such as autoregressive moving average models or long short-term memory networks, but their prediction accuracy tends to decrease when the data anomalies are complex or significantly affected by the external environment. Clustering and classification-based anomaly detection methods can identify anomalous data points, but due to a lack of in-depth understanding of the causal relationships between hydrological variables, they are prone to misjudging sudden events as anomalies or failing to accurately distinguish the true source of anomalous data.

[0004] Furthermore, in terms of tracing the source of hydrological data anomalies, existing technologies mainly rely on statistical correlation analysis or simple rule inference, which makes it difficult to effectively identify the true causal relationship between variables. At the same time, traditional anomaly tracing methods lack intelligent processing means for large-scale hydrological data, making it difficult to automatically track and trace anomalies when faced with complex hydrological systems.

[0005] In summary, existing methods for detecting and tracing anomalies in hydrological data still have many shortcomings in terms of accuracy, real-time performance, and intelligence. On the one hand, existing methods often rely on fixed thresholds or statistical correlations, making it difficult to adapt to the identification of dynamic hydrological anomalies in complex environments, resulting in a high false positive rate. On the other hand, the lack of causal inference mechanisms makes it impossible to accurately locate the root cause of anomalies, leading to a lack of scientific basis in the anomaly tracing process. Furthermore, in the context of massive hydrological data, traditional methods struggle to achieve efficient automated anomaly detection and tracing, resulting in slow response times and affecting the accuracy of water resource management and disaster prevention decisions. Summary of the Invention

[0006] One objective of this invention is to propose a hydrological data management method and system based on machine learning. This invention provides a more efficient and intelligent technical solution for water resource management, disaster prevention and early warning, and environmental monitoring.

[0007] A hydrological data management method based on machine learning according to an embodiment of the present invention includes the following steps:

[0008] S1. Acquire hydrological monitoring data and preprocess it to generate a hydrological monitoring dataset with uniform spatiotemporal resolution;

[0009] S2. Calculate the fuzzy entropy values ​​of each hydrological variable in the hydrological monitoring dataset, form a fuzzy entropy assessment result to measure the uncertainty of each hydrological variable, and construct a fuzzy entropy matrix of the hydrological monitoring data;

[0010] S3. Based on the hydrological monitoring dataset and its fuzzy entropy matrix, the Granger causal analysis method is used to construct a causal relationship model between various hydrological variables in the hydrological monitoring data. The direct and indirect causal relationships between various hydrological variables are determined by causal tests under different time lag conditions, and a causal model of hydrological monitoring data represented in the form of a directed acyclic graph is generated.

[0011] S4. Continuously monitor the hydrological monitoring dataset and use the causal model of hydrological monitoring data combined with the fuzzy entropy optimization strategy to detect anomalies in the real-time hydrological monitoring dataset. Real-time identification of abnormal hydrological monitoring data is achieved by dynamically adjusting the anomaly detection threshold.

[0012] S5. When abnormal hydrological monitoring data is detected, reverse causal analysis is performed on the abnormal hydrological monitoring data based on the causal model of hydrological monitoring data. Based on the Granger causal analysis results and fuzzy entropy evaluation results, candidate paths for tracing the source of the abnormality are generated, forming a causal chain of abnormal hydrological monitoring data.

[0013] S6. Perform fuzzy entropy optimization on the candidate paths for tracing the source of anomalies. Based on the fuzzy entropy values ​​of hydrological variables in each causal chain and their influence weight in the causal chain, select key causal paths for anomalies, and then locate the root cause of the abnormal hydrological monitoring data.

[0014] Optionally, step S1 includes:

[0015] S11. Hydrological monitoring data is acquired in real time through a hydrological monitoring sensor network. The hydrological monitoring data includes rainfall, flow velocity, water level, evaporation, and temperature. The hydrological monitoring data is initially stored according to timestamp t to form an original hydrological dataset.

[0016] ;

[0017] Where P is rainfall, V is flow velocity, H is water level, E is evaporation, and T is temperature. This represents the value of the hydrological variable at time t. For time sets;

[0018] S12. Perform data cleaning on the original hydrological dataset, including detecting and removing outliers, filling in missing data, and removing sensor noise, to form a preliminarily cleaned hydrological monitoring dataset;

[0019] S13. Standardize the hydrological monitoring dataset after preliminary cleaning to ensure that all hydrological variables have the same numerical scale;

[0020] S14. Reconstruct the standardized hydrological monitoring dataset according to timestamps to align all hydrological variables within the same time step, forming a hydrological monitoring dataset with uniform spatiotemporal resolution. :

[0021] ;

[0022] in, This is the reconstructed time set.

[0023] Optionally, step S2 includes the following steps:

[0024] S21. For each hydrological variable in the hydrological monitoring dataset... At each sampling time Calculate its membership value based on the fuzzy membership function. :

[0025] ;

[0026] in, Representing hydrological variables At sampling time The standardized value, Hydrological variables In the dataset The mean of the middle, Hydrological variables The total number of samples, These are adjustment parameters used to reflect hydrological variables. The degree of sensitivity to deviations from the mean is used to strengthen or suppress the impact of abnormal fluctuations in fuzzy membership calculation;

[0027] S22. Based on membership value For each hydrological variable The fuzzy entropy value is calculated by combining the membership degree of each sampling point with its deviation from the mean. :

[0028] ;

[0029] in, Represents the natural logarithm. As a weighting factor, it is used to adjust hydrological variables. The impact of extreme deviations on the contribution of fuzzy entropy. Hydrological variables The standard deviation reflects the degree of dispersion of the data;

[0030] S23. Calculate the fuzzy entropy values ​​of each hydrological variable. Arranged in index order according to the hydrological variables, the fuzzy entropy matrix F of the hydrological monitoring data is formed:

[0031] ;

[0032] in, , , , and These correspond to the fuzzy entropy values ​​of rainfall P, flow velocity V, water level H, evaporation E, and temperature T, respectively.

[0033] Optionally, step S3 includes the following steps:

[0034] S31. Based on the hydrological monitoring dataset and its fuzzy entropy matrix, for each pair of hydrological variables and , Unweighted and weighted vector autoregressive models were constructed under different time lags L, and Granger causality tests were performed.

[0035] ;

[0036] ;

[0037] in, Hydrological variables The autoregressive coefficient at lag k, Hydrological variables At lag k, for Influence coefficient, and These are the error terms of the model. The weighting factor is based on the fuzzy entropy matrix;

[0038] S32. Setting the null hypothesis 0 represents a hydrological variable. right Without Granger causality, i.e. relying solely on an unweighted vector autoregressive model, the alternative hypothesis is set. Hydrological variables right It exhibits Granger causality, meaning it employs a weighted vector autoregressive model at a pre-defined significance level. Next, perform an F-statistic test. If the null hypothesis is rejected... : Then determine the hydrological variables. Granger causally affects hydrological variables Record the direct causal relationship and the corresponding weighting factor. ;

[0039] S33. Based on the test results of step S32, construct a causal model of hydrological monitoring data represented by a directed acyclic graph G = (V, E), where the vertex set corresponds to rainfall P, flow velocity V, water level H, evaporation E, and temperature T, respectively, and the edge set E contains data from hydrological variables. To hydrological variables A directed edge if and only if determined in step S32 Granger causal influence And the weight of that edge is determined by the corresponding weighting factor. The final causal model of the hydrological monitoring data was determined and generated.

[0040] Optionally, step S4 includes the following steps:

[0041] S41. Continuously monitor the real-time hydrological monitoring dataset during the monitoring period, and record the data at each time point. Calculate hydrological variables Deviation from historical distribution of hydrological variables ;

[0042] S42. Using a causal model G based on hydrological monitoring data for each hydrological variable... The set of causal influence variables within the lag time window L Perform causal weighted bias calculation:

[0043] ;

[0044] in, Influence in causal models The set of variables, As a causal weighting factor, Hydrological variables right The time step that has an effect As a measure of deviation of hydrological variables after comprehensive causal influence;

[0045] S43. Combine fuzzy entropy optimization strategy for each hydrological variable Calculate the dynamic anomaly detection threshold :

[0046] ;

[0047] in, Hydrological variables The average deviation measure in historical data The fuzzy entropy value of the hydrological variable. As the baseline anomaly detection coefficient, The fuzzy entropy influencing factor;

[0048] S44. At every moment If satisfied Then determine the hydrological variables. In time If an anomaly occurs, record the abnormal hydrological variables, the time of the anomaly, and the causal path.

[0049] Optionally, step S5 includes the following steps:

[0050] S51. After detecting abnormal hydrological monitoring data, construct a causal source search space for the abnormal hydrological monitoring data based on the causal model of the hydrological monitoring data, define the set of abnormal variables, and expand the source variable set according to the hierarchy of the reverse causal relationship in the causal model:

[0051] ;

[0052] in, Represents the initial set of variables for an anomaly. This represents the set of anomalous impact variables traced back to the m-th level, where M is the maximum tracing depth;

[0053] S52. Calculate the causal influence strength of each variable in the anomaly tracing path. And perform a weighted ranking of abnormal causal relationships:

[0054] ;

[0055] in, This is the dynamic anomaly detection threshold. Hydrological variables right The time step that has an effect Reflects variables For abnormal hydrological variables The degree of impact;

[0056] S53. Sort the anomaly tracing paths according to the strength of causal influence and generate a set of anomaly tracing candidate paths. }:

[0057] ;

[0058] in, As a threshold for causal attribution strength, only those with a causal influence strength greater than [a certain threshold] are retained. The path to tracing the source;

[0059] S54. Combining fuzzy entropy optimization strategy to optimize the set of candidate paths for anomaly tracing. Optimize by calculating the comprehensive entropy weight score for each tracing path. ;

[0060] S55. Based on the comprehensive entropy weight score, select the highest score as the final anomaly tracing path and construct a causal chain of abnormal hydrological monitoring data. :

[0061] ;

[0062] Abnormal causal chain Record the layer-by-layer impact path of anomalous variables to form the final causal analysis results of anomalous hydrological monitoring data.

[0063] A machine learning-based hydrological data management system, used to execute a machine learning-based hydrological data management method, includes the following modules:

[0064] The data acquisition module is used to acquire hydrological monitoring data in real time through a hydrological monitoring sensor network. The hydrological monitoring data includes rainfall, flow velocity, water level, evaporation and temperature. The acquired data is stored according to timestamps to form a hydrological monitoring dataset.

[0065] The data preprocessing module is used to perform data cleaning, noise reduction, missing value imputation, outlier removal and data standardization on the hydrological monitoring dataset to form a hydrological monitoring dataset with uniform spatiotemporal resolution.

[0066] The fuzzy entropy calculation module is used to calculate the fuzzy entropy value of each hydrological variable in the hydrological monitoring dataset and construct the fuzzy entropy matrix of the hydrological monitoring data to measure the uncertainty of the hydrological variables.

[0067] The causal relationship modeling module is used to construct a causal relationship model between various hydrological variables in hydrological monitoring data based on hydrological monitoring datasets and their fuzzy entropy matrices, using Granger causal analysis method. It uses unweighted vector autoregression model and weighted vector autoregression model to perform causality test, determine the direct and indirect causal relationships between hydrological variables, and generate a causal model of hydrological monitoring data represented in the form of a directed acyclic graph.

[0068] The real-time anomaly detection module is used to continuously monitor the hydrological monitoring dataset and detect anomalies in the real-time hydrological monitoring data based on the causal model of the hydrological monitoring data combined with the fuzzy entropy optimization strategy. It calculates the causal weighted deviation, identifies abnormal hydrological monitoring data according to the dynamically adjusted anomaly detection threshold, and records the abnormal variables and the time of occurrence.

[0069] The anomaly tracing analysis module is used to perform reverse causal analysis based on the causal model of hydrological monitoring data after detecting abnormal hydrological monitoring data. It generates candidate paths for anomaly tracing based on Granger causal analysis results and fuzzy entropy evaluation results, and calculates the comprehensive entropy weight score by combining fuzzy entropy optimization strategy to optimize the anomaly tracing path. Finally, it constructs the causal chain of abnormal hydrological monitoring data and forms a complete anomaly analysis result.

[0070] The beneficial effects of this invention are:

[0071] This invention constructs a causal model of hydrological monitoring data based on Granger causality analysis, and adjusts the weights of causal relationships by combining fuzzy entropy optimization. This enables more accurate identification of the true causal relationships between hydrological variables, avoids misjudgments caused by spurious correlations between dependent variables, and improves the reliability of anomaly tracing by introducing causal inference technology, so that the system can automatically trace back the most critical anomaly triggering factors when anomalies occur and quantify their impact.

[0072] In the process of anomaly detection in real-time hydrological monitoring data, this invention combines causal weighted deviation calculation and fuzzy entropy optimization strategy. By dynamically adjusting the anomaly detection threshold, the anomaly identification becomes more accurate and adaptive. This invention calculates the causal weighted deviation of hydrological variables and combines it with fuzzy entropy to dynamically optimize the threshold, enabling the system to adaptively adjust the anomaly judgment criteria based on real-time data characteristics, ensuring stable operation under different hydrological environments.

[0073] This invention constructs a set of candidate paths for anomaly tracing during the anomaly tracing process, and optimizes and filters the anomaly tracing paths by calculating the comprehensive entropy weight score through a fuzzy entropy optimization strategy. This enables the final anomaly causal chain to more accurately reflect the real anomaly transmission process. By calculating the causal influence intensity and combining fuzzy entropy optimization to remove variables with low correlation, the conciseness and effectiveness of the tracing path are ensured, thereby improving the efficiency and interpretability of anomaly tracing. Attached Figure Description

[0074] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0075] Figure 1This is a flowchart of a hydrological data management method and system based on machine learning proposed in this invention. Detailed Implementation

[0076] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0077] refer to Figure 1 A machine learning-based hydrological data management method includes the following steps:

[0078] S1. Acquire hydrological monitoring data and preprocess it to generate a hydrological monitoring dataset with uniform spatiotemporal resolution;

[0079] S2. Calculate the fuzzy entropy values ​​of each hydrological variable in the hydrological monitoring dataset, form a fuzzy entropy assessment result to measure the uncertainty of each hydrological variable, and construct a fuzzy entropy matrix of the hydrological monitoring data;

[0080] S3. Based on the hydrological monitoring dataset and its fuzzy entropy matrix, the Granger causal analysis method is used to construct a causal relationship model between various hydrological variables in the hydrological monitoring data. The direct and indirect causal relationships between various hydrological variables are determined by causal tests under different time lag conditions, and a causal model of hydrological monitoring data represented in the form of a directed acyclic graph is generated.

[0081] S4. Continuously monitor the hydrological monitoring dataset and use the causal model of hydrological monitoring data combined with the fuzzy entropy optimization strategy to detect anomalies in the real-time hydrological monitoring dataset. Real-time identification of abnormal hydrological monitoring data is achieved by dynamically adjusting the anomaly detection threshold.

[0082] S5. When abnormal hydrological monitoring data is detected, reverse causal analysis is performed on the abnormal hydrological monitoring data based on the causal model of hydrological monitoring data. Based on the Granger causal analysis results and fuzzy entropy evaluation results, candidate paths for tracing the source of the abnormality are generated, forming a causal chain of abnormal hydrological monitoring data.

[0083] S6. Perform fuzzy entropy optimization on the candidate paths for tracing the source of anomalies. Based on the fuzzy entropy values ​​of hydrological variables in each causal chain and their influence weight in the causal chain, select key causal paths for anomalies, and then locate the root cause of the abnormal hydrological monitoring data.

[0084] In this embodiment, step S1 includes:

[0085] S11. Hydrological monitoring data is acquired in real time through a hydrological monitoring sensor network. The hydrological monitoring data includes rainfall, flow velocity, water level, evaporation, and temperature. The hydrological monitoring data is initially stored according to the timestamp t to form the raw hydrological dataset.

[0086] ;

[0087] Where P is rainfall, V is flow velocity, H is water level, E is evaporation, and T is temperature. This represents the value of the hydrological variable at time t. For time sets;

[0088] S12. Perform data cleaning on the original hydrological dataset, including detecting and removing outliers, filling in missing data, and removing sensor noise, to form a preliminarily cleaned hydrological monitoring dataset;

[0089] S13. Standardize the hydrological monitoring dataset after preliminary cleaning to ensure that all hydrological variables have the same numerical scale;

[0090] S14. Reconstruct the standardized hydrological monitoring dataset according to timestamps to align all hydrological variables within the same time step, forming a hydrological monitoring dataset with uniform spatiotemporal resolution. :

[0091] ;

[0092] in, This is the reconstructed time set.

[0093] In this embodiment, step S2 includes the following steps:

[0094] S21. For each hydrological variable in the hydrological monitoring dataset... At each sampling time Calculate its membership value based on the fuzzy membership function. :

[0095] ;

[0096] in, Representing hydrological variables At sampling time The standardized value, Hydrological variables In the dataset The mean of the middle, Hydrological variables The total number of samples, These are adjustment parameters used to reflect hydrological variables. The degree of sensitivity to deviations from the mean is used to strengthen or suppress the impact of abnormal fluctuations in fuzzy membership calculation;

[0097] S22. Based on membership value For each hydrological variable The fuzzy entropy value is calculated by combining the membership degree of each sampling point with its deviation from the mean. :

[0098] ;

[0099] in, Represents the natural logarithm. As a weighting factor, it is used to adjust hydrological variables. The impact of extreme deviations on the contribution of fuzzy entropy. Hydrological variables The standard deviation reflects the degree of dispersion of the data;

[0100] S23. Calculate the fuzzy entropy values ​​of each hydrological variable. Arranged in index order according to the hydrological variables, the fuzzy entropy matrix F of the hydrological monitoring data is formed:

[0101] ;

[0102] in, , , , and These correspond to the fuzzy entropy values ​​of rainfall P, flow velocity V, water level H, evaporation E, and temperature T, respectively.

[0103] In this embodiment, step S3 includes the following steps:

[0104] S31. Based on the hydrological monitoring dataset and its fuzzy entropy matrix, for each pair of hydrological variables and , Unweighted and weighted vector autoregressive models were constructed under different time lags L, and Granger causality tests were performed.

[0105] ;

[0106] ;

[0107] in, Hydrological variables The autoregressive coefficient at lag k, Hydrological variables At lag k, for Influence coefficient, and These are the error terms of the model. The weighting factor is based on the fuzzy entropy matrix;

[0108] S32. Setting the null hypothesis 0 represents a hydrological variable. right Without Granger causality, i.e. relying solely on an unweighted vector autoregressive model, the alternative hypothesis is set. Hydrological variables right It exhibits Granger causality, meaning it employs a weighted vector autoregressive model at a pre-defined significance level. Next, perform an F-statistic test. If the null hypothesis is rejected... : Then determine the hydrological variables. Granger causally affects hydrological variables Record the direct causal relationship and the corresponding weighting factor. ;

[0109] S33. Based on the test results of step S32, construct a causal model of hydrological monitoring data represented by a directed acyclic graph G = (V, E), where the vertex set corresponds to rainfall P, flow velocity V, water level H, evaporation E, and temperature T, respectively, and the edge set E contains data from hydrological variables. To hydrological variables A directed edge if and only if determined in step S32 Granger causal influence And the weight of that edge is determined by the corresponding weighting factor. The final causal model of the hydrological monitoring data was determined and generated.

[0110] In this embodiment, step S4 includes the following steps:

[0111] S41. Continuously monitor the real-time hydrological monitoring dataset during the monitoring period, and record the data at each time point. Calculate hydrological variables Deviation from historical distribution of hydrological variables ;

[0112] S42. Using a causal model G based on hydrological monitoring data for each hydrological variable... The set of causal influence variables within the lag time window L Perform causal weighted bias calculation:

[0113] ;

[0114] in, Influence in causal models The set of variables, As a causal weighting factor, Hydrological variables right The time step that has an effect As a measure of deviation of hydrological variables after comprehensive causal influence;

[0115] S43. Combine fuzzy entropy optimization strategy for each hydrological variable Calculate the dynamic anomaly detection threshold :

[0116] ;

[0117] in, Hydrological variables The average deviation measure in historical data The fuzzy entropy value of the hydrological variable. As the baseline anomaly detection coefficient, The fuzzy entropy influencing factor;

[0118] S44. At every moment If satisfied Then determine the hydrological variables. In time If an anomaly occurs, record the abnormal hydrological variables, the time of the anomaly, and the causal path.

[0119] In this embodiment, step S5 includes the following steps:

[0120] S51. After detecting abnormal hydrological monitoring data, construct a causal source search space for the abnormal hydrological monitoring data based on the causal model of the hydrological monitoring data, define the set of abnormal variables, and expand the source variable set according to the hierarchy of the reverse causal relationship in the causal model:

[0121] ;

[0122] in, Represents the initial set of variables for an anomaly. This represents the set of anomalous impact variables traced back to the m-th level, where M is the maximum tracing depth;

[0123] S52. Calculate the causal influence strength of each variable in the anomaly tracing path. And perform a weighted ranking of abnormal causal relationships:

[0124] ;

[0125] in, This is the dynamic anomaly detection threshold. Hydrological variables right The time step that has an effect Reflects variables For abnormal hydrological variables The degree of impact;

[0126] S53. Sort the anomaly tracing paths according to the strength of causal influence and generate a set of anomaly tracing candidate paths. }:

[0127] ;

[0128] in, As a threshold for causal attribution strength, only those with a causal influence strength greater than [a certain threshold] are retained. The path to tracing the source;

[0129] S54. Combining fuzzy entropy optimization strategy to optimize the set of candidate paths for anomaly tracing. Optimize by calculating the comprehensive entropy weight score for each tracing path. ;

[0130] S55. Based on the comprehensive entropy weight score, select the highest score as the final anomaly tracing path and construct a causal chain of abnormal hydrological monitoring data. :

[0131] ;

[0132] Abnormal causal chain Record the layer-by-layer impact path of anomalous variables to form the final causal analysis results of anomalous hydrological monitoring data.

[0133] A machine learning-based hydrological data management system, used to execute a machine learning-based hydrological data management method, includes the following modules:

[0134] The data acquisition module is used to acquire hydrological monitoring data in real time through the hydrological monitoring sensor network. The hydrological monitoring data includes rainfall, flow velocity, water level, evaporation and temperature. The acquired data is stored according to timestamps to form a hydrological monitoring dataset.

[0135] The data preprocessing module is used to perform data cleaning, noise reduction, missing value imputation, outlier removal and data standardization on the hydrological monitoring dataset to form a hydrological monitoring dataset with uniform spatiotemporal resolution.

[0136] The fuzzy entropy calculation module is used to calculate the fuzzy entropy value of each hydrological variable in the hydrological monitoring dataset and construct the fuzzy entropy matrix of the hydrological monitoring data to measure the uncertainty of the hydrological variables.

[0137] The causal relationship modeling module is used to construct a causal relationship model between various hydrological variables in hydrological monitoring data based on hydrological monitoring datasets and their fuzzy entropy matrices, using Granger causal analysis method. It uses unweighted vector autoregression model and weighted vector autoregression model to perform causality test, determine the direct and indirect causal relationships between hydrological variables, and generate a causal model of hydrological monitoring data represented in the form of a directed acyclic graph.

[0138] The real-time anomaly detection module is used to continuously monitor the hydrological monitoring dataset and detect anomalies in the real-time hydrological monitoring data based on the causal model of the hydrological monitoring data combined with the fuzzy entropy optimization strategy. It calculates the causal weighted deviation, identifies abnormal hydrological monitoring data according to the dynamically adjusted anomaly detection threshold, and records the abnormal variables and the time of occurrence.

[0139] The anomaly tracing analysis module is used to perform reverse causal analysis based on the causal model of hydrological monitoring data after detecting abnormal hydrological monitoring data. It generates candidate paths for anomaly tracing based on Granger causal analysis results and fuzzy entropy evaluation results, and calculates the comprehensive entropy weight score by combining fuzzy entropy optimization strategy to optimize the anomaly tracing path. Finally, it constructs the causal chain of abnormal hydrological monitoring data and forms a complete anomaly analysis result.

[0140] Example 1: On July 15, 2024, at 15:35:00, the S12 hydrological monitoring station in a tributary of the middle reaches of the A River detected a sudden increase in water level. The water level jumped from 3.21 meters to 3.96 meters in a short period of time, an increase of 0.75 meters in 10 minutes. According to the historical hydrological data of this basin, the water level increase in this area under similar rainfall conditions in the past five years has never exceeded 0.45 meters / 10 minutes. At the same time, the flow velocity at the upstream S09 hydrological monitoring station increased significantly in a short period of time, jumping from 1.2 m / s to 2.8 m / s, and then recovered to 1.3 m / s within 20 minutes. Such violent fluctuations are usually related to sudden flood discharge, extreme rainfall, or abnormal monitoring equipment. Therefore, it is necessary to automatically detect abnormal data and conduct further source tracing analysis to determine the root cause.

[0141] The system first continuously monitors the hydrological data of the S12 hydrological monitoring station and its surrounding stations through the data acquisition module. The data sampling interval is 5 minutes. At 15:35:00, the system found that the water level data of the S12 station deviated from the historical standard deviation of 3.4 times, which exceeded the dynamically set anomaly detection threshold, triggering the real-time evaluation mechanism of the anomaly detection module.

[0142] Real-time calculation results:

[0143] Site number: S12;

[0144] Monitoring time: 15:35:00 on July 15, 2024;

[0145] Recorded water level: 3.96 meters (water level 5 minutes prior: 3.21 meters);

[0146] The largest increase in the same period in history: 0.45 meters per 10 minutes;

[0147] Current increase: 0.75 meters / 10 minutes (66.7% higher than the historical maximum increase);

[0148] Calculate the deviation metric: 3.4 (exceeding the dynamic anomaly threshold of 3.1);

[0149] The system, through causal weighted deviation calculation, confirmed the water level anomaly and automatically generated an anomaly data report at 15:36:00. The report included the anomaly variables, the time of the sudden increase in water level, the magnitude of the anomaly, and the influencing factors. The report was immediately uploaded to the hydrological management system and triggered the anomaly tracing module.

[0150] The system initiates the anomaly source analysis module. First, based on the causal model of hydrological monitoring data, it tracks the possible sources of the anomaly. Causal network analysis shows that the water level at station S12 has a causal weight of 0.92 with the flow velocity at upstream station S09 and a lag time of approximately 15 minutes.

[0151] Source tracing path construction:

[0152] At 15:20:00, the flow velocity at station S09 suddenly increased from 1.2 m / s to 2.8 m / s.

[0153] At 15:35:00, the water level at the S12 station suddenly rose from 3.21 meters to 3.96 meters.

[0154] Since the flow velocity change at station S09 lagged behind the water level at station S12, the system calculated the causal weighted deviation and determined that station S09 was likely the key influencing factor for the abnormal water level event.

[0155] Source tracing data analysis:

[0156] The abnormal flow rate occurred at station S09 at 15:20:00 on July 15, 2024.

[0157] The abnormal water level occurred at station S12 at 15:35:00 on July 15, 2024.

[0158] Calculated lag time: 15 minutes (consistent with historical causal relationship);

[0159] Causal influence strength: 0.92 (high causal weight);

[0160] To further determine the cause of the abnormal flow velocity, the system analyzed the rainfall data of station S09 and found that the rainfall at station S03 upstream of station S09 showed an abnormal peak at 14:50:00. The rainfall increased from 1.5 mm to 48.7 mm in 10 minutes, and the short-term precipitation intensity reached above the 95th percentile of the historical data, which is an extreme rainfall event.

[0161] The system ultimately determined that the extreme rainfall event at station S03 was the root cause of the abnormal water level rise and constructed a complete anomaly tracing path:

[0162] ;

[0163] The system generated a complete anomaly tracing report at 15:37:00 and pushed it to the hydrological management center. Since the abnormal rainfall at the S03 station may affect multiple downstream basins, the management center, based on the analysis results, made advance arrangements for the downstream reservoir flood discharge strategy to avoid the risk of exceeding the warning water level.

[0164] This embodiment demonstrates that the method of the present invention can achieve efficient and accurate anomaly detection and source tracing analysis in actual hydrological monitoring scenarios, exhibiting higher detection accuracy, shorter source tracing paths, and faster anomaly location capabilities. In watershed hydrological anomaly early warning, this method can detect anomalies 15 minutes in advance and trace the cause of the anomaly within 10 minutes, providing strong data support for water resource management and disaster prevention early warning.

[0165] This invention constructs a causal model of hydrological monitoring data based on Granger causality analysis, and adjusts the weights of causal relationships by combining fuzzy entropy optimization. This enables more accurate identification of the true causal relationships between hydrological variables, avoids misjudgments caused by spurious correlations between dependent variables, and improves the reliability of anomaly tracing by introducing causal inference technology, so that the system can automatically trace back the most critical anomaly triggering factors when anomalies occur and quantify their impact.

[0166] In the process of anomaly detection in real-time hydrological monitoring data, this invention combines causal weighted deviation calculation and fuzzy entropy optimization strategy. By dynamically adjusting the anomaly detection threshold, the anomaly identification becomes more accurate and adaptive. This invention calculates the causal weighted deviation of hydrological variables and combines it with fuzzy entropy to dynamically optimize the threshold, enabling the system to adaptively adjust the anomaly judgment criteria based on real-time data characteristics, ensuring stable operation under different hydrological environments.

[0167] This invention constructs a set of candidate paths for anomaly tracing during the anomaly tracing process, and optimizes and filters the anomaly tracing paths by calculating the comprehensive entropy weight score through a fuzzy entropy optimization strategy. This enables the final anomaly causal chain to more accurately reflect the real anomaly transmission process. By calculating the causal influence intensity and combining fuzzy entropy optimization to remove variables with low correlation, the conciseness and effectiveness of the tracing path are ensured, thereby improving the efficiency and interpretability of anomaly tracing.

[0168] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for hydrological data governance based on machine learning, characterized in that, Comprising the following steps: S1. Obtain hydrological monitoring data and preprocess to generate a hydrological monitoring data set with uniform spatio-temporal resolution; S2. Calculate the fuzzy entropy values of each hydrological variable for the hydrological monitoring data set to form a fuzzy entropy evaluation result for measuring the uncertainty of each hydrological variable, and construct a fuzzy entropy matrix for the hydrological monitoring data; S3. Based on the hydrological monitoring data set and its fuzzy entropy matrix, use Granger causality analysis method to construct a causal relationship model between each hydrological variable in the hydrological monitoring data, determine the direct and indirect causal relationships between each hydrological variable through causal test under different time lags, and generate a hydrological monitoring data causal model represented in the form of a directed acyclic graph; S4. Continuously monitor the hydrological monitoring data set, and use the hydrological monitoring data causal model combined with the fuzzy entropy optimization strategy to detect anomalies in real-time hydrological monitoring data set, and dynamically adjust the anomaly detection threshold for real-time identification of abnormal hydrological monitoring data; The step S4 comprises the following steps: S41. Continuously monitoring the real-time hydrological monitoring dataset over the monitoring period, and for each sampling time instant calculating a hydrological variable value a measure of deviation from a hydrological variable historical distribution ; S42. Combine the hydrological monitoring data causal model G for each hydrological variable Causal impact variable set within lag window L Perform causal weighted bias calculation: ； wherein, is a set of variables influencing the causal model, is a causal weighting factor, is a hydrological variable at a lag time step, a measure of bias of the hydrological variable after integrated causal influence; S43. Combine the fuzzy entropy optimization strategy for each hydrological variable Compute dynamic anomaly detection threshold : ； wherein, is a hydrological variable is a mean deviation measure in historical data, is a hydrological variable is a fuzzy entropy value of the hydrological variable, is a reference anomaly detection coefficient, is a fuzzy entropy influence factor; S44. At each sampling time , if , it is determined that the hydrological variable occurs an anomaly at time , and the abnormal hydrological variable, the time of anomaly occurrence and the causal influence path are recorded; S5. When abnormal hydrological monitoring data is detected, perform reverse causal analysis on the abnormal hydrological monitoring data according to the hydrological monitoring data causal model, generate abnormal source candidate paths based on the Granger causality analysis results and fuzzy entropy evaluation results, and form causal chains of abnormal hydrological monitoring data; S6. Perform fuzzy entropy optimization processing on the abnormal source candidate paths, filter out key abnormal causal paths according to the fuzzy entropy values of the hydrological variables in each causal chain and their influence weights in the causal chain, and further locate the root cause of the abnormal hydrological monitoring data.

2. The hydrological data governance method based on machine learning according to claim 1, characterized in that, The step S1 comprises: S11. Real-time acquisition of hydrological monitoring data through a hydrological monitoring sensor network, wherein the hydrological monitoring data includes rainfall, flow rate, water level, evaporation and temperature, and the hydrological monitoring data is preliminarily stored according to the time stamp t to form an original hydrological data set: ； where P is the rainfall, V is the flow rate, H is the water level, E is the evaporation, and T is the temperature, denotes the value of the hydrological variable at time t, is a set of times; S12. Data cleaning of the original hydrological data set, including detection and elimination of outliers, filling of missing data and removal of sensor noise, to form a preliminarily cleaned hydrological monitoring data set; S13. Data standardization processing of the preliminarily cleaned hydrological monitoring data set to make each hydrological variable have the same numerical scale; S14. Reconstruct the standardized hydrological monitoring dataset according to the timestamp, so that all hydrological variables are aligned in the same time step, forming a unified spatiotemporal resolution hydrological monitoring dataset : ； wherein, is the reconstructed time set.

3. The method of claim 1, wherein, The step S2 comprises the following steps: S21. Calculate the membership value of each hydrological variable value in the hydrological monitoring data set according to the fuzzy membership function At each sampling time According to the fuzzy membership function, the membership value is calculated : ； in, Representing hydrological variables At sampling time The standardized value, Hydrological variables In the dataset The mean of the middle, Hydrological variables The total number of samples, These are adjustment parameters used to reflect hydrological variables. The degree of sensitivity to deviations from the mean is used to strengthen or suppress the impact of abnormal fluctuations in fuzzy membership calculation. Hydrological variables At sampling time The standardized value; S22. Based on the membership value For each hydrological variable value The fuzzy entropy value is calculated based on the membership value and the deviation of each sampling point from the mean value : ； wherein, denotes the natural logarithm, is a weighting factor for adjusting the hydrological variable the influence of extreme outliers on the fuzzy entropy, is a hydrological variable is the standard deviation of the hydrological variable, reflecting the degree of data dispersion; S23. The fuzzy entropy values of each hydrological variable are calculated according to the following formula: According to the index order of the hydrological variables, the fuzzy entropy matrix F of the hydrological monitoring data is formed: ； wherein, , , , and correspond to the fuzzy entropy values of the rainfall P, the flow rate V, the water level H, the evaporation amount E and the temperature T, respectively.

4. The hydrological data governance method based on machine learning according to claim 1, characterized in that, The step S3 comprises the following steps: S31. Based on the hydrological monitoring dataset and its fuzzy entropy matrix, each pair of hydrological variable values With , , the unweighted vector autoregressive model and the weighted vector autoregressive model were constructed under different time lags L, and Granger causality test was performed: ； ； wherein, is a hydrological variable is an autoregressive coefficient at lag k, is a hydrological variable is an influence coefficient of on at lag k, is an influence coefficient of on at lag k, and are error terms of the model, respectively, is a weighting factor based on the fuzzy entropy matrix; S32. Set null hypothesis for hydrological variable value on no Granger causality, i.e. relying on unweighted vector autoregressive model, set alternative hypothesis for hydrological variable value on Granger causality, i.e. employing weighted vector autoregressive model, perform F-statistic test at pre-set significance level , if null hypothesis is rejected : then determine that hydrological variable value Granger-causally affects hydrological variable value and record direct causal relationship and corresponding weighting factor ; S33. Based on the result of the test in step S32, a hydrological monitoring data causal model represented by a directed acyclic graph G = (V, Z) is constructed, where the vertex set respectively corresponds to the rainfall P, the flow rate V, the water level H, the evaporation E and the temperature T, and the edge set Z contains a directed edge from the hydrological variable to the hydrological variable if and only if it is determined in step S32 that Granger causally influences , and the weight of the edge is determined by the corresponding weighted factor , to generate the final hydrological monitoring data causal model.

5. The method of claim 1, wherein, The step S5 comprises the following steps: S51. After detecting abnormal hydrological monitoring data, construct a causal source search space for the abnormal hydrological monitoring data based on the hydrological monitoring data causal model, define an abnormal variable set, and expand the source variable set according to the hierarchical reverse causal relationship in the causal model: ； wherein, represents a set of abnormal initial variables, represents a set of abnormal influence variables traced to the mth layer, M being the maximum trace depth; S52. Calculate the causal influence strength of each variable in the anomaly provenance path and weight and rank the anomaly causal relationships: ； wherein, is a dynamic anomaly detection threshold, is a hydrological variable to an influencing lag time step, reflects the influence of the hydrological variable on the hydrological variable with anomaly; S53. Sort the abnormality backtracking paths according to the causal influence strength, and generate an abnormality backtracking candidate path set : ； wherein, is a causal traceability strength threshold, only traceability paths with a causal impact strength greater than are retained; S54. Combining the fuzzy entropy optimization strategy to trace the abnormal candidate path set Optimization is carried out, and the comprehensive entropy weight score of each trace path is calculated ; S55. According to the comprehensive entropy weight score, the highest score is selected as the final abnormality traceability path, and the abnormal hydrological monitoring data causal chain is constructed : ； Abnormal causal chain Record the layer-by-layer influence path of abnormal variables to form the final abnormal hydrological monitoring data causal analysis result. 6.A hydrological data governance system based on machine learning, configured to perform the method of any one of claims 1-5. Comprising the following modules: A data acquisition module for real-time acquisition of hydrological monitoring data through a hydrological monitoring sensor network, wherein the hydrological monitoring data includes rainfall, flow rate, water level, evaporation and temperature, and the acquired data is stored according to the time stamp to form a hydrological monitoring data set; A data preprocessing module for data cleaning, denoising, missing value filling, outlier elimination and data standardization processing of the hydrological monitoring data set to form a hydrological monitoring data set with uniform spatio-temporal resolution; The fuzzy entropy calculation module is configured to calculate fuzzy entropy values of each hydrological variable in the hydrological monitoring data set and construct a fuzzy entropy matrix of the hydrological monitoring data, which is used to measure the uncertainty of the hydrological variable. The causal relationship modeling module is configured to construct a causal relationship model between each hydrological variable in the hydrological monitoring data based on the hydrological monitoring data set and the fuzzy entropy matrix thereof, perform causality test by using a non-weighted vector autoregressive model and a weighted vector autoregressive model, determine direct and indirect causal relationships between the hydrological variables, and generate a causal model of the hydrological monitoring data in the form of a directed acyclic graph. The real-time anomaly detection module is configured to continuously monitor the hydrological monitoring data set, perform anomaly detection on real-time hydrological monitoring data based on the causal model of the hydrological monitoring data and a fuzzy entropy optimization strategy, calculate a causal weighted deviation, identify abnormal hydrological monitoring data according to a dynamically adjusted anomaly detection threshold, and record abnormal variables and occurrence times. The anomaly tracing analysis module is configured to, after detecting abnormal hydrological monitoring data, perform reverse causal analysis based on the causal model of the hydrological monitoring data, generate an abnormal tracing candidate path based on the Granger causal analysis result and the fuzzy entropy evaluation result, calculate a comprehensive entropy weight score by combining the fuzzy entropy optimization strategy, optimize the abnormal tracing path, finally construct a causal chain of the abnormal hydrological monitoring data, and form a complete abnormal analysis result.

Images