A water resources monitoring method based on big data

Abstract

This invention provides a big data-based water resources monitoring method applicable to the dynamic management of surface water bodies, watershed scheduling, and urban water systems. The method includes: collecting multi-source data such as water quality and water level using IoT and remote sensing technologies to generate structured water resources data; constructing a spectral feedback-TCN model to predict water quality status using multiple variables, dynamically adjusting the model structure through convolutional weight spectral expansion and power-law feature extraction to improve prediction accuracy and interpretability; and constructing a reinforcement learning-driven water level anomaly detection model, combining a penalty objective function, a SemiDICE strategy, and a CORSDICE confidence correction mechanism, outputting the strategy correction result as a dynamic prior embedded Bayesian change point detection model to achieve sensitive response to sudden water level anomalies and identification of non-stationary states. This method effectively improves the adaptability, stability, and real-time early warning capabilities of water resources monitoring systems.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a water resource monitoring method based on big data. Background Technology

[0002] There is an urgent need to build an efficient, intelligent, and dynamically responsive water resource monitoring and early warning system to improve the scientific and forward-looking nature of water resource management. Although many intelligent water resource monitoring methods are widely used in practical scenarios, integrating IoT sensing devices, edge computing nodes, and deep learning models, and capable of automatically collecting and predicting trends of key hydrological parameters, the following technical bottlenecks still exist in dealing with complex hydrological dynamics and the identification of non-stationary states:

[0003] First, most systems adopt an end-to-end training mechanism, which mainly focuses on the mapping relationship between input and output, and lacks structural feedback adjustment of the evolution law of features inside the model. For example, when convolutional neural networks process time series data, the spectral structure information reflected by their weight tensors often contains important dynamic change features. However, existing methods generally ignore the modeling and utilization of such spectral structure features, resulting in insufficient model interpretability, weak robustness, and parameter adjustment that heavily relies on human experience, making it difficult to maintain stable prediction performance in volatile environments.

[0004] Secondly, although some systems have introduced techniques such as Bayesian methods and change point detection algorithms to identify sudden changes in water level, most methods rely on fixed parameter configurations or static prior distributions, lacking the ability to dynamically match environmental conditions and making it difficult to adapt to hydrological processes with non-stationary characteristics. When faced with sudden water level anomalies, landslide fluctuations, or extreme hydrological events, problems such as response delays and low detection accuracy often occur, seriously affecting the effectiveness and reliability of the early warning mechanism. Summary of the Invention

[0005] This invention aims to address the shortcomings of existing intelligent water resource monitoring systems in deep feature feedback modeling and non-stationary hydrological anomaly identification, proposing a big data-based water resource monitoring method. This method integrates a spectral feedback-TCN model with a reinforcement learning-driven water level anomaly detection model to construct an intelligent monitoring and early warning mechanism with adaptability, interpretability, and high response sensitivity. It is applicable to various scenarios such as complex surface water bodies, watershed scheduling, reservoir operation, and urban water management. Specifically, the spectral feedback-TCN model performs spectral structure expansion and power-law feature extraction on the weight tensors of each convolutional layer to form spectral structure indices, and based on these indices… By dynamically adjusting network structure parameters, adaptive modeling and accurate prediction of multivariate water quality status can be achieved, enhancing the model's ability to perceive the structure and stability of complex water quality time series data. The water level anomaly detection model integrates a penalty objective function, a SemiDICE behavior correction strategy, and a CORSDICE confidence factor optimization mechanism. Based on this, the strategy correction results are dynamically embedded into the Bayesian online change point detection model as a priori, enabling sensitive identification and dynamic calibration of non-stationary sudden water level changes. Overall, this method effectively improves the water resources monitoring system's ability to intelligently identify and provide real-time early warning of water quality evolution trends and the risk of sudden water level changes.

[0006] This invention provides a water resource monitoring method based on big data, which includes the following steps:

[0007] Step S1: Use the Internet of Things (IoT) interface protocol to connect to the online water monitoring equipment, collect water quality parameter data in real time, use remote sensing images to obtain water body outline data and sediment content data, and use ETL scheduling tools to achieve synchronous data collection to obtain multi-source water resource data.

[0008] Step S2: Preprocess the multi-source water resources data to generate structured water resources data;

[0009] Step S3: Construct a spectral feedback-TCN model, process structured water resources data through the spectral feedback-TCN model, predict water quality status, and generate multivariate prediction results; the spectral feedback-TCN model includes the TCN model;

[0010] Step S4: Construct a water level anomaly detection model, input structured water resources data into the water level anomaly detection model, perform water level anomaly detection, and generate water level change detection results;

[0011] Step S5: Combine the multivariate prediction results and the water level change detection results to conduct early warning response and visualization analysis.

[0012] Furthermore, the process of generating multivariate prediction results through the spectral feedback-TCN model specifically includes the following steps:

[0013] Step S31: Perform sliding window slicing on the structured water resources data to construct a training sample pair set;

[0014] Step S32: Configure the causal convolution, dilation rate, and residual connections of the TCN model; input the training sample pair set into the TCN model for training, record the weight matrix of each convolution kernel in the TCN model, and obtain the trained TCN model and the set of convolution kernel weight matrices.

[0015] Step S33: Reconstruct the dimensions of each weight tensor in the convolution kernel weight matrix set, expand it into a two-dimensional weight matrix form, and obtain a two-dimensional weight matrix set; use a sliding window strategy with a fixed aspect ratio to divide each two-dimensional weight matrix in the two-dimensional weight matrix set into sub-matrices; calculate the correlation matrix of each sub-matrix and extract the eigenvalue sequence; concatenate all eigenvalue sequences to construct an average spectral distribution, perform power-law fitting on the average spectral distribution, and use the Hill Estimator to obtain the corresponding PL Alpha index; summarize the PL Alpha of all layers into a spectral structure index set.

[0016] Step S34: Combine the spectral structure index set to determine the PL Alpha index status, dynamically adjust the corresponding layer hyperparameters of the TCN model to obtain the optimized TCN model, and generate multivariate prediction results by combining the optimized TCN model.

[0017] Furthermore, the process of the water level anomaly detection model detecting water level anomalies and generating water level change detection results specifically includes the following steps:

[0018] Step S41: Slice the water level monitoring values ​​in the structured water resources data into time-series states by sliding window, and construct hydrological behavior actions, including regulating sluice gates, starting drainage, and alarm status; pair the time-series states and hydrological behavior actions to form state-action pairs; standardize and denoise the state-action pairs, extract time-series statistical features, and obtain state-action sequences.

[0019] Step S42: Based on the state-action sequence, calculate the immediate reward at each time point, introduce a cost function to characterize the potential threat of the state-action combination to system security, and then introduce a penalty factor to construct a penalty objective function to penalize high-risk behaviors and guide the policy to converge to a low-risk trajectory; construct a value function estimation model based on the state-action sequence, train the value function estimation model through the penalty objective function, and evaluate the action reward under the current policy; introduce a dual function and a regularization factor, and calculate the behavior layer correction factor sequence through the SemiDICE semi-gradient policy; thereby characterizing the degree of deviation of the current policy from the historical policy and achieving a sensitive response to high-risk behaviors; the dual function reflects the state value potential of each state;

[0020] Step S43: Input the behavior layer correction factor sequence into the state layer to construct state balance constraints, and learn the correction term of the current policy in the state dimension through the state dual function and the auxiliary estimation function, thereby characterizing the steady-state shift from the historical data distribution; use the two-stage optimization mechanism of CORSDICE to solve the correction term, minimize the expected fitting error of the auxiliary estimation function and the regularization cost of the state dual function respectively, and obtain the state layer confidence correction factor sequence.

[0021] Step S44: Construct a Bayesian online change point detection model and embed the state layer confidence correction factor sequence as a prior dynamic weight into the model to achieve dynamic calibration of the prior probability of abrupt changes; improve the sensitivity to non-stationary water level changes through this mechanism and output the water level abrupt change detection results.

[0022] By adopting the above solution, the beneficial effects achieved by the present invention are as follows:

[0023] This invention introduces a spectral feedback-TCN model to achieve structural understanding and adaptive prediction of multivariate water quality time series data. Based on the traditional TCN architecture, this model incorporates convolutional layer weight spectral expansion and power-law feature extraction mechanisms to establish a spectral structure feedback loop, dynamically adjusting the model's hyperparameters according to the PL Alpha index of each layer. This mechanism enhances the system's adaptability to the evolution of water quality parameters and its ability to capture abnormal fluctuations, solving the problems of rigid structure and poor prediction stability in existing systems, and significantly improving the ability to identify water quality trends in complex hydrological environments.

[0024] This invention constructs a reinforcement learning-driven water level anomaly detection model, achieving accurate identification and dynamic response to sudden, non-stationary water level changes. The model combines a penalty objective function and the SemiDICE strategy to correct behavioral biases, further introduces the CORSDICE mechanism to extract state-level confidence correction factors, and embeds a Bayesian online change point detection model as a dynamic prior, realizing probabilistic self-calibration of abrupt change detection. This method improves the accuracy and response speed of water level anomaly detection, solves the problem that existing systems cannot adapt to non-stationary changes, and enhances the real-time early warning capability for extreme hydrological events.

[0025] In summary, the spectral feedback-TCN model and the water level anomaly detection model proposed in this invention form a synergistic mechanism in dynamic water resource monitoring, which not only improves the ability to predict water quality change trends but also enhances the intelligent identification capability of water level abrupt change risks. The overall system has high interpretability, adaptability, and real-time performance, and can be widely applied to watershed management, reservoir scheduling, and urban water systems, providing high-precision and highly reliable intelligent technical support for water resource security management. Attached Figure Description

[0026] Figure 1 The water level prediction curve and abrupt change probability heat map proposed in Example 6;

[0027] Figure 2 This is the multivariate radar chart of water quality proposed in Example 6;

[0028] Figure 3 This is a line graph of the spectral structure index of the TCN model proposed in Example 6.

[0029] Figure 1 In the middle, the blue line represents the water level prediction, and the red bars represent the probability of sudden changes at each time point; Figure 2 In this study, multiple variables include ammonia nitrogen, dissolved oxygen, COD, and pH. Figure 3 In the middle, the bottom side represents the number of model layers, and the left side represents the PL Alpha value. Detailed Implementation

[0030] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0031] Example 1: This invention provides a water resource monitoring method based on big data, which includes the following steps:

[0032] Step S1: Use the Internet of Things (IoT) interface protocol to connect to the online water monitoring equipment, collect water quality parameter data in real time, use remote sensing images to obtain water body outline data and sediment content data, and use ETL scheduling tools to achieve synchronous data collection to obtain multi-source water resource data.

[0033] Step S2: Preprocess the multi-source water resources data to generate structured water resources data;

[0034] Step S3: Construct a spectral feedback-TCN model, process structured water resources data through the spectral feedback-TCN model, predict water quality status, and generate multivariate prediction results;

[0035] Step S4: Construct a water level anomaly detection model, input structured water resources data into the water level anomaly detection model to detect water level anomalies, and generate water level change detection results;

[0036] Step S5: Combine the multivariate prediction results and the water level change detection results to conduct early warning response and visualization analysis.

[0037] Example 2, based on Example 1, describes the process of generating multivariate prediction results using the spectral feedback-TCN model, specifically including the following steps:

[0038] Step S31: Perform sliding window slicing on the structured water resources data to construct a training sample pair set;

[0039] Step S32: Construct the TCN model, configure causal convolution, dilation rate, and residual connections; input the training sample pair set into the TCN model for training, record the weight matrix of each convolution kernel in the TCN model, and obtain the trained TCN model and the set of convolution kernel weight matrices.

[0040] Step S33: Reconstruct the dimensions of each weight tensor in the convolutional kernel weight matrix set, expanding it into a two-dimensional weight matrix form to obtain a two-dimensional weight matrix set; using a sliding window strategy with a fixed aspect ratio, divide each two-dimensional weight matrix in the two-dimensional weight matrix set into sub-matrices; calculate the correlation matrix of each sub-matrix and extract the eigenvalue sequence; concatenate all eigenvalue sequences to construct an average spectral distribution, perform power-law fitting on the average spectral distribution, and use the Hill Estimator to obtain the corresponding PL Alpha index; summarize the PL Alpha of all layers into a spectral structure index set, using the following formula:

[0041] Hill Estimator power-law exponential estimation formula:

[0042] ;

[0043] in, Represents the power-law exponent. This represents the layer index of the TCN model. Indicates the first The power-law exponent estimate corresponding to the layer, i.e., the heavy-tailedness index; This indicates the number of fitted points at the tail of the power law. Indicates the tail sample index. Indicates the total number of eigenvalues; Indicates the first eigenvalues, Indicates the first eigenvalues; Represent the natural logarithm function;

[0044] The Hill Estimator is a method for estimating the tail exponent (i.e., the degree of heavy tail) of a power-law distribution. It is commonly used in extreme value theory and heavy-tailed distribution analysis, and is especially suitable for analyzing empirical data distributions with long-tailed or heavy-tailed characteristics.

[0045] Step S34: Combine the spectral structure index set to determine the PL Alpha index status, dynamically adjust the corresponding layer hyperparameters of the TCN model to obtain the optimized TCN model, and generate multivariate prediction results by combining the optimized TCN model.

[0046] Example 3, based on Example 1, describes the process of generating multivariate prediction results, which specifically includes the following steps:

[0047] Step E1: Perform sliding window slicing on the structured water resources data to construct a training sample pair set;

[0048] Step E2: Construct the TCN model, configure causal convolution, dilation rate, and residual connections; input the training sample pair set into the TCN model for training to obtain the trained TCN model;

[0049] Step E3: Use the trained TCN model to infer the input samples, extract the feature representations of each time step, and generate a multi-dimensional feature output sequence; aggregate the output features to construct the overall feature representation vector;

[0050] Step E4: Based on the feature representation vector, use the trained TCN model to make predictions and output the corresponding multivariate prediction results, including the future trend prediction results of water level, water temperature, and sediment content.

[0051] Example 4, based on Example 2, describes the process of the water level anomaly detection model detecting water level anomalies and generating water level change detection results. The specific steps include:

[0052] Step S41: Slice the water level monitoring values ​​in the structured water resources data into time-series states by sliding window, and construct hydrological behavior actions, including regulating sluice gates, starting drainage, and alarm status; pair the time-series states and hydrological behavior actions to form state-action pairs; standardize and denoise the state-action pairs, extract time-series statistical features, and obtain state-action sequences.

[0053] Step S42: Based on the state-action sequence, calculate the immediate reward at each time point, introduce a cost function to characterize the potential threat of the state-action combination to system security, and then introduce a penalty factor to construct a penalty objective function to penalize high-risk behaviors and guide the policy to converge towards a low-risk trajectory; construct a value function estimation model based on the state-action sequence, train the value function estimation model through the penalty objective function, and evaluate the action reward under the current policy; introduce a dual function and a regularization factor, and calculate the behavior layer correction factor sequence through the SemiDICE semi-gradient policy; thereby characterizing the deviation of the current policy from historical policies and achieving a sensitive response to high-risk behaviors; the dual function reflects the state value potential energy of each state, and the formula used is as follows:

[0054] The formula for the penalty objective function is:

[0055] ;

[0056] in, Indicates timing state, Indicates hydrological behavior and actions. This represents the reward function after the penalty. Represents the original reward function. Indicates the penalty factor. Represents the cost function;

[0057] Formula for behavior correction ratio in SemiDICE semi-gradient policy:

[0058] ;

[0059] in, Indicates the behavior correction ratio. Indicates the current policy in state Select action The probability of; Represents historical datasets, This represents the behavior policy in the historical dataset in the state. Select action The probability of; The derivative of f-divergence inverse function, Describe the dual function. Represents the regularization factor. This represents the value function estimation model. Indicates nonnegative correction;

[0060] Step S43: Input the behavior layer correction factor sequence into the state layer to construct state balance constraints. Then, through the state dual function and the auxiliary estimation function, learn the correction term of the current policy in the state dimension to characterize the steady-state shift from the historical data distribution. Solve for the correction term using the two-stage optimization mechanism of CORSDICE, minimizing the expected fitting error of the auxiliary estimation function and the regularization cost of the state dual function respectively, to obtain the state layer confidence correction factor sequence. The formula used is as follows:

[0061] First-stage formula:

[0062] ;

[0063] ;

[0064] in, Indicates the next state. Denotes the auxiliary estimation function. Representing the state dual function The value in the current state, Representing the state dual function The value in the next state, Indicates the discount factor. This represents the dual residual term of the state transition; Expressing expected value based on experience; This represents minimizing the auxiliary estimation function;

[0065] Second-stage formula:

[0066] ;

[0067] in, Let represent the objective function defined by minimizing the dual function of the state. express The external objective function, It is defined as; Represents the initial state dual value, Represents the initial state distribution Medium sampling Expectations The derivative of the f-divergence conjugate function is represented;

[0068] The construction of state equilibrium constraints proposes structural conditions that must be satisfied, while optimizing the state dual function and the auxiliary estimation function is an efficient strategy and technical path to achieve the constraints. The two constitute a unity of goal and means.

[0069] Step S44: Construct a Bayesian online change point detection model and embed the state layer confidence correction factor sequence as a prior dynamic weight into the model to achieve dynamic calibration of the prior probability of abrupt changes; improve the sensitivity to non-stationary water level changes through this mechanism and output the water level abrupt change detection results.

[0070] Example 5, based on Example 2, specifically includes step S4: constructing a Bayesian online change point detection model, taking structured water resources data as input, extracting water level time series features, performing online posterior inference and mutation detection based on the model, and outputting water level mutation detection results; this method achieves rapid response and real-time detection of mutation points by continuously evaluating the changing trend of water level data, and is suitable for standard monitoring scenarios without prior reinforcement information.

[0071] Example 6, according to Figure 1 , Figure 2 , Figure 3 This embodiment is based on embodiment five. In this embodiment, step S5: combine the multivariate prediction results and the water level change detection results to perform early warning response and visualization analysis.

[0072] In this embodiment:

[0073] Table 1 shows the multivariate prediction results:

[0074] Table 1

[0075] ;

[0076] Analysis results: Predictions show that COD exceeds the national Class III surface water quality limit by more than double, and the water level has risen from 3.85m to the predicted 5.33m, showing a continuous upward trend and a dangerous trend.

[0077] Table 2 shows the results of the water level change detection:

[0078] Table 2

[0079] ;

[0080] Analysis results: Starting at 03:00, the mutation probability increased significantly, reaching 74%, and reached 86% at 04:00. This segment was marked as the "water level mutation interval", triggering a mutation alarm.

[0081] Visual analysis: water level prediction curve and hot zone map of mutation probability, water quality multivariate radar map and TCN model spectral structure index line chart.

[0082] The present invention and its embodiments have been described above. This description is not restrictive. The accompanying drawings are only one embodiment of the present invention, and the actual structure is not limited thereto. In short, if a person skilled in the art is inspired by this description and designs a similar structure and embodiment without departing from the spirit of the present invention, such design should fall within the protection scope of the present invention.

Claims

1. A water resources monitoring method based on big data, characterized in that: The method includes the following steps: Step S1: Collect multi-source water resource data; Step S2: Preprocess the multi-source water resources data to generate structured water resources data; Step S3: Construct a spectral feedback-TCN model, process structured water resources data through the spectral feedback-TCN model, predict water quality status, and generate multivariate prediction results; the spectral feedback-TCN model includes the TCN model; Step S4: Construct a water level anomaly detection model. Input structured water resources data into the water level anomaly detection model to detect water level anomalies and generate water level change detection results. The water level anomaly detection model includes a state construction unit, a correction calculation unit, a state correction unit, and a change detection unit. Step S5: Combine the multivariate prediction results and the water level change detection results to issue an early warning response; The process of generating multivariate prediction results using the spectral feedback-TCN model includes the following steps: Step S31: Construct a training sample pair set based on structured water resources data; Step S32: Input the training sample pair set into the TCN model for training to obtain the set of convolutional kernel weight matrices; Step S33: Combining the set of convolution kernel weight matrices, a set of spectral structure indices is obtained through matrix feature extraction and power-law fitting of the average spectral distribution; Step S34: Combine the set of spectral structure indicators to generate multivariate prediction results; The state construction unit obtains the state-action sequence based on structured water resources data; The correction computation unit calculates the immediate reward at each time point of the state-action sequence, introduces a cost function and a penalty factor, and constructs a penalty objective function; a value function estimation model is constructed based on the state-action sequence, the value function estimation model is trained through the penalty objective function, and a dual function and a regularization factor are introduced. The behavior layer correction factor sequence is obtained through the SemiDICE semi-gradient strategy. The state correction unit constructs state balance constraints based on the behavior layer correction factor sequence, and learns the correction term of the current policy in the state dimension through the state dual function and the auxiliary estimation function; it solves the correction term using the two-stage optimization mechanism of CORSDICE, minimizing the expected fitting error of the auxiliary estimation function and the regularization cost of the state dual function respectively, and obtains the state layer confidence correction factor sequence. The mutation detection unit combines the state layer confidence correction factor sequence to obtain the water level mutation detection results.

2. The water resource monitoring method based on big data according to claim 1, characterized in that: Step S33 specifically includes: reconstructing the dimensions of each weight tensor in the set of convolutional kernel weight matrices to obtain a two-dimensional weight matrix set; using a sliding window strategy with a fixed aspect ratio, dividing each two-dimensional weight matrix in the two-dimensional weight matrix set into sub-matrices; calculating the correlation matrix of each sub-matrix and extracting the eigenvalue sequence; concatenating all eigenvalue sequences to construct an average spectral distribution; performing power-law fitting on the average spectral distribution; using the Hill Estimator to obtain the corresponding PL Alpha index; and summarizing them into a set of spectral structure indices.

3. The water resource monitoring method based on big data according to claim 1, characterized in that: The state construction unit constructs a time-series state and hydrological behavior action based on structured water resources data, resulting in a state-action sequence.

4. The water resource monitoring method based on big data according to claim 1, characterized in that: The mutation detection unit constructs a Bayesian online change point detection model and combines it with the state layer confidence correction factor sequence to obtain the water level mutation detection results.

Images