Underground water level prediction method and system based on space-time attention gating neural network

Abstract

The invention discloses an underground water level prediction method and system based on a space-time attention gating neural network. The method comprises the following steps: constructing a historical underground water level space-time sequence data set; performing principal component analysis on the space-time sequence data set to obtain a dimension-reduced space-time sequence data set; for the spatio-temporal sequence data set after dimension reduction, constructing a plurality of spatio-temporal characteristic matrixes through a sliding window method; training an underground water level prediction model by using the plurality of spatial-temporal characteristic matrixes to obtain a trained underground water level prediction model; obtaining time-space sequence data of historical underground water level of a to-be-measured position, constructing a time-space characteristic matrix of the underground water level, and predicting the underground water level of the to-be-measured position by using the trained underground water level prediction model; according to the method, rapid and accurate prediction of the underground water level is realized, and the generalization ability, the prediction precision and the interpretability of an underground water level prediction model are greatly improved while the modeling cost is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of groundwater level prediction, and in particular to a groundwater level prediction method and system based on a spatiotemporal attention gated neural network. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] The dynamics of groundwater levels are a comprehensive reflection of hydrological elements. Groundwater level information can accurately grasp the amount of groundwater resources and provide a scientific basis for the rational development and utilization of groundwater. Therefore, accurate and rapid long-term groundwater level prediction is crucial to the safety, availability and sustainable management of groundwater resources.

[0004] Traditional methods for simulating groundwater levels using numerical models based on physical processes, such as modflow, GMS, FEFLOW, VisualMODEFLOW, etc., often require a large amount of information related to hydrogeological parameters, boundary conditions, domain properties, and aquifer structure, which results in high computational and time costs.

[0005] Deep learning (DL) technology has powerful data mining and data-driven capabilities. Deep learning agent models can be functionally close to physical models while significantly reducing cumbersome information requirements and computational burdens. Traditional machine learning methods have been widely studied in groundwater level prediction, such as support vector machines, adaptive neural fuzzy inference systems (ANFIS), genetic editing (GP), Gaussian process regression (GPR), and multivariate adaptive regression splines (MARS). However, due to the complexity and uncertainty of dynamic multivariate groundwater time series, these machine learning methods are difficult to learn the evolution characteristics of long time series.

[0006] The Gated Recurrent Unit (GRU) can mine and store the feature information of the selected time series and capture the dependencies of the input feature series in different time ranges. However, it cannot reasonably allocate the weights of the feature variables, and the random weight initialization may affect the model performance. In the process of training complex and long-term dependent data, the gradient disappearance or explosion problem is prone to occur, thus affecting the model performance and the accuracy of water level prediction.

[0007] In addition, in the groundwater system, the groundwater levels at different locations in the region have certain spatiotemporal correlations. The inability of GRU to extract this dynamic spatiotemporal correlation may lead to the groundwater level prediction model being unable to fully utilize the spatiotemporal information of the data. Summary of the invention

[0008] In order to solve the above problems, the present invention proposes a groundwater level prediction method and system based on spatiotemporal attention gated neural network, which extracts the main components of multi-location groundwater level data through principal component analysis, fully mines the spatiotemporal characteristics of the principal component data by using spatiotemporal attention, and combines the gated neural network (GRU) to quickly learn the groundwater level change characteristics, so as to achieve accurate prediction of the groundwater level.

[0009] In some embodiments, the following technical solutions are adopted:

[0010] A groundwater level prediction method based on spatiotemporal attention gated neural network, characterized by comprising:

[0011] Construct a historical groundwater level spatiotemporal series dataset;

[0012] Perform principal component analysis on the spatiotemporal series data set to obtain the spatiotemporal series data set after dimensionality reduction;

[0013] For the space-time series data set after dimensionality reduction, multiple space-time feature matrices are constructed by the sliding window method; the groundwater level prediction model is trained using multiple space-time feature matrices to obtain a trained groundwater level prediction model;

[0014] Obtain the spatiotemporal sequence data of the historical groundwater level at the location to be measured, construct the spatiotemporal characteristic matrix of the groundwater level, and use the trained groundwater level prediction model to predict the groundwater level at the location to be measured;

[0015] Among them, the groundwater level prediction model includes a spatiotemporal attention module and a gated neural network module. The spatiotemporal attention module includes a spatial attention unit and a temporal attention unit; the spatiotemporal feature matrix obtains weighted spatial data through the spatial attention unit, and the spatiotemporal feature matrix obtains weighted temporal data through the temporal attention unit. The weighted spatial data and weighted temporal data are fused as the output of the spatiotemporal attention module; the output of the spatiotemporal attention module is input into the gated neural network module to output the groundwater level prediction result.

[0016] Furthermore, the historical groundwater level spatiotemporal series dataset can be constructed as follows:

[0017] Construct a groundwater flow simulation model to calculate the spatiotemporal series data set of groundwater levels at multiple locations under changing environments;

[0018] The groundwater flow simulation model is specifically as follows:

[0019]

[0020] Where h is the height of the groundwater level, Kx, Ky, Kz are the hydraulic conductivity coefficients in the x, y, and z directions respectively, Ss is the specific storage rate, t is the time, and W is the source-sink term, which represents the amount of water flowing into or out of the system.

[0021] Alternatively, to construct a historical groundwater level spatiotemporal series dataset, you can also:

[0022] Obtain historical groundwater level information, filter and remove abnormal data points. The specific process is as follows:

[0023] Calculate each historical groundwater level data point P(P x1 ,P y1 ) and other data points (P x2 , P y2 ) between ;

[0024] Calculate the distance d between the kth point closest to the historical groundwater level data point P and point P k (P); take data point P as the center and d k (P) is the radius, and the k-neighborhood of the data point P is obtained;

[0025] Calculate the reachable distance from data point P to all data points in its k-neighborhood: Assuming point O is a data point in the k-neighborhood of data point P, compare the k-neighborhood radius of point O with the Euclidean distance between point O and point P, and select the larger one as the reachable distance resch_dist from data point P to point O k (P,O);

[0026] The limit reachable density of data point P is calculated based on the reachable distance, and the outlier factor Limit-LOF(P) is calculated based on the limit reachable density of historical groundwater level data point P; if the outlier factor Limit-LOF(P) is greater than 1, the data point P is determined to be an abnormal data point;

[0027] After screening out all the abnormal data points of the historical groundwater level, the mean point of each abnormal data point and each normal data point in its k-neighborhood is calculated. After adding the obtained mean point to the historical groundwater level dataset, the outlier factors of these abnormal data points are recalculated. If the obtained outlier factor is still greater than 1, it is regarded as the final abnormal data point and removed. Otherwise, it is retained as a normal data point.

[0028] Furthermore, the spatiotemporal series data set includes: corresponding information of time, location and water level.

[0029] Furthermore, principal component analysis is performed on the spatiotemporal series data set. The specific process is as follows:

[0030] Standardize the spatiotemporal series data set, construct the standardized time series data set into a spatiotemporal data matrix, and calculate the covariance matrix of the data matrix;

[0031] Perform eigenvalue decomposition on the covariance matrix to obtain eigenvalues ​​and eigenvectors;

[0032] Start accumulating from the largest eigenvalue until the cumulative explained variance ratio reaches or exceeds the set threshold; then the eigenvectors corresponding to these accumulated eigenvalues ​​are used as the main eigenvectors;

[0033] A dimensionality-reduced data set is formed based on the main eigenvectors.

[0034] Furthermore, for the space-time series data set after dimensionality reduction, the space-time feature matrix is ​​constructed by the sliding window method, specifically:

[0035] Assume that the time step of the sliding window is T; then each time series window includes T groups of M-dimensional spatial data after dimensionality reduction; that is, the spatiotemporal feature matrix

[0036] Sort by time, get X 1 , X 2 , X 3 , …, X k , can be used to predict the target groundwater level matrix O i , i∈(1, 2,…, k), k is the number of sliding windows.

[0037] Furthermore, the spatiotemporal feature matrix is ​​passed through the spatial attention unit to obtain weighted spatial data, specifically:

[0038] For each spatial dimension data in each spatiotemporal feature matrix X, calculate the spatial attention score;

[0039] Normalize the spatial attention scores to obtain the attention weights;

[0040] Based on the attention weight, the data of each spatial dimension is weighted and summed to obtain the weighted spatial data x′ t .

[0041] The spatiotemporal feature matrix passes through the temporal attention unit to obtain weighted temporal data, specifically:

[0042] For each time dimension data in each spatiotemporal feature matrix X, calculate the time attention score;

[0043] Normalize the temporal attention scores to obtain the attention weights;

[0044] Based on the attention weight, the data of each time dimension is weighted and summed to obtain the weighted time data h′t .

[0045] Furthermore, when training the gated neural network, an adaptive compensation algorithm is used to determine the optimization step size, specifically:

[0046]

[0047] Among them, l represents the step size used in each round of optimization, l max and l min They represent the maximum step size and the minimum step size, both of which are set values; p represents the number of iterations, and q represents the number of steps required to reach l. max .

[0048] In other embodiments, the following technical solutions are adopted:

[0049] A groundwater level prediction system based on spatiotemporal attention gated neural network, comprising:

[0050] Training data acquisition module, used to construct historical groundwater level spatiotemporal series dataset;

[0051] The data dimension reduction module is used to perform principal component analysis on the spatiotemporal series data set to obtain the spatiotemporal series data set after dimension reduction;

[0052] The model training module is used to construct multiple spatiotemporal feature matrices for the spatiotemporal series data set after dimensionality reduction through the sliding window method; the groundwater level prediction model is trained using the multiple spatiotemporal feature matrices to obtain a trained groundwater level prediction model;

[0053] The groundwater level prediction module is used to obtain the spatiotemporal sequence data of the historical groundwater level at the location to be measured, construct the spatiotemporal characteristic matrix of the groundwater level, and use the trained groundwater level prediction model to predict the groundwater level at the location to be measured;

[0054] Among them, the groundwater level prediction model includes a spatiotemporal attention module and a gated neural network module. The spatiotemporal attention module includes a spatial attention unit and a temporal attention unit; the spatiotemporal feature matrix obtains weighted spatial data through the spatial attention unit, and the spatiotemporal feature matrix obtains weighted temporal data through the temporal attention unit. The weighted spatial data and weighted temporal data are fused as the output of the spatiotemporal attention module; the output of the spatiotemporal attention module is input into the gated neural network module to output the groundwater level prediction result.

[0055] In other embodiments, the following technical solutions are adopted:

[0056] A terminal device includes a processor and a memory, wherein the processor is used to implement instructions; the memory is used to store multiple instructions, and the instructions are suitable for the processor to load and execute the above-mentioned groundwater level prediction method based on spatiotemporal attention gated neural network.

[0057] Compared with the prior art, the present invention has the following beneficial effects:

[0058] (1) The present invention extracts the features of the original sequence data from different spatial and temporal dimensions at the same time, processes the spatiotemporal sequence data through the spatiotemporal attention module, gradually extracts the response mechanism of the prediction variables to the spatiotemporal evolution, alleviates the negative impact of non-prediction variables, and solves the spatiotemporal information conflict to a certain extent.

[0059] The output of the spatiotemporal attention module can be quickly learned through a gated neural network (GRU) to learn the changing characteristics of the groundwater level and achieve fast and accurate prediction of the groundwater level. This greatly improves the generalization ability, prediction accuracy and interpretability of the groundwater level prediction model while reducing the modeling cost.

[0060] (2) The present invention firstly screens abnormal data to accurately find and remove abnormal data, thereby improving the accuracy of groundwater level data; then, the original time series data set is subjected to dimensionality reduction processing by principal component analysis, thereby reducing the complexity of the water level prediction model and reducing the risk of overfitting. At the same time, by retaining the main components, the influence of noise on the water level prediction model can be eliminated, thereby improving the generalization ability of the model.

[0061] By combining the spatiotemporal attention mechanism with data dimensionality reduction, the model can focus on important features during training, thereby reducing unnecessary calculations and invalid information propagation, and alleviating the problem of gradient vanishing or exploding. In addition, the gated neural network (GRU) has better gradient propagation capabilities than the standard recurrent neural network (RNN), which further enhances the performance of the model in long-term dependent data.

[0062] (3) When training the gated neural network, the present invention adopts an improved step size calculation method, and the corresponding step size varies according to the number of iterations. In the early iteration rounds, the step size is larger, and the global optimal area can be quickly found. As the number of iterations increases, the step size becomes smaller, and the local optimal point can be accurately found, while the speed of iterative optimization is improved. At the same time, the present invention randomly sets the neurons in the hidden layer to 0 with a certain probability in each iterative training, which can further improve the speed of iterative optimization, reduce the dependence of the prediction results on the hidden layer nodes, and improve the prediction accuracy.

[0063] Other features and additional advantages of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through the practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 It is a flow chart of a method for predicting groundwater level based on a spatiotemporal attention gated neural network in an embodiment of the present invention;

[0065] Figure 2 This is a schematic diagram of the structure of the spatiotemporal attention module in an embodiment of the present invention;

[0066] Figure 3 This is a schematic diagram of the structure of the groundwater level prediction model in an embodiment of the present invention;

[0067] Figure 4 It is a schematic diagram of comparison between the predicted value and the true value of the groundwater level prediction model in an embodiment of the present invention;

[0068] Figure 5 This is a diagram of ablation experiment results in an embodiment of the present invention. DETAILED DESCRIPTION

[0069] It should be noted that the following detailed descriptions are illustrative and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used in the present invention have the same meanings as those commonly understood by those skilled in the art to which the present application belongs.

[0070] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0071] Embodiment 1

[0072] In one or more embodiments, a groundwater level prediction method based on spatiotemporal attention gated neural network is disclosed, combined with Figure 1 , specifically including the following process:

[0073] S101: Obtain historical groundwater level information and construct a historical groundwater level spatiotemporal series dataset;

[0074] In this embodiment, the historical groundwater level information can be calculated by constructing a groundwater flow simulation model; in this embodiment, the specific formula of the groundwater flow simulation model is as follows:

[0075]

[0076] Where h is the water head (groundwater level height), Kx, Ky, Kz are the hydraulic conductivity coefficients in the x, y, and z directions respectively, Ss is the specific storage rate, t is the time, and W is the source-sink term, which represents the amount of water flowing into or out of the system (such as rainfall, pumping, recharge, etc.).

[0077] The groundwater flow simulation model is constructed through GMS-MODFLOW. Through detailed geological condition analysis (such as conductivity, porosity and boundary conditions), historical hydrological and climate data (such as rainfall, groundwater extraction and groundwater level data), accurate boundary conditions and initial conditions, and consideration of various reasonable source and sink items (such as precipitation, evaporation, recharge and discharge of surface water bodies, human activities (pumping wells, irrigation), etc.), the groundwater flow can be effectively simulated and the corresponding spatiotemporal series data set can be generated.

[0078] Of course, historical groundwater level information can also be obtained from real historical groundwater level record data; in order to ensure the accuracy of historical data, it is necessary to screen and remove abnormal data points from the acquired historical groundwater level record data; in order to ensure that abnormal data points can be accurately screened out, this embodiment adopts the following method to screen abnormal data points:

[0079] ① Calculate each historical groundwater level data point P(P x1 ,P y1 ) and other data points (P x2 , P y2 ) is:

[0080]

[0081] Among them, P x is the horizontal axis, indicating time; P y is the vertical axis, representing the groundwater level.

[0082] ② Calculate the distance d between the kth point closest to the historical groundwater level data point P and point P k (P); take data point P as the center and d k (P) is the radius, and the k-neighborhood of the data point P is obtained;

[0083] ③ Calculate the reachable distance from data point P to all data points in its k-neighborhood: Assuming point O is a data point in the k-neighborhood of data point P, compare the k-neighborhood radius of point O with the Euclidean distance between point O and point P, and select the larger one as the reachable distance resch_dist from data point P to point O k (P,O); Based on this method, the reachable distance from the data point P to each data point in its k-neighborhood is obtained;

[0084] ④ Calculate the limit reachable density of data point P based on the reachable distance:

[0085]

[0086] Based on the limit reachable density of the historical groundwater level data point P, the outlier factor Limit-LOF(P) is calculated to determine whether the data point P is an abnormal data point:

[0087]

[0088] Wherein, lrd(O) is the limit reachable density of data point O in the k neighborhood of historical groundwater level data point P, and lrd(P) is the limit reachable density of historical groundwater level data point P; d k (P) is the radius value of the k-neighborhood of data point P.

[0089] If the value of the outlier factor Limit_LOF(P) is greater than 1, it means that the historical groundwater level data point P is an abnormal data point.

[0090] ⑤ After all the historical groundwater level abnormal data points are screened out based on the above method, the mean point of each abnormal data point and each normal data point in its k-neighborhood is calculated. After adding the obtained mean point to the historical groundwater level data set, the outlier factors of these abnormal data points are recalculated. If the obtained outlier factor is still greater than 1, it is regarded as the final abnormal data point and eliminated. Otherwise, it is retained as a normal data point.

[0091] The method of this embodiment can avoid removing normal data points as outliers, thereby ensuring the accuracy of screening outlier data points and thus ensuring accurate training of the groundwater level prediction model.

[0092] The spatiotemporal series data set of this embodiment includes: time and water level data after outlier screening and corresponding spatial information, providing accurate training data for the deep learning model.

[0093] Among them, time data is the specific time when the groundwater level data is recorded, and the format can be date and time (such as 2024-07-27 14:00); spatial data includes the coordinate information of the water level observation point (longitude, latitude, depth, etc.). Water level data is the groundwater level value (meter) at a specific time and spatial location. In addition, the spatial characteristics of the groundwater level are the distribution of the water level data in space.

[0094] S102: Perform principal component analysis on the spatiotemporal sequence data set to obtain a space-time sequence data set after dimensionality reduction.

[0095] In this embodiment, the specific process of performing principal component analysis on the time series data set is:

[0096] S1021: Standardize the time series dataset, since the groundwater level time series at each location are numerical features, but there are differences between their values.

[0097] In order to facilitate the next step of data processing, the water level data is preprocessed, that is, standardized. The standardization formula is as follows:

[0098] y=(x-μ) / σ

[0099]

[0100] Among them, y is the standardized water level value, x represents the individual water level value of the data, n represents the number of x, μ represents the mean of all water level data, σ represents the standard deviation of all water level data, which reflects the degree of discreteness of a data set, and mean represents the average value of all water level data.

[0101] S1022: Construct the standardized time series data set into a spatiotemporal data matrix.

[0102] The spatiotemporal data matrix is ​​a high-dimensional data structure formed by combining time and space data; for example, assuming that there are 3 time points in a time step and each time point has 4 observation points, the structure of the spatiotemporal data matrix is ​​as shown in Table 1.

[0103] Table 1 Spatiotemporal data matrix

[0104] Time / Observation point Point A Point B Point C Point D Time 1 0.149867445 -0.004564648 -5.247485638 -5.251085758 Time 2 0.097516112 -0.042865194 -5.463336468 -5.466853142 Time 3 0.044204205 -0.085632294 -5.58537674 -5.589818478

[0105] In this matrix, the rows represent timestamps, the columns represent different observation points, and the value in each cell represents the water level at the corresponding time and observation point.

[0106] The spatiotemporal data matrix effectively integrates temporal and spatial data, providing a basic data structure for subsequent deep learning model training and analysis.

[0107] S1023: Calculate the covariance matrix of the data matrix, perform eigenvalue decomposition on the covariance matrix, and obtain eigenvalues ​​and eigenvectors; each eigenvalue represents the proportion of its corresponding eigenvector (principal component) in the total variance of the data.

[0108] S1024: Select the main eigenvectors as principal components according to the proportion of the cumulative explained variance.

[0109] The proportion of cumulative explained variance refers to the proportion of the total variance of the data that can be explained by the selected eigenvectors in principal component analysis. Usually, a threshold is set, such as 90% or 95%, indicating the proportion of the total variance of the data that the first few principal components are expected to explain. Starting from the largest eigenvalue, accumulate until the cumulative explained variance proportion reaches or exceeds the set threshold; the corresponding eigenvectors are the main eigenvectors.

[0110] S1025: Form a reduced-dimension data set based on the main eigenvectors.

[0111] The specific derivation process is as follows:

[0112] C x = E[(S - E[S])(S - E[S]) T ;

[0113] Among them, S represents a spatio-temporal data matrix, Cx is the covariance matrix, and E is the average function.

[0114] Assume that S is averaged, ES = 0, and it is deduced that:

[0115]

[0116] Assume that the matrix Y can be obtained by transforming the original sample data matrix:

[0117] Y = PS;

[0118] Among them is the dimensionality reduction matrix, where R represents the matrix, k*t indicates that the matrix is k rows and t columns; k is the number of features after dimensionality reduction, and t is the number of time points; is the target matrix, where k is the number of features after dimensionality reduction (usually k < n), and n is the number of features of the original data; is the original data matrix, where n is the number of features (dimensions) of the original data, and t is the number of time points.

[0119] Satisfy the following two conditions: (1) The elements on the diagonal are as large as possible. (2) The off-diagonal elements are 0. The formula can be transformed into:

[0120]

[0121] Therefore, a matrix P needs to be found such that PC x P T is a diagonal matrix, and the main eigenvector matrix P can be solved by the above eigenvalue decomposition method; furthermore, the reduced-dimension data Y is obtained, and the data is reduced from an n*t matrix to an m*t matrix.

[0122] The first 70% of the data after principal component analysis is used as the training set, and the last 30% of the data is used as the test set for the next step of model training and testing.

[0123] S103: For the space-time sequence training set after dimensionality reduction, multiple space-time feature matrices are constructed by the sliding window method; the groundwater level prediction model is trained using the multiple space-time feature matrices to obtain a trained groundwater level prediction model; and then the model is tested using the space-time sequence test set after dimensionality reduction.

[0124] In this embodiment, it is assumed that the time step of the sliding window is T; then each time series window includes T groups of M-dimensional spatial data after dimensionality reduction; the spatiotemporal feature matrix

[0125] Starting from the first time step of the time series, slide the window backward, move one step at a time, extract the data in the window, sort it in chronological order, and get the spatiotemporal feature matrix sequence X 1 , X 2 , X 3 , …, X k , k is the number of sliding windows; the spatiotemporal feature matrix at each moment can be used to predict the water level at each location at the next moment.

[0126] In this embodiment, combined with Figure 3 The groundwater level prediction model includes a spatiotemporal attention module and a gated neural network module, combined with Figure 2 The spatiotemporal attention module includes a spatial attention unit and a temporal attention unit; the spatiotemporal feature matrix passes through the spatial attention unit to obtain weighted spatial data, and the spatiotemporal feature matrix passes through the temporal attention unit to obtain weighted temporal data. The weighted spatial data and weighted temporal data are fused as the output of the spatiotemporal attention module; the output of the spatiotemporal attention module is input into the gated neural network module to output the groundwater level prediction result.

[0127] In this embodiment, the spatiotemporal feature matrix of each time window is input into the spatial attention unit. For the data of T time steps Calculate the spatial attention score:

[0128]

[0129] Normalize the spatial attention scores:

[0130]

[0131] Based on the normalized spatial attention score, the spatial data is weighted and summed to obtain the weighted spatial data:

[0132]

[0133] The spatiotemporal feature matrix of each time window is input into the temporal attention unit. For the data of T time steps Calculate the temporal attention score:

[0134] e t，i =W T tanh(W h ,H t +U h h i );

[0135] Normalize the temporal attention scores:

[0136]

[0137] Based on the normalized time attention score, the time data is weighted and summed to obtain the weighted time data:

[0138]

[0139] Among them, H t is the hidden state at the current time step, is the data of the jth spatial position at the tth time step, h i is the hidden state of the historical time step i; W and U are both weight matrices, V T and W T is a weight vector used to project the result of nonlinear activation into the scalar space.

[0140] The weighted spatial data x′ t and the weighted time data h′ t Combined together, they serve as the input to the gated neural network module.

[0141] The gated neural network module of this embodiment includes an update gate (z t ) and reset gate (r t ). Update gate (z t ) is used to control the current state (h t ) needs to be from the previous state (h t-1 ) how much information is retained (without nonlinear transformation), and how much information is obtained from the candidate state (h t ) How much information is received. Reset gate (r t ) controls the candidate state (h t ) depends on the previous state (h t-1 ).

[0142] At time step t, the gated neural network is calculated using the following formula:

[0143] z t=σ(W z [x′ t ; h′ t ]+U z h t-1 +b z );

[0144] r t =σ(W r [x′ t ; h′ t ]+U r h t-1 +b r );

[0145]

[0146] Where W and b are the weight matrix and bias vector of (learnable) parameters; σ is the sigmoid activation function of the update gate and reset gate; tanh is the hyperbolic tangent function. z To update the input weight matrix of the gate, learn the current input x′ t and hidden state h′ t Impact on the update gate. W r To reset the input weight matrix of the gate, learn the current input x′ t and hidden state h′ t Impact on the reset gate. W is the input weight matrix of the candidate hidden state, which is used to calculate the input weight of the candidate hidden state, combined with the current input x′ t and hidden state h′ t Information. z is the hidden state weight matrix of the update gate, which is used to learn the hidden state h at the previous moment t-1 Impact on the update gate. r is the hidden state weight matrix of the reset gate, which is used to learn the hidden state h at the previous moment t-1 The impact on the reset gate. U is the hidden state weight matrix of the candidate hidden state, which is used to calculate the hidden state of the previous moment h t-1 The impact of z is the bias vector of the update gate, which is used to adjust the output of the update gate, b r is the bias vector of the reset gate, which is used to adjust the output of the reset gate, and b is the bias vector of the candidate hidden state, which is used to adjust the output of the candidate hidden state. is the candidate hidden state vector, which is the candidate hidden state at the current time t. It is the "new" state of the hidden state at the previous time after being filtered by the reset gate, and is used to generate the current hidden state h t .

[0147] Finally, a fully connected layer is used to transform the output h of the gated neural network tMapped to the target groundwater level matrix O i , the fully connected layer can be expressed as:

[0148] O i =W O H t +b o ;

[0149] Among them, O i is the target groundwater level matrix, W O is the weight matrix of the fully connected layer, b o is the bias vector of the fully connected layer.

[0150] The gated neural network is trained using the gradient descent method; the setting of the training step size is crucial for the selection of the optimal point. If the step size is set too long, the parameters may move back and forth on both sides of the optimal value point, and it is difficult to converge to an extreme point; if the step size is set too small, more iterations are required to find the optimal point, reducing the optimization speed. In this embodiment, the training step size is determined by improving the step size calculation method, and the specific calculation method is as follows:

[0151]

[0152] Among them, l represents the step size used in each round of optimization, l max and l min They represent the maximum step size and the minimum step size, both of which are set values; p represents the number of iterations, and q represents the number of steps required to reach l. max .

[0153] Through the improved step size calculation method mentioned above, in the early iteration rounds, the step size is larger, and the global optimal area can be quickly found. As the number of iterations increases, the step size becomes smaller, and the local optimal point can be accurately found, while the speed of iterative optimization is improved.

[0154] At the same time, in order to reduce the dependence of the prediction results on the hidden layer nodes, this embodiment randomly sets the neurons in the hidden layer to 0 with a certain probability during each iterative training; before the next iteration, all neurons are reactivated, and in the next iteration, the neurons in the hidden layer are randomly set to 0 with a certain probability; this cycle repeats until the model training is completed. In this way, the gated neural network can be effectively prevented from overfitting, further improving the iterative optimization speed of the model and improving the prediction accuracy.

[0155] After training is completed, this embodiment uses four performance criteria to evaluate the groundwater level prediction model. In addition to the commonly used mean absolute error (MAE), root mean square error (RMSE) and mean absolute percentage error (MAPE), a correlation coefficient (R2) is introduced to evaluate the performance of time series prediction. RMSE reflects the average magnitude of the error, and R2 assigns higher weights to large errors; MAE indicates the size of the error, and MAPE indicates the relative size of the error; R2 is one of the best indicators for measuring linear regression, which reduces the regression results to between 0 and 1, indicating the strength and direction of the linear relationship between the target value and the output value. The above performance criteria are mathematically expressed as follows:

[0156]

[0157]

[0158] in the formula and i They are the predicted value, average value and actual observed value of water level data respectively.

[0159] After training is completed, the test data set is input into the model with the optimal parameters, and the performance evaluation indicators are used to evaluate whether the model performance meets the expected standards, and finally the model is stored.

[0160] In this embodiment, the groundwater level prediction model processes the spatiotemporal feature data by introducing a spatiotemporal attention mechanism (including a spatial attention unit and a temporal attention unit), and performs weighted processing on the input spatiotemporal feature matrix through the spatial attention unit and the temporal attention unit; the spatial attention can capture the mutual influence between different geographical locations, and the temporal attention can capture the mutual dependence between different time points. By considering the dependencies of the spatial and temporal dimensions at the same time, the spatiotemporal attention mechanism enhances the learning ability of the model in complex scenarios; enables the model to focus more on the most useful information for groundwater level prediction, and ignores irrelevant or noise features; effectively reduces the redundant information in the feature space, improves the generalization ability and stability of the model, and helps to improve the prediction accuracy.

[0161] Combining the spatiotemporal attention mechanism with the gated neural network (GRU) can optimize the flow of gradients and reduce the risk of gradient vanishing or gradient explosion during deep network training. It can significantly improve the training efficiency and prediction performance of the model when processing spatiotemporal series data with strong long-term dependencies. Through the visualization of attention weights, we can intuitively understand which time points or spatial locations have an important impact on groundwater level prediction. This helps to gain a deeper understanding of the driving factors and spatiotemporal dynamics of groundwater level changes and formulate more reasonable water resources management policies.

[0162] In order to demonstrate the importance of extracting the spatiotemporal context of groundwater time series prediction and the effectiveness of the key components of the model, an ablation study was conducted. In the ablation experiment, one level of attention was removed from the groundwater level prediction model (denoted as: MSA-GRU) in turn. The neural network layers and sizes were kept the same in the experiment. The ablation architecture of the groundwater level prediction model is as follows:

[0163] TA-GRU, directly removes spatial attention from MSA-GRU, verifying the importance of extracting spatial correlation;

[0164] SA-GRU, after input data structure conversion, removes temporal attention from MSA-GRU and detects the impact of temporal attention on the model framework;

[0165] NA-GRU, directly removes the attention mechanism from MSA-GRU.

[0166] The results are shown in Table 1 and Figure 5 As shown in the figure, compared with other single attention and no attention models, the MSA-GRU model of this embodiment achieves the best model performance (highlighted in bold). In addition, the attention mechanism can improve the performance of the model. The MAE, MAPE and RMSE of TA-GRU and SA-GRU are lower than those of NA-GRU, and the R2 is higher than that of NA-GRU.

[0167] Table 1 Model performance indicators

[0168]

[0169] In summary, in the scientific time series of groundwater systems, it is necessary to extract the spatiotemporal context between variables, and spatiotemporal attention effectively extracts the spatiotemporal context between variables.

[0170] S104: Obtain the spatiotemporal sequence data of the historical groundwater level at the location to be measured, filter and remove abnormal data points; construct a spatiotemporal feature matrix of the groundwater level, and use the trained groundwater level prediction model to predict the groundwater level at the location to be measured.

[0171] Specifically, the spatiotemporal series data of the historical groundwater level at the location to be measured is obtained, and the method in S101 is used to filter and eliminate abnormal data points for the acquired water level data.

[0172] After the data screening is completed, the spatiotemporal feature matrix of the groundwater level is constructed using the same method as above, and the spatiotemporal feature matrix is ​​input into the trained groundwater level prediction model to obtain the prediction result of the groundwater level at the location to be tested.

[0173] When predicting groundwater levels, in order to predict the groundwater level in a certain period of time in the future (such as next month), it is necessary to construct a spatiotemporal feature matrix and input it into the trained groundwater level prediction model. Specifically, the historical data that need to be obtained include historical groundwater level data and spatial geographic location information of the location to be measured and its surrounding spatial locations. These data are usually time series data, which are groundwater level values ​​recorded at multiple times. The data can be recorded monthly, weekly, or even daily. In this case, groundwater level data from the past 12 months were used to predict the groundwater level next month. Using these data to construct a spatiotemporal feature matrix, the predicted groundwater level value for a certain period of time in the future (such as next month) can be obtained through the groundwater level prediction model; Figure 4 A schematic diagram comparing the predicted value of the groundwater level obtained by using the groundwater level prediction model of this embodiment with the true value is given. It can be seen that the deviation between the predicted value and the true value is not large, and accurate prediction of the groundwater level can be achieved.

[0174] The method of this embodiment simultaneously extracts the features of the original sequence data from different spatial and temporal dimensions, processes the spatiotemporal sequence data through the spatiotemporal attention module, gradually extracts the response mechanism of the prediction variables to the spatiotemporal evolution, alleviates the negative impact of non-prediction variables, and solves the spatiotemporal information conflict to a certain extent.

[0175] Embodiment 2

[0176] In one or more embodiments, a groundwater level prediction system based on a spatiotemporal attention gated neural network is disclosed, specifically comprising:

[0177] Training data acquisition module, used to construct historical groundwater level spatiotemporal series dataset;

[0178] The data dimension reduction module is used to perform principal component analysis on the spatiotemporal series data set to obtain the spatiotemporal series data set after dimension reduction;

[0179] The model training module is used to construct multiple spatiotemporal feature matrices for the spatiotemporal series data set after dimensionality reduction through the sliding window method; the groundwater level prediction model is trained using the multiple spatiotemporal feature matrices to obtain a trained groundwater level prediction model;

[0180] The groundwater level prediction module is used to obtain the spatiotemporal sequence data of the historical groundwater level at the location to be measured, construct the spatiotemporal characteristic matrix of the groundwater level, and use the trained groundwater level prediction model to predict the groundwater level at the location to be measured;

[0181] Among them, the groundwater level prediction model includes a spatiotemporal attention module and a gated neural network module. The spatiotemporal attention module includes a spatial attention unit and a temporal attention unit; the spatiotemporal feature matrix obtains weighted spatial data through the spatial attention unit, and the spatiotemporal feature matrix obtains weighted temporal data through the temporal attention unit. The weighted spatial data and weighted temporal data are fused as the output of the spatiotemporal attention module; the output of the spatiotemporal attention module is input into the gated neural network module to output the groundwater level prediction result.

[0182] The specific implementation of the above modules is the same as that in Example 1 and will not be described in detail.

[0183] Embodiment 3

[0184] In one or more embodiments, a terminal device is disclosed, including a server, the server including a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the groundwater level prediction method based on spatiotemporal attention gated neural network in Example 1 when executing the program. For the sake of brevity, the specific process is not repeated.

[0185] It should be understood that in this embodiment, the processor may be a central processing unit CPU, and the processor may also be other general-purpose processors, digital signal processors DSP, application-specific integrated circuits ASIC, off-the-shelf programmable gate arrays FPGA or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0186] The memory may include a read-only memory and a random access memory, and provide instructions and data to the processor. A portion of the memory may also include a non-volatile random access memory. For example, the memory may also store information about the device type.

[0187] In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in a processor or an instruction in the form of software.

[0188] Although the above describes the specific implementation mode of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.

Claims

1. A groundwater level prediction method based on spatiotemporal attention gated neural network, characterized in that: include: Construct a historical groundwater level spatiotemporal series dataset; Perform principal component analysis on the spatiotemporal series data set to obtain the spatiotemporal series data set after dimensionality reduction; For the space-time series data set after dimensionality reduction, multiple space-time feature matrices are constructed by the sliding window method; the groundwater level prediction model is trained using multiple space-time feature matrices to obtain a trained groundwater level prediction model; Obtain the spatiotemporal sequence data of the historical groundwater level at the location to be measured, construct the spatiotemporal characteristic matrix of the groundwater level, and use the trained groundwater level prediction model to predict the groundwater level at the location to be measured; Among them, the groundwater level prediction model includes a spatiotemporal attention module and a gated neural network module. The spatiotemporal attention module includes a spatial attention unit and a temporal attention unit; the spatiotemporal feature matrix obtains weighted spatial data through the spatial attention unit, and the spatiotemporal feature matrix obtains weighted temporal data through the temporal attention unit. The weighted spatial data and weighted temporal data are fused as the output of the spatiotemporal attention module; the output of the spatiotemporal attention module is input into the gated neural network module to output the groundwater level prediction result.

2. A method for predicting groundwater level based on spatiotemporal attention gated neural network as claimed in claim 1, characterized in that: Construct a historical groundwater level spatiotemporal series dataset, specifically: construct a groundwater flow simulation model to calculate the spatiotemporal series dataset of groundwater levels at multiple locations under a changing environment; The groundwater flow simulation model is specifically as follows: Where h is the height of the groundwater level, Kx, Ky, Kz are the hydraulic conductivity coefficients in the x, y, and z directions respectively, Ss is the specific storage rate, t is the time, and W is the source-sink term, which represents the amount of water flowing into or out of the system.

3. A method for predicting groundwater level based on spatiotemporal attention gated neural network as claimed in claim 1, characterized in that: Construct a historical groundwater level spatiotemporal series dataset, specifically: Obtain historical groundwater level information, filter and remove abnormal data points, specifically: Calculate each historical groundwater level data point P(P x1 ,P y1 ) and other data points (P x2 , P y2 ) between ; Calculate the distance d between the kth point closest to the historical groundwater level data point P and point P k (P); take data point P as the center and d k (P) is the radius, and the k-neighborhood of the data point P is obtained; Calculate the reachable distance from data point P to all data points in its k-neighborhood: Assuming point O is a data point in the k-neighborhood of data point P, compare the k-neighborhood radius of point O with the Euclidean distance between point O and point P, and select the larger one as the reachable distance resch_dist from data point P to point O k (P,O); The limit reachable density of data point P is calculated based on the reachable distance, and the outlier factor Limit-LOF(P) is calculated based on the limit reachable density of historical groundwater level data point P; if the outlier factor Limit-LOF(P) is greater than 1, the data point P is determined to be an abnormal data point; After screening out all the abnormal data points of the historical groundwater level, the mean point of each abnormal data point and each normal data point in its k-neighborhood is calculated. After adding the obtained mean point to the historical groundwater level dataset, the outlier factors of these abnormal data points are recalculated. If the obtained outlier factor is still greater than 1, it is regarded as the final abnormal data point and removed. Otherwise, it is retained as a normal data point.

4. A method for predicting groundwater level based on spatiotemporal attention gated neural network as claimed in claim 1, characterized in that: The spatiotemporal series data set includes: corresponding information of time, location and water level.

5. The groundwater level prediction method based on spatiotemporal attention gated neural network according to claim 1, characterized in that: The principal component analysis of the spatiotemporal series data set is performed as follows: Standardize the spatiotemporal series data set, construct the standardized time series data set into a spatiotemporal data matrix, and calculate the covariance matrix of the data matrix; Perform eigenvalue decomposition on the covariance matrix to obtain eigenvalues ​​and eigenvectors; Start accumulating from the largest eigenvalue until the cumulative explained variance ratio reaches or exceeds the set threshold; then the eigenvectors corresponding to these accumulated eigenvalues ​​are used as the main eigenvectors; A dimensionality-reduced data set is formed based on the main eigenvectors.

6. A method for predicting groundwater level based on spatiotemporal attention gated neural network as claimed in claim 1, characterized in that: For the space-time series data set after dimensionality reduction, the space-time feature matrix is ​​constructed by the sliding window method, specifically: Assume that the time step of the sliding window is T; then each time series window includes T groups of M-dimensional spatial data after dimensionality reduction; that is, the spatiotemporal feature matrix Sort by time, and get X1, X2, X3, ..., X k , can be used to predict the target groundwater level matrix O i , i∈(1, 2,…, k), k is the number of sliding windows.

7. A method for predicting groundwater level based on spatiotemporal attention gated neural network as claimed in claim 1, characterized in that: The spatiotemporal feature matrix is ​​converted into weighted spatial data by the spatial attention unit, which is: For each spatial dimension data in each spatiotemporal feature matrix X, calculate the spatial attention score; Normalize the spatial attention scores to obtain the attention weights; Based on the attention weight, the data of each spatial dimension is weighted and summed to obtain the weighted spatial data x′ t .

8. The groundwater level prediction method based on spatiotemporal attention gated neural network according to claim 1, characterized in that: When training the gated neural network, an adaptive compensation algorithm is used to determine the optimization step size, specifically: Among them, l represents the step size used in each round of optimization, l max and l min They represent the maximum step size and the minimum step size, both of which are set values; p represents the number of iterations, and q represents the number of steps required to reach l. max .

9. A groundwater level prediction system based on spatiotemporal attention gated neural network, characterized in that: include: Training data acquisition module, used to construct historical groundwater level spatiotemporal series dataset; The data dimension reduction module is used to perform principal component analysis on the spatiotemporal series data set to obtain the spatiotemporal series data set after dimension reduction; The model training module is used to construct multiple spatiotemporal feature matrices for the spatiotemporal series data set after dimensionality reduction through the sliding window method; the groundwater level prediction model is trained using the multiple spatiotemporal feature matrices to obtain a trained groundwater level prediction model; The groundwater level prediction module is used to obtain the spatiotemporal sequence data of the historical groundwater level at the location to be measured, construct the spatiotemporal characteristic matrix of the groundwater level, and use the trained groundwater level prediction model to predict the groundwater level at the location to be measured; Among them, the groundwater level prediction model includes a spatiotemporal attention module and a gated neural network module. The spatiotemporal attention module includes a spatial attention unit and a temporal attention unit; the spatiotemporal feature matrix obtains weighted spatial data through the spatial attention unit, and the spatiotemporal feature matrix obtains weighted temporal data through the temporal attention unit. The weighted spatial data and weighted temporal data are fused as the output of the spatiotemporal attention module; the output of the spatiotemporal attention module is input into the gated neural network module to output the groundwater level prediction result.

10. A terminal device, comprising a processor and a memory, wherein the processor is used to implement instructions; and the memory is used to store multiple instructions, characterized in that: The instructions are suitable for being loaded by a processor and executing the groundwater level prediction method based on a spatiotemporal attention gated neural network as described in any one of claims 1-8.

Images