A method for predicting daily runoff based on Long Short-Term Component Neural Network (LSTCNet)

Abstract

This invention discloses a daily runoff prediction method based on the Long Short-Term Component Neural Network (LSTCNet) model. LSTCNet leverages the advantages of convolutional layers to extract features. An attention mechanism (AM) is introduced in the long-term component to automatically select relative times across all time steps of the long-term data. The short-term component utilizes a multi-layer residual structure to fuse multi-level features from the short-term data, effectively improving model performance. Furthermore, a traditional AR model is used as a linear neural network component to enhance the learning of linear dependencies between variables and modify the output prediction values. Experimental results show that the proposed method has good performance in daily runoff prediction; the long-term and short-term components can effectively learn from their respective data, demonstrating the strength of the ensemble model.

Description

Technical Field

[0001] This invention relates to the problem of runoff prediction in the field of hydrology, and specifically to a daily runoff prediction method based on Long Short-Term Component Neural Network (LSTCNet). Background Technology

[0002] Accurate prediction of daily runoff is beneficial for the effective allocation of water resources and provides strong guidance for flood control. Due to various factors such as meteorological conditions, watershed surface and mantle, and human activities, runoff processes are characterized by randomness, chaos, and ambiguity. Machine learning, an end-to-end learning method, can accurately predict daily runoff in a watershed without relying on experience or considering physical processes. Since its introduction into the field of hydrology, it has flourished and shown great potential.

[0003] Simple data-driven models struggle to capture the complexities of hydrological systems, prompting researchers to propose Recurrent Neural Networks (RNNs) for learning dependencies between time series data. While RNNs offer significant advantages in time series tasks, they suffer from gradient explosion and vanishing gradient problems after multiple time steps of backpropagation. To address this bottleneck, researchers have proposed two different types of RNNs: Long Short-Term Memory (LSTM) and Gated Recurrent Networks (GRUs). Both summarize cell states across all time steps, overcoming the shortcomings of traditional RNNs and learning long-term dependencies. Furthermore, with the widespread application of attention mechanisms, researchers have begun applying Attention Mechanisms (AMs) to runoff prediction, further improving prediction accuracy.

[0004] However, existing research does not adequately address the differences between long-term and short-term data. While researchers have improved LSTM and GRU networks and proposed various network architectures, these networks use the same structure to extract and learn long-term and short-term features, ignoring the distinctions between them. Therefore, these networks are insufficient to extract highly nonlinear and complex hydrological features, and accurate and reliable daily runoff prediction remains a challenging task for these networks. Summary of the Invention

[0005] The technical problem to be solved by this invention is to provide a daily runoff prediction method based on Long Short-Term Component Neural Network (LSTCNet) to improve the prediction accuracy of daily runoff prediction network models.

[0006] The following technical measures are adopted to solve the above technical problems: a daily runoff prediction method based on Long Short-Term Component Neural Network (LSTCNet), which mainly includes the following steps:

[0007] (1) Using a set of long-term time series historical data and short-term data As input, long-term data is fed into the long-term component, and short-term data is fed into the nonlinear short-term component and the linear component.

[0008] (2) Long-term component for processing long-term data: The long-term component consists of convolutional layers, GRU layers, and attention layers. Long-term data is input into the convolutional layers, which extract features of the temporal distribution and local dependencies between variables. The recurrent component is a recurrent layer with GRU. Supported by the attention mechanism, the GRU layer automatically learns to extract feature values ​​and assigns different weights based on the importance of the features. The specific implementation details of this module are as follows:

[0009] The input matrix is Convolutional layers are composed of The size is The kernel composition, the output matrix vector of the convolutional layer is It can be viewed as a length of of A sequence of dimensional vectors, where , Represents the time step. It is fed into two GRU layers n Each GRU layer contains g The output of the cyclic component is [number] units. The output of the entire long-term component process is .

[0010] (3) Short-term components process short-term data: Short-term components utilize residual structures composed of convolutional layers, i.e., RC layers, to fully leverage multi-level feature information. The residual structure layers can fuse multi-level feature information, thereby improving the comprehensiveness of feature information.

[0011] The structure of the RC layer is as follows Figure 2 As shown, an RC layer contains two The function consists of two convolutional layers. Each convolutional layer contains... One core, with a core size of [missing information]. The first one For the time dimension, the latter The RC layer has a variable dimension. go through The activation function is then input into the convolutional layer to obtain the shallow feature matrix. Then Enter the next one Activation functions and convolutional layers yield a deep feature matrix. Finally, through the input... sum matrix Integrating the residual structure yields the RC output.

[0012] Input matrix Input to contain In the RCs of an RC layer, it can be represented as:

[0013]

[0014] in I The value is determined based on the specific circumstances. and These are the learnable parameters of the previous convolutional layer. and These are the learnable parameters for the next convolutional layer. It is the input of the RC layer. It is the output; if the padding is the same, the output is... It will be a The vector.

[0015] Then, through The activation function will convert the output of RCs. Input another convolutional layer, the convolutional layer consists of The size is The core composition yields the output as It is connected through a fully connected layer (FCL). This refers to the prediction results of short-term data.

[0016] (4) Linear Part for Short-Term Data: The linear part employs an AR model to enhance the ability to capture linear dependencies between variables and corrects the predicted values ​​of the nonlinear part. Let the input matrix be... The AR model is described as follows:

[0017]

[0018] in, This represents the output of the AR model. and For learnable parameters, yes t The input of this layer in terms of time, q It is the input window size, representing the amount of past information required by the model.

[0019] (5) The model's final prediction is derived from the sum of the outputs of the long-term, short-term, and AR components. The FCL (Flexible Chaining) is used to combine the long-term and short-term data outputs of the nonlinear component. The FCL output is:

[0020]

[0021] in and These are learnable parameters in FCL, and relatively speaking, and It is the output of predictions based on long-term and short-term historical data. This represents the nonlinear output of the model. Finally, by integrating the outputs of the nonlinear and linear parts, the final prediction of the model is obtained:

[0022]

[0023] (6) The method of the present invention trains and predicts by minimizing the mean absolute error between the predicted and observed values. The Adam optimizer was used in the model training process and the objective function, with the objective function being:

[0024]

[0025] in, The number of training samples, The dimension of the target data. Attached Figure Description

[0026] Figure 1 This is a block diagram of the Long Short-Term Component Neural Network (LSTCNet) structure proposed in this invention.

[0027] Figure 2 Details of the residual structure (RC layer) composed of convolutional layers.

[0028] Figure 3 The data source for evaluating model performance is the Qingxi River Basin map.

[0029] Figure 4 A visualization comparison of LSTCNet's predicted and observed values ​​for the Qingxi dataset.

[0030] Figure 5 A detailed view comparing the visualization results of LSTCNet predictions and observations. Detailed Implementation

[0031] The present invention will be further described below with reference to the accompanying drawings. It should be noted that the following embodiments are only used to further illustrate the present invention and should not be construed as limiting the scope of protection of the present invention. Any non-essential improvements and adjustments made to the present invention by those skilled in the art based on the above description of the present invention still fall within the scope of protection of the present invention.

[0032] (1) The method proposed in this invention is developed on the Tensorflow deep learning framework. In order to learn dependencies more effectively, the parameter values ​​shown in Table 1 were selected to train the model.

[0033] (2) When verifying the predictive performance of the method proposed in this invention, historical runoff data of the Qingxi River Basin from the Sichuan Provincial Hydrological and Water Resources Bureau were selected. The locations of the basic hydrological station and three rain gauge stations are as follows: Figure 3 As shown, the site information, sequence length of the collected data, and Pearson correlation coefficients (PCCs) are listed in Table 2. Meanwhile, 20 years of data (1986-2005) were used for model calibration. Statistical information for the training (1986-2003) and testing (2004-2005) data, including minimum, maximum, and average values, is shown in Table 3.

[0034] (3) Five commonly used statistical indicators, namely mean absolute error (MAE), root mean square error (RMSE), Nash-Sutcliffe efficiency (NSE), correlation coefficient (CC), and Willmott index (WI), were used to quantitatively evaluate all experimental models. MAE and RMSE were used to evaluate the prediction accuracy and robustness of the proposed method; NSE was used to measure the ability to predict variables that differ from the mean and to give the proportion of the initial variance accounted for by the model; CC represents the strength and direction of the linear relationship between the observed and predicted values.

[0035] (4) The runoff prediction network compared with the method of this invention is:

[0036] Method 1: The method proposed by Yin Z, Feng Q, Wen X, et al., referenced in "Design and evaluation of SVR, MARS and M5Tree models for 1, 2 and 3-day lead time forecasting of river flow data in a semiarid mountainous catchment[C] / / StochEnviron Res Risk Assess.2018:2457–2476";

[0037] Method 2: The method proposed by Qi Y, Zhou Z, Yang L, et al., see reference "A Decomposition-Ensemble Learning Model Based on LSTM Neural Network for Daily ReservoirInflow Forecasting[C] / / Water Resour Manage. 2019: 4123–4139";

[0038] Method 3: The method proposed by Y. Wu, Z. Liu, W. Xu et al., see reference "Context-AwareAttention LSTM Network for Flood Prediction[C] / / International Conference on Pattern Recognition (ICPR). 2018: 1301-1306";

[0039] Method 4: The method proposed by Bi XY, Li B, Lu WL et al., see reference "Daily runoffforecasting based on data-augmented neural network model[J] / / Journal of Hydroinformatics. 2020: 900–915".

[0040] (5) As can be seen from Table 4, compared with MARS, LSTM, AMLSTM and CAGANet runoff prediction networks, the method proposed in this invention has achieved the best performance. Figure 4 The thick solid line at the top represents the actual time series of surface rainfall, while the solid and dashed lines at the bottom represent the actual and predicted values ​​of surface runoff, respectively. Figure 4 Detailed view Figure 5 It can be observed that LSTCNet has good overall prediction performance, with peak predictions close to the observed values ​​at most points, and LSTCNet is better at predicting small-scale floods than large-scale floods.

[0041] (6) To demonstrate the effectiveness of this method framework, some ablation experiments were conducted on the original dataset. The experimental results are shown in Tables 5, 6, and 7. Table 5 shows the indispensable role of all components of LSTCNet, Table 6 shows the effectiveness of the Long+Short (i.e., using long-term components to process long-term data and short-term components to process short-term data) combination scheme, and Table 7 shows that the multi-level information generated by stacked RCs can improve accuracy, and the optimal number of RC layers is 3.

[0042] Table 1 Parameter Selection

[0043]

[0044] Table 2 Information from Qingxi Hydrological Station

[0045]

[0046] Table 3 Qingxi Dataset Information

[0047]

[0048] Table 4 Comparison results of different methods on the Qingxi dataset

[0049]

[0050] Table 5 Comparison results after removing different components

[0051]

[0052] Table 6. Comparison of different combinations of components for processing long-term and short-term data.

[0053]

[0054] Table 7. Comparison of prediction results using different amounts of RC

[0055]

Claims

1. A long short-term memory component-based neural network-based daily runoff prediction method, characterized by The method comprises the following steps: (1) a set of long-term historical daily runoff observation data and a set of short-term historical daily runoff observation data As input, the long-term historical daily runoff observation data is input to the long-term component, and the short-term historical daily runoff observation data is input to the nonlinear short-term component and the linear component; (2) The long-term component processes the long-term historical observation data of daily runoff: the long-term component is composed of a convolution layer, a GRU layer and an attention layer; the long-term historical observation data of daily runoff is input into the convolution layer, which is used to extract the time distribution features and the local dependence relationship between variables; the GRU layer and the attention layer are combined, which are used to automatically learn and extract features, and different weights are assigned according to the importance of the features; the specific implementation details of the long-term component for processing the long-term historical observation data of daily runoff are as follows: Input is denoted as , the convolutional layer consists of kernels with size , the output matrix vector of the convolutional layer is is a sequence of dimensional vectors with length , where , represents the time step; is input into two GRU layers n times, each GRU layer contains g units, and the output of the recurrent component is , and the output of the entire long-term component process is ; (3) The short-term component processes the short-term historical observation data of daily runoff: the short-term component uses a residual structure RC layer composed of a convolution layer to fully utilize multi-level feature information, so as to realize the fusion of multi-level feature information; the specific implementation details of the short-term component for processing the short-term historical observation data of daily runoff are as follows: An RC layer contains two The function and two convolutional layers; each convolutional layer contains One core, with a core size of [missing information]. The first one For the time dimension, the latter Variable dimension; input of RC layer go through The activation function is then fed into the convolutional layer to obtain the shallow feature matrix. Then Enter the next one Activation functions and convolutional layers yield a deep feature matrix. Finally, by analyzing the input and deep feature matrix Add them together to get the output of RC; if there is If there are RC layers, then the final output of RCs is: wherein the value of I is determined according to the specific situation, and are the learnable parameters of the previous convolutional layer, and are the learnable parameters of the subsequent convolutional layer, and then the output of RCs is passed through an activation function to obtain as the output of the convolutional layer, which is input into another convolutional layer composed of kernels with a size of to obtain an output of , which is connected through a fully connected layer FCL to obtain , which is the prediction result of the short-term component. (4) Linear component processes daily runoff short-term historical observation data: the linear component uses the autoregressive model, that is, the AR model, to enhance the ability to capture the linear dependence between variables and correct the predicted value of the nonlinear part; let the input matrix be The AR model is expressed as follows: wherein, denotes the output of the AR model, and are learnable parameters, is the input of the layer at time t, q is the input window size, denotes the amount of past information required by the model; (5) The final prediction of the model comes from the sum of the outputs of the long-term component, the short-term component and the AR component; the FCL is used to combine the outputs of the nonlinear long-term component and the short-term component; the output of the FCL is: wherein and are learnable parameters in FCL, and are the outputs of long-term component and short-term component prediction, respectively, denotes the nonlinear output of the model, and finally the final daily runoff prediction value of the model is obtained by adding the outputs of the nonlinear part and the linear part: , which is suitable for hydrological regulation and flow prediction.

2. The long-short-term-memory component-based neural network based daily runoff prediction method of claim 1, wherein An attention mechanism is introduced to automatically select the relative time in all time steps of the long-term historical observation data of daily runoff.

Images