Bidirectional long-short-term memory recurrent neural network-based runoff prediction method

Abstract

The invention discloses a runoff prediction method based on a bidirectional long-short-term memory recurrent neural network, and belongs to the technical field of runoff prediction. The method comprises the following steps: preprocessing original data to obtain a data set, and dividing the data set into a training set and a verification set; constructing a runoff prediction network model, wherein the runoff prediction network model comprises a forward LSTM module, a reverse LSTM module, an Adam optimizer, a Dropout regularization layer and a Dense full connection layer which are connected in sequence; setting a hyper-parameter and a loss function, and training the constructed model by adopting the training set to obtain a trained runoff prediction model; and inputting verification set data into the trained runoff prediction model to obtain predicted runoff. According to the method, the association between the past and future runoff time series data can be mined, the problem of poor feature extraction and memory effect of early learning part data caused by unidirectional learning is effectively avoided, and the prediction precision is improved while better applicability is achieved.

Description

Technical Field

[0001] The invention relates to the technical field of runoff prediction, and in particular to a runoff prediction method based on a bidirectional long short-term memory recurrent neural network. Background Art

[0002] River runoff, as an important component of the hydrological system, has always attracted much attention. River water conditions are related to regional economic development and the development and utilization of water resources. Therefore, efficient and accurate river runoff prediction is of great practical significance.

[0003] Scholar Chen Liang proposed a long short-term memory network method (LSTM) based on the encoder-decoder framework to predict the water level of Dongting Lake, but this method does not have the ability to learn information in future data. Scholar Shaolei G et al. proposed a method combining complete ensemble empirical mode decomposition (CEEMDAN), least squares support vector machine (LSSVM) and gray model (GM(1,1)) to achieve river runoff, but this method decomposes more signal components, which increases the model calculation amount.

[0004] Patent document No. CN114219131A uses the LSTM model to establish a mapping relationship between input and output to compare the performance of LSTM runoff prediction under different learning schemes, and finally selects the optimal learning scheme for basin runoff prediction performance. However, using the LSTM model alone will introduce noise irrelevant to runoff prediction, and will also be affected by large and small values ​​in time series data, resulting in low prediction accuracy and large prediction uncertainty. Patent document No. CN110490366A proposes a runoff prediction method based on variational mode decomposition and iterative decision tree, which divides the runoff process into different sub-modes, and uses iterative decision trees to train each sub-mode to obtain a prediction model for runoff flow. However, iterative decision trees are prone to overfitting, which makes the model's generalization ability very low, its stability is low, and it is easy to produce different trees.

[0005] In summary, the existing runoff prediction methods have the following problems: first, they do not consider time series data or only predict runoff based on the previous time series, without fully considering the connection between the characteristics before and after the prediction data; second, the traditional model has large computational complexity, high complexity, slow model response, and is prone to data overfitting. Summary of the invention

[0006] In view of this, the present invention aims at the deficiencies in the prior art and provides a runoff prediction method based on a bidirectional long short-term memory recurrent neural network.

[0007] In order to solve the above technical problems, the technical solution adopted by the present invention is: a runoff prediction method based on a bidirectional long short-term memory recurrent neural network, comprising the following steps:

[0008] S1. Collect runoff data of multiple adjacent hydropower stations;

[0009] S2, data preprocessing, using deviation standardization method to normalize the collected data and divide it into training set and test set;

[0010] S3. Build a BiLSTM deep learning prediction model, including a forward LSTM channel and a reverse LSTM channel;

[0011] S4. Set network structure parameters and hyperparameters;

[0012] S5, model training, first initialize the weight of each layer of the BiLSTM deep learning prediction model constructed in step S3 according to the initialization method, then input the initial runoff data, first perform feature extraction of the bidirectional LSTM channel on the data of each node, calculate the loss of the output data obtained by the bidirectional calculation according to the loss function and the true label, calculate the gradient back propagation to update the weight of each layer, use the adam optimizer to fit the model through multiple iterations of gradient descent to achieve the optimal, then add Dropout regularization to prevent data overfitting, and finally input the output multi-dimensional latent vector features into the Dense fully connected layer for dimensionality reduction to obtain the final runoff data, and obtain the trained BiLSTM deep learning prediction model;

[0013] S6. Input the test set data into the trained BiLSTM deep learning prediction model to obtain the predicted runoff volume, and evaluate the prediction results based on the real data.

[0014] Furthermore, in S1, the runoff data includes daily rainfall, daily water surface evaporation, and daily flow of multiple hydropower stations.

[0015] Furthermore, in S2, the formula for normalization is as follows:

[0016] Where: x min is the minimum value of the sample data, x max is the maximum value of the sample data.

[0017] Furthermore, in S3,

[0018] The forward LSTM channel is used to predict the future trend of runoff data, and its output expression is:

[0019]

[0020] The reverse LSTM is used to smooth the forward runoff prediction results, and its output expression is:

[0021]

[0022] The output of each network path is fed back to the same series layer and then sent to the next layer through the transmission path. The output expression at time t is:

[0023]

[0024] Where: F LSTM , B LSTM are the forward and reverse LSTM network unit structures, W F , W B are the forward and backward LSTM network weights, b h is the bias factor of the output layer;

[0025] The BiLSTM network channel is composed of LSTM units in series. A single LSTM consists of a cell structure and a gate control unit, where the gate control unit is divided into an input gate, a forget gate, and an output gate.

[0026] The function of the forget gate is to filter out the useful information from the previous cell state, and output the daily runoff forecast value of the hydropower station in the hidden layer of the previous LSTM unit h t―1 and the current runoff characteristic input x t , as the input of the forget gate, the threshold output is obtained after passing through the forget gate σ. The forget gate expression is:

[0027] F t =σ(w f ·[h t―1 ,x t ]+b f )

[0028] F t ∈[0,1]

[0029] The input gate determines what information is stored in the cell state. If it is 1, the input is allowed, and if it is 0, it is not allowed. The output of the input gate is I t and I t Determine the value that needs to be updated, is the newly learned memory, combined with the forgetting factor, the decayed memory C t , the input gate expression is:

[0030] I t =σ(w i ·[h t―1 ,x t ]+b i)

[0031]

[0032] The output gate is controlled by the σ layer and the tanh layer according to the previous input information, and outputs the data information h t , the output gate expression is:

[0033] O t =σ(w o ·[h t―1 ,x t ]+b o )

[0034] h t =O t tanh(C t )

[0035] In the above formula: w f 、w i 、w o is the connection weight value of each gating unit, σ is the sigmoid activation function, and tanh is the hyperbolic tangent activation function, and its expression is:

[0036]

[0037] Further, in S5, the loss function uses mean square error, and the loss function formula is:

[0038]

[0039] Where: x t is the input runoff data, Output runoff data.

[0040] Furthermore, in S5, the Adam optimizer completes parameter optimization through gradient transformation, and the gradient mean m t and the squared gradient v t Perform adaptive parameter update adjustment, the network model update parameter expression is θ t :

[0041] m t =β1m t―1 +(1―β1)g t

[0042]

[0043] Where: β1, β2 are introduced hyperparameters, α is the default learning rate, ε is the default adjustment value, g t is the current gradient.

[0044] Furthermore, in S5, the Dropout regularization formula is:

[0045]

[0046] In the formula: r represents the network loss rate, which obeys the Bernoulli distribution, l represents the network layer, x represents the input vector, h represents the output vector, w represents the bias, b represents the bias, and f represents the activation function.

[0047] Furthermore, in S5, the Dense fully connected layer is an ordinary fully connected feedforward neural network, which is used to convert the multi-dimensional state vector of BiLSTM into the runoff at each moment, so as to reduce the dimension of the output result.

[0048] Compared with the prior art, the present invention has the following beneficial effects:

[0049] The present invention predicts the runoff according to the constructed and trained BiLSTM deep learning prediction model. The bidirectional input structure can be used to learn the data sequence characteristics from both positive and negative directions, and recursion and feedback are performed on it to mine the past and future connections between the daily runoff time series data. Not only the runoff characteristic information at past moments is considered, but also the runoff data at future moments is learned, which can improve the model prediction accuracy. In addition, the method of the present invention has simple steps, convenient implementation and simple operation, good use effect, high model accuracy, fast calculation speed, and can effectively solve the problem of low prediction accuracy of existing runoff prediction methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 is a schematic diagram of a flow chart of an embodiment of the present invention;

[0051] Figure 2 It is a structural diagram of a single LSTM unit in an embodiment of the present invention;

[0052] Figure 3 This is a structural diagram of the BiLSTM deep learning prediction model in an embodiment of the present invention;

[0053] Figure 4 is a training set fitting curve diagram in an embodiment of the present invention;

[0054] Figure 5 It is a test set fitting curve diagram in an embodiment of the present invention. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solution and advantages of the embodiment of the present invention clearer, the technical solution of the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings of the embodiment of the present invention. Obviously, the described embodiment is a part of the embodiment of the present invention, not all of the embodiments. Based on the described embodiment of the present invention, all other embodiments obtained by ordinary technicians in this field belong to the scope of protection of the present invention.

[0056] Example

[0057] Reference Figures 1 to 3 The runoff prediction method based on a bidirectional long short-term memory recurrent neural network in the embodiment of the present invention comprises the following steps:

[0058] S1. Collect runoff data of multiple adjacent hydropower stations. Taking Taojiang River Basin as an example, the collected data include daily rainfall at Shaoyang Station, daily rainfall at Lengshui River Station, daily rainfall at Taojiang Station, daily water surface evaporation at Lengshui River Station, daily water surface evaporation at Taojiang Station, daily flow at Shaoyang Station, and daily flow at Lengshui River Station;

[0059] S2. Data preprocessing. Since the order of magnitude of input data is sometimes quite different, the collected data is normalized using the deviation standardization method and divided into a training set and a test set with a ratio of 7:3. The normalization formula is:

[0060]

[0061] Where: x min is the minimum value of the sample data, x max is the maximum value of the sample data.

[0062] S3. Build a BiLSTM deep learning prediction model, including a forward LSTM channel and a reverse LSTM channel. The forward LSTM channel is used to predict the future trend of runoff data. Its output expression is:

[0063]

[0064] The reverse LSTM is used to smooth the forward runoff prediction results, and its output expression is:

[0065]

[0066] The output of each network path is fed back to the same series layer and then sent to the next layer through the transmission path. The output expression at time t is:

[0067]

[0068] Where: F LSTM , B LSTMare the forward and reverse LSTM network unit structures, W F , W B are the forward and backward LSTM network weights, b h is the bias factor of the output layer;

[0069] The BiLSTM network channel is composed of LSTM units in series. A single LSTM consists of a cell structure and a gate control unit, where the gate control unit is divided into an input gate, a forget gate, and an output gate.

[0070] The function of the forget gate is to filter out the useful information from the previous cell state, and output the daily runoff forecast value of the hydropower station in the hidden layer of the previous LSTM unit h t―1 and the current runoff characteristic input x t , as the input of the forget gate, the threshold output is obtained after passing through the forget gate σ. The forget gate expression is:

[0071] F t =σ(w f ·[h t―1 ,x t ]+b f )

[0072] F t ∈[0,1]

[0073] The input gate determines what information is stored in the cell state. If it is 1, the input is allowed, and if it is 0, it is not allowed. The output of the input gate is I t and I t Determine the value that needs to be updated, is the newly learned memory, combined with the forgetting factor, the decayed memory C t , the input gate expression is:

[0074] I t =σ(w i ·[h t―1 ,x t ]+b i )

[0075]

[0076] The output gate is controlled by the σ layer and the tanh layer according to the previous input information, and outputs the data information h t , the output gate expression is:

[0077] O t =σ(w o ·[h t―1 ,x t ]+b o )

[0078] h t =O t tanh(C t )

[0079] In the above formula: w f 、w i 、w o is the connection weight value of each gating unit, σ is the sigmoid activation function, and tanh is the hyperbolic tangent activation function, and its expression is:

[0080]

[0081] S4. Set the network structure parameters and hyperparameters, as shown in Table 1 and Table 2 below:

[0082] Table 1 Network structure parameters

[0083] parameter Value Number of units in the forward hidden layer [5,30,50,5] Number of reverse hidden layer units [5,50,30,5]

[0084] Table 2 Hyperparameters

[0085]

[0086]

[0087] S5, model training, first initialize the weight of each layer of the BiLSTM deep learning prediction model constructed in step S3 according to the initialization method, then input the initial runoff data of the training set, first perform feature extraction on the data of each node through the bidirectional LSTM channel, calculate the loss of the output data obtained by the bidirectional calculation according to the loss function and the true label, calculate the gradient back propagation to update the weight of each layer, use the adam optimizer to fit the model through multiple iterations of gradient descent to reach the optimal, then add Dropout regularization to prevent data overfitting, finally input the output multi-dimensional latent vector features into the Dense fully connected layer for dimensionality reduction, obtain the final runoff data, and obtain the trained BiLSTM deep learning prediction model;

[0088] The loss function uses the mean square error, and the formula is:

[0089]

[0090] Where: x t is the input runoff data, Output runoff data.

[0091] The Adam optimization method has an adaptive adjustment effect. The Adam optimizer completes parameter optimization through gradient transformation, and its implementation process is simple and efficient. When updating parameters, it is not affected by the scaling transformation of the gradient.t and the squared gradient v t Perform adaptive parameter update adjustment, the network model update parameter expression is θ t :

[0092] m t =β1m t―1 +(1―β1)g t

[0093]

[0094]

[0095] Where: β1, β2 are introduced hyperparameters, α is the default learning rate, ε is the default adjustment value, g t is the current gradient.

[0096] Dropout allows some network nodes to be randomly disabled during model training, which makes each hidden unit layer more robust, reduces dependence on other hidden units, and reduces the complex adaptation relationship between neurons. The Dropout regularization formula is:

[0097]

[0098] In the formula: r represents the network loss rate, which obeys the Bernoulli distribution, l represents the network layer, x represents the input vector, h represents the output vector, w represents the bias, b represents the bias, and f represents the activation function.

[0099] The Dense fully connected layer is an ordinary fully connected feedforward neural network, which is used to convert the multi-dimensional state vector of BiLSTM into the runoff at each moment and to reduce the dimension of the output result. That is, the daily runoff of Taojiang River is used as the output, and the required 1-dimensional dimension can be output through the Dense fully connected layer.

[0100] S6. Input the test set data into the trained BiLSTM deep learning prediction model to obtain the predicted runoff volume, and evaluate the prediction results based on the real data.

[0101] The root mean square error (RMSE) and mean absolute error (MAE) are used to evaluate the prediction accuracy. RMSE represents the degree of error dispersion, and MAE represents the absolute mean of the deviation between the predicted value and the observed value. The smaller the values ​​of the two evaluation indicators, the higher the prediction accuracy of the model.

[0102]

[0103] In the formula, y t is the test data at time t, is the predicted data at time t, and N is the length of the test set.

[0104] according to Figure 4 It can be clearly seen that the BP neural network has large fluctuations during the training process and is prone to large peaks. Due to the introduction of a bidirectional network channel, the overall training results of BiLSTM are more accurate and stable than those of LSTM.

[0105] according to Figure 5 After iterative training of the training set data, the training effect is tested through the test set data, and the test set data is predicted. The prediction effect of the BP neural network is poor, with more mutation values. The LSTM and BiLSTM fitting effects are better, but when fitting some data with large changes and fluctuations, LSTM cannot correctly predict the data, resulting in deviations in the results. BiLSTM maintains a more accurate prediction performance on the overall data, while having high efficiency and accuracy.

[0106] In summary, the runoff prediction result of the BiLSTM model proposed in the present invention is accurate and reliable, and can achieve good runoff prediction effect.

[0107] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A runoff prediction method based on a bidirectional long short-term memory recurrent neural network is characterized in that: The following steps are involved: S1. Collect runoff data of multiple adjacent hydropower stations; S2, data preprocessing, using deviation standardization method to normalize the collected data and divide it into training set and test set; S3. Build a BiLSTM deep learning prediction model, including a forward LSTM channel and a reverse LSTM channel; S4. Set network structure parameters and hyperparameters; S5, model training, first initialize the weight of each layer of the BiLSTM deep learning prediction model constructed in step S3 according to the initialization method, then input the initial runoff data, first perform feature extraction of the bidirectional LSTM channel on the data of each node, calculate the loss of the output data obtained by the bidirectional calculation according to the loss function and the true label, calculate the gradient back propagation to update the weight of each layer, use the adam optimizer to fit the model through multiple iterations of gradient descent to achieve the optimal, then add Dropout regularization to prevent data overfitting, and finally input the output multi-dimensional latent vector features into the Dense fully connected layer for dimensionality reduction to obtain the final runoff data, and obtain the trained BiLSTM deep learning prediction model; S6. Input the test set data into the trained BiLSTM deep learning prediction model to obtain the predicted runoff volume, and evaluate the prediction results based on the real data.

2. The runoff prediction method based on bidirectional long short-term memory recurrent neural network according to claim 1 is characterized in that: In S1, the runoff data includes daily rainfall, daily water surface evaporation, and daily flow of multiple hydropower stations.

3. The runoff prediction method based on bidirectional long short-term memory recurrent neural network according to claim 1 is characterized in that: In S2, the normalization formula is as follows: Where: x min is the minimum value of the sample data, x max is the maximum value of the sample data.

4. The method for predicting runoff based on a bidirectional long short-term memory recurrent neural network according to claim 1 is characterized in that: In S3, The forward LSTM channel is used to predict the future trend of runoff data, and its output expression is: The reverse LSTM is used to smooth the forward runoff prediction results, and its output expression is: The output of each network path is fed back to the same series layer and then sent to the next layer through the transmission path. The output expression at time t is: Where: F LSTM , B LSTM are the forward and reverse LSTM network unit structures, W F , W B are the forward and backward LSTM network weights, b h is the bias factor of the output layer; The BiLSTM network channel is composed of LSTM units in series. A single LSTM consists of a cell structure and a gate control unit, where the gate control unit is divided into an input gate, a forget gate, and an output gate. The function of the forget gate is to filter out the useful information from the previous cell state, and output the daily runoff forecast value of the hydropower station in the hidden layer of the previous LSTM unit h t―1 and the current runoff characteristic input x t , as the input of the forget gate, the threshold output is obtained after passing through the forget gate σ. The forget gate expression is: F t =σ(w f ·[h t―1 ,x t ]+b f ) F t ∈[0,1] The input gate determines what information is stored in the cell state. If it is 1, the input is allowed, and if it is 0, it is not allowed. The output of the input gate is I t and I t Determine the value that needs to be updated, is the newly learned memory, combined with the forgetting factor, the decayed memory C t , the input gate expression is: I t =σ(w i ·[h t―1 ,x t ]+b i ) The output gate is controlled by the σ layer and the tanh layer according to the previous input information, and outputs the data information h t , the output gate expression is: The t =σ(w o ·[h t―1 ,x t ]+b o ) h t =O t ·tanh(C t ) In the above formula: w f 、w i 、w o is the connection weight value of each gating unit, σ is the sigmoid activation function, and tanh is the hyperbolic tangent activation function, and its expression is:

5. The method for predicting runoff based on a bidirectional long short-term memory recurrent neural network according to claim 4 is characterized in that: In S5, the loss function uses mean square error, and the loss function formula is: Where: x t is the input runoff data, Output runoff data.

6. The runoff prediction method based on bidirectional long short-term memory recurrent neural network according to claim 1 is characterized in that: In S5, the Adam optimizer completes parameter optimization through gradient transformation and uses the gradient mean m t and the squared gradient v t Perform adaptive parameter update adjustment, the network model update parameter expression is θ t : m t =β1m t―1 +(1―β1)g t Where: β1, β2 are introduced hyperparameters, α is the default learning rate, ε is the default adjustment value, g t is the current gradient.

7. The method for predicting runoff based on a bidirectional long short-term memory recurrent neural network according to claim 1, characterized in that: In S5, the Dropout regularization formula is: In the formula: r represents the network loss rate, which obeys the Bernoulli distribution, l represents the network layer, x represents the input vector, h represents the output vector, w represents the bias, b represents the bias, and f represents the activation function.

8. The method for predicting runoff based on a bidirectional long short-term memory recurrent neural network according to claim 1, characterized in that: In S5, the Dense fully connected layer is an ordinary fully connected feedforward neural network, which is used to convert the multi-dimensional state vector of BiLSTM into the runoff at each moment, so as to reduce the dimension of the output result.

Images