An interpretable multi-scale LSTM daily runoff prediction method

Abstract

The invention discloses an interpretable multi-scale LSTM daily runoff prediction method, comprising the following steps: collecting daily runoff data and candidate prediction factors thereof, preprocessing the collected data, and dividing the processed data into a training set and a test set; decomposing the processed data into detail components and approximate components by àtrous wavelet decomposition; selecting significant prediction factors from candidate prediction factor components for each daily runoff component by a Boruta algorithm; constructing an LSTM model, taking the significant prediction factors as model input, training the LSTM model of each daily runoff component, and obtaining a daily runoff component prediction model; testing the daily runoff components in a test period by the daily runoff component prediction model to obtain prediction results of each component, reconstructing the prediction results, obtaining a final daily runoff prediction value, and evaluating the prediction performance; and calculating the influence of each significant prediction factor on the prediction result by an LRP algorithm.

Description

Technical Field

[0001] The present invention belongs to the technical field of hydrological data prediction, and specifically relates to an interpretable multi-scale LSTM daily runoff prediction method. Background Art

[0002] Reliable and accurate runoff prediction is essential for flood control, disaster management, agricultural irrigation, and water resources planning and management. Existing runoff prediction methods can be divided into process-driven models and data-driven models. Process-driven runoff prediction models rely on the physical process of runoff formation and can provide insights into hydrological mechanisms. However, such models have high data requirements and require many simplifying assumptions, which brings great difficulties to calibration and verification. Data-driven runoff prediction models do not require clear physical mechanisms, have low demand for data information, and have a fast development time. They have been widely used. With the development of artificial intelligence and computer technology, deep learning models have also shown great potential in runoff prediction. The recurrent neural network (RNN) model is a neural network model specially designed to process time series data. The model takes into account the sequence information of the time series, can memorize previous information, and capture temporal dynamic characteristics. However, the main weakness of traditional RNNs in training is the lack of long-term memory. The extension of training time and the increase of network layers will lead to gradient explosion and gradient vanishing problems. The Long Short-Term Memory Network (LSTM) is an improved RNN that regulates the inflow and outflow of information in storage units through gating units, which can effectively overcome the problems of gradient explosion and gradient disappearance in traditional RNN.

[0003] Although deep learning models can effectively improve the accuracy of runoff prediction, due to their unique decision-making methods and "black box" nature, it is impossible to track the relative importance of input variables in their final predictions, making it difficult to understand their complex prediction process, which has caused widespread doubts among experts about their prediction effectiveness. Hierarchical Relevance Propagation (LRP) is a technology specifically designed to provide interpretability for highly complex deep neural networks. It uses the importance scores of input features to explain neural network predictions, effectively alleviating the "black box" problem of deep learning models. In addition, runoff is affected by many factors, resulting in its nonlinear and non-stationary nature. Directly predicting the nonlinear and non-stationary original time series of runoff often leads to low prediction accuracy. Introducing signal processing methods such as àtrous wavelet decomposition into the prediction model can effectively reduce the non-stationary characteristics of time series data and improve the prediction accuracy of the model. At the same time, using the Boruta factor selection method to select significant predictors from a large number of candidate predictors is also one of the methods to improve the prediction accuracy of the model. Summary of the invention

[0004] The present invention aims to address the deficiencies of the prior art and constructs an interpretable multi-scale LSTM daily runoff prediction method by effectively coupling àtrous wavelet decomposition, Boruta algorithm, LRP algorithm and LSTM model, which improves the accuracy of daily runoff prediction and enhances the interpretability of the prediction results.

[0005] To achieve the above objectives, the following solutions are provided:

[0006] An interpretable multi-scale LSTM daily runoff prediction method includes the following steps:

[0007] Collecting daily runoff data and candidate prediction factors thereof, preprocessing the collected data, and dividing the processed data into a training set and a test set;

[0008] Decomposing the processed data into detail components and approximate components at different scales;

[0009] For each daily runoff component, selecting a significant predictor from the components of the candidate predictors;

[0010] Constructing an LSTM model, taking the significant predictor as a model input, training the LSTM model of each of the daily runoff components, and obtaining a daily runoff component prediction model;

[0011] The daily runoff prediction model is used to test the daily runoff detail component and the approximate component during the test period to obtain prediction results of each component, the prediction results are reconstructed to obtain the final daily runoff prediction value and to evaluate the prediction performance;

[0012] The LRP algorithm is used to calculate the influence of each significant predictor on the prediction result.

[0013] Preferably, the candidate prediction factors include: precipitation, temperature, evaporation, sunshine hours, air pressure and relative humidity.

[0014] The preprocessing method includes: outlier processing, null value filling and standardization.

[0015] The standardized calculation formula is:

[0016]

[0017] Among them, X(t) represents the data at time t; X(t) norm represents the standardized data; max(X) represents the maximum daily data; min(X) represents the minimum daily data.

[0018] Preferably, the method for obtaining the detail component and the approximate component comprises:

[0019] The processed data is decomposed into detail components of different scales using àtrous wavelet decomposition and approximate components

[0020]

[0021]

[0022]

[0023] g l =(-1) l+1 h L-1-t

[0024] in, represents the standardized data at time t; j represents the decomposition level, j = 1, 2, ..., J, J represents the highest decomposition level; N represents the number of samples; mod represents the modulo operation; g represents the scale filter; h represents the wavelet filter; l = 0, 1, ..., L-1, L represents the filter length.

[0025] Preferably, the method for selecting the significant predictor comprises: selecting the significant predictor using the Boruta algorithm:

[0026] creating a copy of the historical hydrometeorological data and adding randomness to the copy to reduce correlation between the shadow features and the target;

[0027] Using a random forest model with the historical hydrometeorological data and the replica as input, predicting the target by the random forest model, and calculating the z score of each variable by dividing the mean of the precision loss of each variable by the standard deviation of the precision loss;

[0028] The maximum z-score in the copy is found, the variables with z-scores greater than the maximum z-score are marked as important, and the variables with z-scores less than the maximum z-score are considered unimportant, and finally the copy is removed, and the data marked as important are selected as the important predictors.

[0029] Preferably, the method for evaluating prediction performance includes: using Nash efficiency coefficient NSE and root mean square error RMSE to evaluate the model prediction performance.

[0030] Preferably, the method for calculating the influence of each of the significant predictors on the prediction result comprises: using an LRP algorithm to calculate the influence of each of the significant predictors on the prediction result.

[0031] Compared with the prior art, the present invention has the following beneficial effects:

[0032] The present invention uses àtrous wavelet decomposition to decompose data into components at different scales, and can extract effective information from the data; the Boruta algorithm is used to select prediction factors that significantly affect daily runoff from candidate prediction factors, reducing the problem of data redundancy; the LRP algorithm is used to perform interpretable analysis on the prediction results of the LSTM model, thereby improving the interpretability of the LSTM model. The present invention constructs an interpretable multi-scale LSTM daily runoff prediction method, which improves the accuracy of daily runoff prediction and enhances the interpretability of the prediction results. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the technical solution of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0034] Figure 1 A schematic diagram of a method flow of an embodiment of the present invention;

[0035] Figure 2 This is a diagram of the àtrous wavelet decomposition process of an embodiment of the present invention. DETAILED DESCRIPTION

[0036] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0037] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0038] In this embodiment, if Figure 1 As shown in FIG. 1 , an interpretable multi-scale LSTM daily runoff prediction method includes the following steps:

[0039] S1. Collect daily runoff data and its candidate predictors (precipitation, temperature, evaporation, sunshine hours, air pressure and relative humidity), preprocess the collected data, and divide the processed data into training set and test set.

[0040] Collect daily runoff, precipitation, temperature, evaporation, sunshine hours, air pressure and relative humidity data from 2000 to 2024 at Xingzi Station in the Poyang Lake Basin; preprocess the collected data, including outlier processing and null value filling, and standardize the daily runoff and its prediction factors. The processed data are divided into a training set and a test set, of which the data from 2000 to 2019 are the training set and the data from 2020 to 2024 are the test set. The standardization calculation formula is:

[0041]

[0042] Among them, X(t) represents the data at time t; X(t) norm represents the standardized data; max(X) represents the maximum daily data; min(X) represents the minimum daily data.

[0043] S2. Decompose the processed data into detail components and approximate components at different scales.

[0044] In this embodiment, if Figure 2 As shown, àtrous wavelet decomposition is used to decompose the processed data into detail components of different scales. and approximate components

[0045]

[0046]

[0047]

[0048] g l =(-1) l+1 h L-1-t

[0049] in, represents the standardized data at time t; j represents the decomposition level, j = 1, 2, ..., J, J represents the highest decomposition level; mod represents the modulo operation; g represents the scale filter; h represents the wavelet filter; l = 0, 1, ..., L-1, L represents the filter length.

[0050] Since the data of traditional wavelet decomposition is affected by boundary conditions, it is necessary to remove the information affected by the boundary from the beginning of the time series when performing àtrous wavelet decomposition. J It can be expressed as:

[0051] L J =(2 J -1)(L-1)+1.

[0052] In the decomposition process, the wavelet basis function is set to db2 wavelet (length is 4), and the decomposition level is 3. Therefore, the number of affected L that needs to be deleted J =22.

[0053] S3. Select significant predictors from the components of candidate predictors for each daily runoff component:

[0054] In this embodiment, the Boruta algorithm is used to select significant predictors from the hydrometeorological data component for each daily runoff component. The Boruta algorithm factor selection process is as follows: (1) Initialization: Create a copy (shadow variable) of the original variable (hydrometeorological data) and add randomness to reduce the correlation between the shadow feature and the target; (2) Variable importance evaluation: Use a random forest model with the original variable and the shadow variable as input, predict the target through the random forest model, and calculate the z score (importance) of each variable by dividing the mean of the precision loss of each variable by the standard deviation of the precision loss; (3) Determine important variables: Find the maximum z score (MZSV) in the shadow variable, mark the variables with z value>MZSV as important, and consider the variables with z value lower than MZSV as unimportant. Then, remove the shadow variable; (4) Iteration: Repeat the above variable importance evaluation and determination of important variables steps until all important variables are confirmed or the iteration threshold is reached. In order to prevent data overfitting and improve computational efficiency, the maximum number of runs of the Boruta algorithm is set to 100 times.

[0055] S4. Construct an LSTM model, use the significant predictor as model input, train the LSTM model of each daily runoff component, and obtain a daily runoff component prediction model.

[0056] In this embodiment, the typical LSTM storage unit C constructed t It consists of three gates: input gate, forget gate and output gate. The forget gate looks at the previous output h t-1 and the current input x t To decide which information to remove from the cell state, it is defined as:

[0057] f t =σ(W f h t-1 +U f x i +b f )

[0058] Among them, f t represents the output of the forget gate at a certain moment; x t is the input at time t; W f and U f represents the weight of the forget gate; b fis the bias of the forget gate; σ represents the sigmoid function. The input gate determines whether to store the input information in the state of the current time step and is defined as:

[0059] i t =σ(W i h t-1 +U i x i +b i )

[0060] Among them, i t is the output of the input gate; W i and U i represents the network weight matrix of the input gate; b i represents the bias of the input gate. In the next step, the candidate cell states are derived by the following equation:

[0061]

[0062] Among them, W C and U C represents the weight of the storage unit; b C represents the bias of the storage unit; tanh is the hyperbolic tangent function. Next, the old cell state C is converted into t-1 Update to the new cell state C t :

[0063]

[0064] The * indicates multiplication. Finally, the output gate is used. First, the sigmoid layer is used to determine which part of the cell state will be used for output. Then, the cell state is processed by tanh and multiplied with the output of the sigmoid gate to finally determine the output part. The current output h of LSTM t is determined based on the result of the output gate and the cell state:

[0065] οt=σ(Woht-1+Uoxt+bo)

[0066]

[0067] Among them, t represents the output of the output gate at time t; W o and U o represents the weight of the output gate; b o Indicates bias; h t is the output of the hidden layer at time t.

[0068] During the LSTM training process, ReLU is used as the activation function, Adam is used as the optimizer, and mean square error is used as the loss function. The number of hidden layer units is set to 30. The learning rate is set to 0.001, and the number of iterations is set to 1000. After the training is completed, the daily runoff prediction model is obtained.

[0069] S5. The daily runoff component prediction model is used to test the daily runoff detail component and approximate component during the test period to obtain the prediction results of each component, reconstruct the prediction results, obtain the final daily runoff prediction value and evaluate the prediction performance.

[0070] The Nash efficiency coefficient NSE and root mean square error RMSE are used to evaluate the prediction performance of the model. The expressions of NSE and RMSE are:

[0071]

[0072]

[0073] Where M represents the total number of sequences; y for,i represents the predicted value of the i-th sequence; y obs,i represents the observed value of the i-th sequence; Represents the mean of the observed values. The closer the RMSE value is to 0 and the closer the NSE value is to 1, the better the model performance.

[0074] S6. Calculate the influence of each of the significant predictors on the prediction result.

[0075] In this embodiment, the relevance score of each important predictor is calculated using LRP (Layered Relevance Propagation), which is a technique used in deep learning to explain the predictions made by neural networks; it assigns a relevance score to each input to judge the relevance of each input to the final decision. The LRP explanation technique is used to integrate interpretability into highly complex deep learning model networks. LRP backpropagates the prediction results in the neural network through a specially designed backpropagation rule. The propagation process it implements follows the conservation property, that is, what a neuron receives must be redistributed to lower layers in equal quantities. Let i and j be neurons in two consecutive layers of a deep neural network. The relevance score R on a given layer is calculated by applying the following rule: j Propagate to the next layer of neurons:

[0076]

[0077] Among them, R j is the correlation value of the jth neuron in the upper layer; R i is the correlation value of the th neuron in the lower neural network; a i It is the input of the upper layer neurons; represents the weight between two neurons, where “+” means only positive weights are considered; b j is the bias of the jth neuron. According to the relevance score of each predictor (model input), the relative importance and contribution of different predictors to runoff are analyzed.

[0078] The embodiments described above are only descriptions of the preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the protection scope determined by the claims of the present invention.

Claims

1. An interpretable multi-scale LSTM daily runoff prediction method, characterized in that: The following steps are involved: Collect daily runoff data and its candidate predictors, preprocess the collected data, and divide the processed data into training set and test set; Decomposing the processed data into detail components and approximate components at different scales; For each daily runoff component, selecting a significant predictor from the components of the candidate predictors; Constructing an LSTM model, taking the significant predictor as a model input, training the LSTM model of each of the daily runoff components, and obtaining a daily runoff component prediction model; The daily runoff component prediction model is used to test the daily runoff detail component and the approximate component of the test set to obtain the prediction results of each component, the prediction results are reconstructed to obtain the final daily runoff prediction value and the prediction performance is evaluated; Using the LRP algorithm to calculate the impact of each of the significant predictors on the prediction results; The candidate prediction factors include: precipitation, temperature, evaporation, sunshine hours, air pressure and relative humidity; The preprocessing method includes: outlier processing, null value filling and standardization; The standardized calculation formula is: Among them, X(t) represents the data at time t; X(t) norm represents the standardized data; max(X) represents the maximum daily data; min(X) represents the minimum daily data; The method for calculating the influence of each of the significant predictors on the prediction results includes: using the LRP algorithm to integrate interpretability into a highly complex deep learning model network, calculating the influence of each significant predictor on the prediction results, using LRP to calculate the relevance score of each important predictor, and the propagation process implemented therefor follows the conservation property, that is, the neurons received must be redistributed to the lower layers in equal quantities, let i and j be neurons in two consecutive layers of the deep neural network, and by applying the rule, the relevance score R on a given layer is j Propagate to the next layer of neurons: Among them, R j is the correlation value of the jth neuron in the upper layer; R i is the correlation value of the ith neuron in the lower neural network; a i It is the input of the upper layer neurons; represents the weight between two neurons, where "+" means only positive weights are considered; b j is the bias of the jth neuron; according to the correlation score of each predictor, the relative importance and contribution of different predictors to runoff are analyzed.

2. The explainable multi-scale LSTM daily runoff prediction method according to claim 1, characterized in that: The method of obtaining the detail component and the approximate component comprises: The processed data is decomposed into detail components of different scales using àtrous wavelet decomposition and approximate components g l =(-1) l+1 h L-1-t in, represents the standardized data at time t; j represents the decomposition level, j = 1, 2, ..., J, J represents the highest decomposition level; N represents the number of samples; mod represents the modulo operation; g represents the scale filter; h represents the wavelet filter; l = 0, 1, ..., L-1, L represents the filter length.

3. The explainable multi-scale LSTM daily runoff prediction method according to claim 1, characterized in that: The method of selecting the significant predictor includes: selecting the significant predictor using the Boruta algorithm: creating copies of the hydrometeorological data and adding randomness to the copies to reduce correlation between the shadow features and the target; Using a random forest model with the hydrometeorological data and the replicas as input, predicting the target by the random forest model, and calculating the z score of each variable by dividing the mean of the precision loss of each variable by the standard deviation of the precision loss; The maximum z-score in the copy is found, the variables with z-scores greater than the maximum z-score are marked as important, and the variables with z-scores less than the maximum z-score are considered unimportant, and finally the copy is removed, and the data marked as important are selected as the important predictors.

4. According to claim 1, an interpretable multi-scale LSTM daily runoff prediction method is characterized in that: The prediction performance evaluation method includes: using Nash efficiency coefficient NSE and root mean square error RMSE to evaluate the model prediction performance.

Images