River flow prediction method
The DTansformer-LSTM method, which combines wavelet denoising and differential transform, solves the problem of insufficient accuracy in river flow prediction for extreme changes, and improves the prediction accuracy for sudden changes such as floods. In particular, it significantly improves the prediction effect in nonlinear and highly volatile scenarios.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- CHINA THREE GORGES UNIV
- Filing Date
- 2025-12-08
- Publication Date
- 2026-05-01
AI Technical Summary
Existing methods for predicting river flow lack accuracy during periods of extreme change, especially in sudden hydrological events such as floods, where they struggle to accurately capture abrupt changes. Traditional models rely on large amounts of high-precision data and exhibit significant prediction biases in nonlinear and highly volatile scenarios.
The DTransformer-LSTM method, which combines wavelet denoising with differential transform, enhances the sensitivity to abrupt changes by using Haar wavelet denoising and first-order differential transform. Furthermore, by combining the DTransformer encoder and LSTM decoder, the attention mechanism is improved to enhance the ability to capture abrupt changes in traffic flow.
It significantly improves the prediction accuracy of extreme flow events such as flood peaks and low water levels, effectively avoids the prediction bias of traditional models under high-frequency noise and non-stationarity, and improves the model's sensitivity to sudden changes in flow and prediction accuracy.
Smart Images
Figure CN121960109A_ABST
Abstract
Description
A method for predicting river flow Technical Field
[0001] This invention relates to the fields of computer technology and river monitoring technology, and in particular to a method for predicting river flow based on time series prediction technology, specifically a method for predicting river flow based on wavelet denoising technology and differential transform using DTansformer-LSTM. Background Technology
[0002] Accurate prediction of river flow is of great significance for disaster prevention and mitigation, water resource management, and ecological protection. Traditional prediction methods mainly rely on process-based physical and conceptual models to simulate hydrological processes through physical mechanisms or empirical equations. However, their application has obvious limitations: on the one hand, they require a large amount of accurate input data and parameters, making implementation difficult and limiting their applicability in areas with scarce data; on the other hand, when faced with complex and variable actual hydrological conditions, the models often suffer from increased uncertainty due to simplification assumptions and difficulties in parameter calibration, thus limiting prediction accuracy.
[0003] For example, the article "Multi-objective Calibration and Evaluation of SWAT Model—Taking the Meichuan River Basin as an Example" records that the uncertainty of SWAT model parameters leads to doubts about the accuracy of model results, and the compensation effect between parameters makes the model susceptible to interference from the phenomenon of different parameters having the same effect. In contrast, data-driven models do not rely on a deep understanding of physical processes, but capture the complex nonlinear relationships between multiple factors such as meteorology and flow by analyzing historical data. They have strong adaptability and flexibility, and can still build effective prediction models even under limited data conditions. Therefore, they are gradually being widely used in the field of hydrology. However, existing time series models generally have limitations in the accurate prediction of extreme hydrological events, especially in the accurate capture of abrupt changes such as sudden increases and decreases in river flow. This problem is particularly prominent in the forecasting of sudden hydrological events such as floods.
[0004] To enhance the ability of methods and models to identify dynamic changes in sequences, especially abrupt changes, and thus improve the prediction effect of river flow, this invention proposes a DTansformer-LSTM method for predicting river flow based on wavelet denoising technology combined with differential transform. Summary of the Invention
[0005] The purpose of this invention is to address the technical problem of insufficient prediction accuracy in existing technologies for predicting extreme changes in river flow, and to propose a method for predicting river flow. This invention can significantly improve the prediction accuracy of extreme flow events such as flood peaks and low water levels, and effectively avoids the problems of traditional hydrological models that rely on a large amount of high-precision data, have difficult parameter calibration, and have prediction biases in nonlinear and highly volatile scenarios.
[0006] To solve the above-mentioned technical problems, the technical solution adopted by this invention is as follows: a method for predicting river flow, comprising the following steps: Step 1: dividing the daily runoff dataset, using Haar wavelet to perform multi-level decomposition on the training set to obtain denoising parameters (thresholds); Step 2: using the parameters obtained in Step 1 to reuse the validation set and test set to obtain processed data; Step 3: performing a first-order difference transformation on the data obtained in Step 2, and then scaling it to the [0,1] interval using MinMaxScaler to obtain the normalized value of the difference sequence; Step 4: mapping the data obtained in Step 3 to a high-dimensional space through a linear embedding layer, and then performing position encoding; Step 5: inputting the data obtained after position encoding in Step 4 into a DTransformer encoder to give higher attention weights to abrupt change points, thereby strengthening the difference features of adjacent time steps, and obtaining the encoder output; Step 6: inputting the encoder output obtained in Step 5 into an LSTM decoder for feature recombination, and obtaining the decoder output; Step 7: mapping the output obtained in Step 6 to a 7-dimensional output through a fully connected layer; Step 8: performing inverse normalization and inverse difference transformation on the data obtained in Step 7 to obtain the final output.
[0007] In step 1, the daily average flow data from the Taojiang and Taoyuan hydrological stations from January 1, 2013 to December 31, 2022 are used as the basis for modeling. The entire sequence is divided into a training set (2013-2018), a validation set (2019-2020), and a test set (2021-2022) in chronological order to meet the needs of model training, tuning, and performance evaluation. The training set is decomposed into sub-bands of different scales and frequencies using Haar wavelets. The noise and effective signals are separated by the frequency characteristics of the signals in each sub-band. The number of decomposition levels and the threshold F1 are determined based on the training set. The detail coefficients are soft-thresholded while the approximation coefficients remain unchanged.
[0008] In step 2, the parameters obtained in step 1 are applied to the validation and test sets to ensure that the validation and test sets do not use future information. Wavelet reconstruction is then performed to obtain the denoised parameters. During wavelet reconstruction, the Haar wavelet, which is sensitive to abrupt changes, is selected for denoising to suppress noise interference while ensuring the complete preservation of key abrupt changes such as flood peaks. The Haar scaling function is then used... With wavelet function The approximation coefficients and detail coefficients are obtained by performing an inner product operation with the signal, specifically: (1); (2); where i is the translation parameter and j is the scale parameter, so that the Haar wavelet function can perform multi-scale and multi-location analysis of the signal, thereby capturing different frequency components and local features of the signal in the wavelet transform; retain the approximation coefficients, use the soft threshold function to perform adaptive threshold calculation, and use the detail coefficients after threshold processing to reconstruct the signal.
[0009] In step 3, the data obtained in step 2 is subjected to first-order differencing. By calculating the difference between adjacent observations in the sequence, the absolute flow rate is converted into a relative change, which can effectively eliminate the inherent trend and seasonal components of the data, thereby reducing the interference of non-stationarity on model training. This makes it easier to capture short-term fluctuations and sudden changes in the data and improve the model's sensitivity to changing trends. The differencing sequence is normalized using the MinMaxScaler method to scale it to the [0,1] interval, ensuring that the model can learn effectively and converge stably, while improving the model's generalization ability and simplifying the hyperparameter tuning process.
[0010] In step 4, the data obtained in step 3 is input into the linear embedding layer and its dimensionality is expanded to map it to a high-dimensional space in order to capture its short-term fluctuations and trend information; then it is positionally encoded to explicitly inject position information; in step 5, the data obtained in step 4 is input into the DTransformer encoder for processing. The DTransformer encoder is composed of four stacked DTransformer encoding layers; specifically, the following steps are adopted: S5-1: The four stacked DTransformer encoding layers form the DTransformer encoder. The first layer of the encoder learns the shallow feature local pattern, the second layer learns the mid-level feature combination pattern, the third layer learns the deep feature global dependency, and the fourth layer finally represents the rich context; S5-2: Input signal X input The attention scores are calculated by obtaining Q, K, and V through linear transformation. A multi-head differential attention mechanism is then applied, subtracting an adaptive mean offset from the original attention scores to learn the parameter-weighted average attention score. The differential attention scores are then converted into a probability distribution using softmax and dropout is applied, multiplying them by V to obtain the attention output. This process gives higher attention weights to elements that significantly deviate from the average pattern (mutation points), thereby strengthening the difference features between adjacent time steps. S5-3: The output obtained from the multi-head differential attention mechanism is processed through a feedforward neural network, residual connections, and layer normalization.
[0011] In step S5-3, the input of the feedforward neural network is extended from 128 dimensions to 256 dimensions through the first linear layer, and then through the ReLU activation function to introduce nonlinearity. Its output is protected against overfitting by Dropout, and then through the second linear layer to compress the 256 dimensions to 128 dimensions. It passes through residual connections to prevent gradient vanishing and promote deep network training. Layer normalization is used to stabilize the training process, accelerate convergence, and reduce sensitivity to initialization. In step 6, the output obtained in step 5 is input to the LSTM decoder for sequence information integration and temporal pattern learning. Specifically, the output of the DTransformer encoder is used as the input of the LSTM decoder, and the LSTM decoder performs hidden state transfer and memory cell transfer.
[0012] The gating mechanism of LSTM and cell state; the gating mechanism includes: input gate i t Forgotten Gate t With output gate o t The input gate uses the sigmoid activation function to calculate the input control signal and the tanh activation function to generate candidate memory content. Candidate memory content is only updated in the memory unit when the input gate is active. The forget gate uses only the sigmoid activation function to output weight values in the 0-1 range, corresponding to the proportion of the memory unit state to be forgotten, thus dynamically controlling the retention and clearing of information in the memory unit. The output gate first calculates the output control signal using the sigmoid activation function, and then generates the state features to be output using the tanh activation function, precisely controlling the contribution of the memory unit state to the output result of the current time step. Dropout is used for regularization between LSTM decoder layers, randomly discarding some neuron outputs, forcing the network not to depend on any specific neuron or feature combination. This ensures that the model maintains its temporal modeling ability while preventing the model from over-relying on certain specific patterns or time steps of the encoder output, thereby improving generalization ability.
[0013] In step 7, the output obtained in step 6 is nonlinearly transformed by the fully connected layer and mapped to a 7-dimensional output. The complex interactions between features are learned through ReLU activation, and the rich hidden representations are mapped to the specific prediction target dimension. The fully connected layer is divided into two stages. The first stage is from 128 dimensions to 64 dimensions, providing sufficient nonlinear transformation space to prevent possible information loss when directly from 128 dimensions to 7 dimensions. The second stage is from 64 dimensions to 7 dimensions, accurately mapping to the target dimension and maintaining the correlation between predicted values.
[0014] In step 9, following step 8, the data obtained in step 7 is inversely normalized to obtain the difference value of the original scale, and then the final output is obtained through inverse difference transformation.
[0015] Compared with the prior art, the present invention has the following technical effects: 1) The method and model proposed in this application can effectively solve the key problems of the current model having large errors in peak flow prediction and difficulty in effectively handling abnormal flow fluctuations caused by extreme weather events; 2) In response to the key problem of insufficient accuracy in predicting sudden changes in river flow, the present invention proposes a data preprocessing strategy that combines wavelet denoising and differential transform. Specifically, the Haar wavelet, which is sensitive to abrupt change features, is used for denoising. While suppressing high-frequency noise interference in the time series, it fully preserves the main trend and key abrupt change points (such as flood event features) of the flow series, effectively solving the problem of high-frequency noise masking abrupt change information in traditional preprocessing and significantly improving the identifiability of flood peaks. Furthermore, the differential transformation is combined to eliminate the trend and seasonal components of the hydrological series, alleviate the model learning bias caused by non-stationarity, and provide high-quality, highly stationary input data for subsequent modeling. 3) This invention proposes a differential attention mechanism to improve the Transformer's native self-attention module and enhance the model's ability to capture abrupt change features in flow. Based on the traditional self-attention score calculation, this mechanism introduces a differential operation of adaptive attention mean offset, so that the model focuses on relative change features within the sequence rather than absolute similarity. The improved differential attention mechanism can be equivalent to a module with adaptive high-pass filtering characteristics, which can actively strengthen the attention weight of abrupt change information in the sequence and significantly improve the model's sensitivity to abnormal dynamics such as sudden increases and decreases in flow. 4) This invention designs an asymmetric hybrid architecture of DTransformer encoder-LSTM decoder, breaking through the conventional design ideas of existing LSTM-Transformer combined models. This architecture retains only the Transformer encoder module to handle global feature extraction, while using LSTM as the decoder for sequence decoding. This achieves a deep fusion of the Transformer encoder's powerful global feature modeling capabilities and the LSTM decoder's excellent long-sequence dependency capture capabilities, making it particularly suitable for hydrological time series prediction tasks with long-term dependencies and abrupt changes. Attached Figure Description
[0016] The present invention will be further described below with reference to the accompanying drawings and embodiments: Figure 1 is a flowchart of the overall workflow of the present invention; Figure 2 is a block diagram of the model structure in Figure 2; Figure 3 is a network diagram of the DTransformer encoder in Figure 1; Figure 4 is a diagram of the multi-head differential attention mechanism in Figure 1; Figure 5 is a network diagram of the LSTM network in Figure 1. Detailed Implementation
[0017] As shown in Figure 1, this invention discloses a DTansformer-LSTM method for predicting river flow based on wavelet denoising technology combined with differential transform. The specific steps include: Step 1: Divide the daily runoff dataset and use Haar wavelets to perform multi-level decomposition on the training set to obtain denoising parameters (thresholds) F1; Step 2: Use the parameters obtained in Step 1 to reuse the validation and test sets to obtain data F2; Step 3: Perform a first-order differential transform on the data obtained in Step 2, and then use MinMaxScaler to scale it to the [0,1] interval to obtain the normalized value F3 of the difference sequence; Step 4: Step 3: The data obtained in step 3 is passed through a linear embedding layer, mapped to a high-dimensional space, and then positionally encoded to obtain data F4; Step 5: The data obtained in step 4 is input into a DTransformer encoder to give higher attention weights to the mutation points, thereby strengthening the difference features between adjacent time steps, and the encoder output F5 is obtained; Step 6: The data obtained in step 5 is input into an LSTM decoder for feature reconstruction, and the decoder output F6 is obtained; Step 7: The data obtained in step 6 is passed through a fully connected layer to map to a 7-dimensional output; Step 8: The data obtained in step 7 is inversely normalized and subjected to inverse differential transformation to obtain the final output.
[0018] Steps 4 to 7 are shown in Figure 2. In step 1, the specific method is as follows: The daily average flow data from the Taojiang and Taoyuan hydrological stations from January 1, 2013 to December 31, 2022 are used as the basis for modeling. The entire sequence is divided chronologically into a training set (2013-2018), a validation set (2019-2020), and a test set (2021-2022) to meet the needs of model training, optimization, and performance evaluation. The training set is decomposed into sub-bands of different scales and frequencies using Haar wavelets. Noise and effective signals are separated by the frequency characteristics of each sub-band signal. The number of decomposition levels and the threshold F1 are determined based on the training set. Soft thresholding is applied to the detail coefficients, while retaining the approximation coefficients unchanged.
[0019] In step 2, the specific steps are as follows: the parameters obtained in step 1 are applied to the validation and test sets to ensure that the validation and test sets do not use future information. Wavelet reconstruction is then performed to obtain the denoised data F2. This invention selects the Haar wavelet, which is sensitive to abrupt change signals, for denoising to ensure the complete preservation of key abrupt change information such as flood peaks while suppressing noise interference. This is achieved through the Haar scaling function. (1) with wavelet function (2) Perform an inner product operation with the signal to obtain the approximation coefficients and detail coefficients; (1); (2); Here, i is the translation parameter and j is the scaling parameter, enabling the Haar wavelet function to perform multi-scale and multi-location analysis of the signal, thereby capturing different frequency components and local features of the signal in the wavelet transform. Approximate coefficients are retained, and an adaptive threshold calculation is performed using a soft thresholding function. The signal is then reconstructed using the thresholded detail coefficients.
[0020] In step 3, the specific steps are as follows: First-order differencing is applied to the data obtained in step 2. By calculating the difference between adjacent observations in the sequence, the absolute flow rate is converted into a relative change, which can effectively eliminate the inherent trend and seasonal components of the data, thereby reducing the interference of non-stationarity on model training. This facilitates better capture of short-term fluctuations and abrupt changes in the data and improves the model's sensitivity to changing trends. The differrated sequence is then normalized using the MinMaxScaler method, scaling it to the [0,1] interval to ensure that the model can learn effectively and converge stably, while also improving the model's generalization ability and simplifying the hyperparameter tuning process.
[0021] In step 4, the specific steps are as follows: the data obtained in step 3 is input into the linear embedding layer, and its dimensionality is expanded to map it to a high-dimensional space in order to capture its short-term fluctuations and trend information. Then, it is positionally encoded to explicitly inject position information.
[0022] In step 5, the data obtained in step 4 is input into the DTransformer encoder for processing. The DTransformer encoder consists of four stacked DTransformer coding layers. Specifically, the following steps are adopted: S1: The four stacked DTransformer coding layers form the DTransformer encoder. The first encoder layer learns shallow feature local patterns, the second encoder layer learns mid-level feature combination patterns, the third encoder layer learns deep feature global dependencies, and the fourth encoder layer finally represents the rich context; S2: Input signal X inputThe process begins by linearly transforming Q, K, and V to calculate attention scores. A multi-head differential attention mechanism is then applied, subtracting an adaptive mean offset from the original attention scores to learn the parameter-weighted average attention score. The differential attention scores are then converted to a probability distribution using softmax and dropout is applied, multiplying them by V to obtain the attention output. This process gives higher attention weights to elements that significantly deviate from the average pattern (mutation points), thereby strengthening the difference features between adjacent time steps. S3: The output obtained from the multi-head differential attention mechanism is processed through a feedforward neural network and residual connections, followed by layer normalization. The input to the feedforward neural network passes through a first linear layer, expanding the 128-dimensional network to 256 dimensions. A ReLU activation function is then applied to introduce non-linearity. The output is protected against overfitting by Dropout, then passes through a second linear layer, compressing the 256-dimensional network to 128 dimensions. Residual connections prevent gradient vanishing and promote deep network training. Layer normalization stabilizes the training process, accelerates convergence, and reduces sensitivity to initialization.
[0023] In step 6, when the data obtained in step 5 is input into the LSTM decoder for sequence information integration and temporal pattern learning, the following steps are specifically adopted: S1: The output of the DTransformer encoder is used as the input of the LSTM decoder. The gating mechanism of LSTM includes: input gate i t Forgotten Gate t With output gate o t The input gate uses a sigmoid activation function to calculate the input control signal and a tanh activation function to generate candidate memory content. Candidate memory content is only updated in the memory unit when the input gate is active. The forget gate uses only a sigmoid activation function to output weight values in the 0-1 range, corresponding to the proportion of the memory unit state to be forgotten, thus dynamically controlling the retention and clearing of information in the memory unit. The output gate first calculates the output control signal using a sigmoid activation function, and then generates the state features to be output using a tanh activation function, precisely controlling the contribution of the memory unit state to the output result of the current time step. S2: Dropout is used for regularization between LSTM layers, randomly discarding some neuron outputs, forcing the network not to depend on any specific neuron or feature combination. This ensures that the model maintains its temporal modeling ability while preventing the model from over-relying on certain specific patterns or time steps of the encoder output, thereby improving generalization ability.
[0024] In step 7, the specific steps are as follows: the 128-dimensional output obtained in step 6 is nonlinearly transformed by a fully connected layer and mapped to a 7-dimensional output. ReLU activation is used to learn the complex interactions between features, mapping the rich hidden representations to the specific prediction target dimension. The fully connected layer consists of two stages: the first stage reduces the dimensions from 128 to 64, providing sufficient nonlinear transformation space to prevent potential information loss during the direct reduction from 128 to 7 dimensions; the second stage reduces the dimensions from 64 to 7, accurately mapping to the target dimension while maintaining the correlation between predicted values.
[0025] In step 8, the specific steps are as follows: the data obtained in step 7 is inversely normalized to obtain the difference value of the original scale, and then the final output is obtained by inverse difference transformation.
[0026] Example: The DTransformer encoder network structure is shown in Figure 3. The specific implementation details are as follows: The encoder receives input data, maps it to a high-dimensional feature space through an embedding layer, and superimposes positional encoding to inject temporal information. Subsequently, the data undergoes deep feature extraction through four encoding layers, each containing a multi-head differential attention mechanism to enhance the ability to capture sequence mutation features. The training process is then stabilized through residual connections and layer normalization, and finally, a feedforward neural network enhances the nonlinear representation capability. The three components work together to ensure efficient feature extraction and a stable training process.
[0027] The detailed flowchart of the multi-head differential attention mechanism is shown in Figure 4. The specific steps are as follows: The multi-head self-attention mechanism, through the design of multiple parallel attention heads, enables the model to extract and enhance key features of the input sequence from different dimensions, significantly improving its ability to understand complex temporal patterns. The underlying implementation of this mechanism is based on scaled dot product attention. First, a query matrix Q, a key matrix K, and a value matrix V are generated from the same input through a linear transformation. The dot product of Q and K is calculated to evaluate the similarity between each query and all keys. Then, the dot product result is divided by... (d) k The scaled values (where K is the dimension of the vector) are scaled to avoid the vanishing gradient problem. The scaled scores are then applied using the Softmax function to obtain the attention weights. Finally, these weights are multiplied by matrix V to generate a weighted feature representation. In the multi-head mechanism, the model executes h independent scaled dot-product attention calculations in parallel (h is the number of attention heads). Each calculation uses independent linear transformation parameters to produce its own feature representation. The output features of all attention heads are concatenated dimensionally and fused through a single linear transformation. The final output is then passed to a feedforward neural network for further processing.
[0028] (1) Scaling point attention (3); (2) Multi-head scaling point attention (4); among which, , , , h represents the number of attention heads. d is the dimension, d k Let K be the dimension of the vectors, and T denote the transpose of the matrix. This invention introduces a differential operation on top of the traditional multi-head attention mechanism to enhance the model's sensitivity to abrupt changes in the sequence (sudden increases or decreases in traffic data). The differential attention mechanism emphasizes relative changes in the sequence rather than absolute similarity. The improvement is performed on each query vector of each head, subtracting an adaptive mean offset from the original attention score to learn the parameters. The weighted attention scores are averaged, and the differencing attention scores are transformed into a probability distribution using softmax, followed by dropout. This process assigns higher attention weights to elements that significantly deviate from the average pattern (mutation points), thereby amplifying the differences between adjacent time steps.
[0029] (5); It is a learnable parameter, with an initial value of 0.7.
[0030] The LSTM network diagram is shown in Figure 5. The specific steps are as follows: Since the time axis structure of the encoder output is naturally aligned with the input structure of the decoder, the output feature vector is reconstructed into a time series format and directly input into the LSTM decoder without explicit projection. The number of decoding layers is the same as the number of encoding layers to ensure consistency of feature dimensions. A Dropout mechanism is introduced between each LSTM layer to reduce the risk of overfitting. The LSTM memory network unit is the cell state C from the previous time step. t-1 and hidden state h t-1 Starting with the current input, the system systematically regulates the information flow through its core gating mechanism: First, the forget gate uses the sigmoid function to determine which information to discard from historical cell states; then, the input gate uses another sigmoid function to filter out new information worth retaining from the current input and uses the tanh function to generate candidate values to update the cell state; finally, the output gate again uses the sigmoid function to regulate the information to be output at this moment and uses the tanh function to process the updated cell state C. t This generates the hidden state output h at the current time step. t .
[0031] The following equation describes the forward propagation process in LSTM: (6); (7); (8); (9); (10); (11); Equation (6) selects to forget certain past information, Equations (7) and (8) represent remembering certain present information, Equation (9) merges past and present memories, Equations (10) and (11) represent the final output content, X t The input data is time t. h represents the cell state of the LSTM. t At h t-1 C represents the current and previous time steps. t and C t-1 Let W represent the cell state at the current and previous time steps, where W is the weight matrix and b is the bias.
[0032] Finally, after being output by the decoder, the 128-dimensional features are mapped to a 7-dimensional output (prediction for the next 7 days) through a two-layer fully connected network. A funnel-shaped structure is used to progressively compress the feature dimension. After the first layer, it is reduced to 64 dimensions, and then input into the second layer via a ReLU activation function, reducing it from 64 dimensions to the target dimension of 7. The computation process of the fully connected layer can be represented as follows: (12); where x is the input vector, W is the weight matrix containing the weights of all connections, and b is the bias vector used to adjust the linear transformation of the output.
[0033] ReLU activation function expression: (13); When the input x>0, the output is equal to the input; when the input x≤0, the output is 0.
[0034] This invention addresses the technical problem of insufficient prediction accuracy in existing techniques for predicting extreme variations in river flow by proposing a new method for river flow prediction. The model enhances the identifiability of flood peaks and improves the model's sensitivity to abrupt gradient changes through a wavelet denoising combined with a differential transform preprocessing method, thereby improving the model's prediction performance. A differential attention mechanism is introduced to make the model pay more attention to abrupt changes in the flow sequence, improving prediction accuracy. The powerful global modeling capabilities of the Transformer and the local temporal advantages of the LSTM are integrated to effectively improve the accuracy of river flow prediction. The comparative experimental results are shown in Table 1: Results of each method on the Taojiang dataset.
[0035] Table 2 shows the performance indicators for each method on the Taoyuan dataset.
[0036] As shown in Tables 1 and 2, the method proposed in this invention scores the highest among all indicators, with R and NSE scores of 0.9866, 0.9733, and 0.9710, 0.9417, respectively. The prediction results are very close to reality, and the correlation coefficient R is 0.246, 0.2788, 0.2389, 0.2349, 0.2466 and 0.1857, 0.1837, 0.1838, 0.1853, 0.1827 higher than the benchmark models LSTM, Transformer, GRU, KAN, and TCN, respectively. This demonstrates the superiority of this model in predicting flow with abrupt changes. Among the five benchmark models, the correlation coefficients (R) at the same hydrological station are similar, indicating comparable predictive performance. However, the performance of the Taojiang hydrological station is slightly lower than that of the Taoyuan hydrological station. The coefficients of variation (CV%) for the two datasets are 100.55% and 94.78%, respectively. Both datasets show high CVs, indicating significant relative volatility. However, the data from the Taojiang hydrological station exhibits greater volatility and is more difficult to predict or analyze due to its higher variability. After wavelet denoising combined with differencing, the correlation coefficient (R) of the LSTM model on the Taojiang hydrological station dataset increased from 0.7406 to 0.8902; the correlation coefficient (R) of the Transformer model increased from 0.7873 to 0.8849, exceeding the increase seen at the Taoyuan hydrological station. This demonstrates that this preprocessing method is more effective for data with greater volatility, highlighting its advantage in predicting flow rates from abrupt changes and extreme values.
[0037] Experimental results verify the effectiveness of a DTansformer-LSTM method based on wavelet denoising and differential transform for predicting river flow. To address the insufficient accuracy in predicting extreme river flow variations, this invention designs an innovative algorithm model that applies wavelet denoising and differential transform to the DTansformer-LSTM model for river flow prediction. The improved differential attention mechanism (DTransformer) for temporal feature extraction is combined with LSTM sequence decoding to construct a future river flow prediction model, significantly improving the prediction accuracy for extreme flow events such as flood peaks and low water levels. This effectively solves the prediction bias problem of traditional hydrological models in nonlinear and highly volatile scenarios. In summary, this invention outperforms existing methods and achieves advanced performance in the field of river flow prediction.
Claims
1. A method for predicting river flow, characterized in that, The process includes the following steps: Step 1: Divide the daily runoff dataset and use Haar wavelets to perform multi-level decomposition on the training set to obtain denoising parameters; Step 2: Use the parameters obtained in Step 1 to reuse the validation and test sets to obtain processed data; Step 3: Perform a first-order difference transformation on the data obtained in Step 2, and then use MinMaxScaler to scale it to the [0,1] interval to obtain the normalized value of the difference sequence; Step 4: Map the data obtained in Step 3 to a high-dimensional space through a linear embedding layer, and then perform position encoding; Step 5: Input the data obtained after position encoding in Step 4 into a DTransformer encoder to give higher attention weights to abrupt changes, thereby strengthening the difference features between adjacent time steps, and obtain the encoder output; Step 6: Input the encoder output obtained in Step 5 into an LSTM decoder for feature recombination, and obtain the decoder output; Step 7: Map the output obtained in Step 6 to a 7-dimensional output through a fully connected layer; Step 8: Perform inverse normalization and inverse difference transformation on the data obtained in Step 7 to obtain the final output.
2. The method according to claim 1, characterized in that, In step 2, the parameters obtained in step 1 are applied to the validation set and the test set to ensure that the validation set and the test set do not use future information. Wavelet reconstruction is then performed to obtain the denoised parameters. When performing wavelet reconstruction, the Haar wavelet, which is sensitive to abrupt change signals, is selected to carry out denoising processing in order to suppress noise interference while ensuring the complete preservation of key abrupt change information such as flood peaks. Using the Haar scaling function With wavelet function The approximation coefficients and detail coefficients are obtained by performing an inner product operation with the signal, specifically: （1）； (2); where i is the translation parameter and j is the scaling parameter, so that the Haar wavelet function can perform multi-scale and multi-location analysis of the signal, thereby capturing different frequency components and local features of the signal in the wavelet transform; retain the approximation coefficients, use the soft threshold function to perform adaptive threshold calculation, and use the thresholded detail coefficients to reconstruct the signal; It refers to The scaling function after scaling and translation. It refers to The wavelet function after scaling and translation.
3. The method according to claim 1, characterized in that, In step 3, the data obtained in step 2 is subjected to first-order difference. By calculating the difference between adjacent observations in the sequence, the absolute flow rate is converted into a relative change. This can effectively eliminate the inherent trend and seasonal components of the data, thereby reducing the interference of non-stationarity on model training. This makes it easier to capture short-term fluctuations and sudden changes in the data and improve the model's sensitivity to changing trends. The differenced sequences are normalized using the MinMaxScaler method to scale them to the [0,1] interval, ensuring that the model can learn effectively and converge stably, while improving the model's generalization ability and simplifying the hyperparameter tuning process.
4. The method according to claim 1, characterized in that, In step 4, the data obtained in step 3 is input into the linear embedding layer and its dimensions are expanded to map it to a high-dimensional space in order to capture its short-term fluctuations and trend information. Then, position encoding is performed, and position information is explicitly injected.
5. The method according to claim 4, characterized in that, In step 5, the data obtained in step 4 is input into the DTransformer encoder for processing. The DTransformer encoder consists of four stacked DTransformer coding layers. Specifically, the following steps are taken: S5-1: The four stacked DTransformer coding layers form the DTransformer encoder. The first layer of the encoder learns shallow feature local patterns, the second layer learns mid-level feature combination patterns, the third layer learns deep feature global dependencies, and the fourth layer finally represents the rich context; S5-2: Input signal X input The attention scores are calculated by obtaining Q, K, and V through linear transformation. A multi-head differential attention mechanism is applied to subtract an adaptive mean offset from the original attention scores to learn the parameter-weighted average attention scores. The differential attention scores are then converted into a probability distribution through softmax and dropout is applied. The result is multiplied by V to obtain the attention output. This process gives higher attention weights to elements that significantly deviate from the average pattern, thereby strengthening the difference features between adjacent time steps. S5-3: The output obtained from the multi-head differential attention mechanism is processed through a feedforward neural network, residual connections, and layer normalization.
6. The method according to claim 5, characterized in that, In step S5-3, the input of the feedforward neural network is extended from 128 dimensions to 256 dimensions through the first linear layer, and then through the ReLU activation function to introduce nonlinearity. Its output is prevented from overfitting by Dropout, and then through the second linear layer to compress the 256 dimensions to 128 dimensions. It passes through residual connections to prevent gradient vanishing and promote deep network training. Layer normalization is used to stabilize the training process, accelerate convergence and reduce sensitivity to initialization.
7. The method according to any one of claims 1 to 6, characterized in that, In step 6, the output obtained in step 5 is input to the LSTM decoder for sequence information integration and temporal pattern learning. Specifically, the output of the DTransformer encoder is used as the input of the LSTM decoder, and the hidden state and memory cell are transferred through the LSTM decoder.
8. The method according to claim 7, characterized in that, The gating mechanism of LSTM and cell state; the gating mechanism includes: input gate i t Forgotten Gate t With output gate o t The input gate uses the sigmoid activation function to calculate the input control signal and the tanh activation function to generate candidate memory content. Candidate memory content is only updated in the memory unit when the input gate is active. The forget gate uses only the sigmoid activation function to output weight values in the 0-1 range, corresponding to the proportion of the memory unit state to be forgotten, thus dynamically controlling the retention and clearing of information in the memory unit. The output gate first calculates the output control signal using the sigmoid activation function, and then generates the state features to be output using the tanh activation function, precisely controlling the contribution of the memory unit state to the output result of the current time step. Dropout is used for regularization between LSTM decoder layers, randomly discarding some neuron outputs, forcing the network not to depend on any specific neuron or feature combination. This ensures that the model maintains its temporal modeling ability while preventing the model from over-relying on certain specific patterns or time steps of the encoder output, thereby improving generalization ability.
9. The method according to claim 7, characterized in that, In step 7, the output obtained in step 6 is nonlinearly transformed by the fully connected layer and mapped to a 7-dimensional output. The complex interactions between features are learned through ReLU activation, and the rich hidden representations are mapped to the specific prediction target dimension. The fully connected layer is divided into two stages. The first stage is from 128 dimensions to 64 dimensions, providing sufficient nonlinear transformation space to prevent possible information loss when directly from 128 dimensions to 7 dimensions. The second stage is from 64 dimensions to 7 dimensions, accurately mapping to the target dimension and maintaining the correlation between predicted values.
10. The method according to claim 9, characterized in that, In step 9, following step 8, the data obtained in step 7 is inversely normalized to obtain the difference value of the original scale, and then the final output is obtained through inverse difference transformation.