Water flow real-time forecasting method based on LSTM (Long Short Term Memory) time sequence prediction and residual analysis

Abstract

The invention provides a water flow real-time forecasting method based on LSTM (Long Short Term Memory) time sequence prediction and residual analysis, which comprises the following steps: collecting historical multi-source hydrometeorological data for preprocessing, and constructing a standardized feature data set; training a basic LSTM prediction model based on the data set; predicting by using the basic model to obtain a historical basic prediction sequence, calculating a historical residual sequence of the historical basic prediction sequence and a measured value, and performing stationarity verification, time sequence dependency analysis, positioning and classification on the sequence; for different types of residual errors, adaptively fusing GRU, LSTM and GARCH models to construct and train a residual error prediction model; and collecting data in real time, predicting a basic prediction sequence and a residual prediction sequence in a future time period by using the trained basic model and residual model, and generating a final water flow prediction sequence through a dynamic fusion strategy. According to the method, through a collaborative architecture of'basic prediction-residual modeling-fusion correction ', the real-time performance is ensured, and meanwhile, the forecasting precision and the extreme flood peak capturing capability are remarkably improved.

Description

Technical Field

[0001] This invention relates to the field of hydrological and water resources monitoring technology, and in particular to a real-time water flow forecasting method based on LSTM time series prediction and residual analysis. It is applicable to various application scenarios such as rivers, reservoirs, and watersheds, and can provide reliable technical support for flood control, disaster reduction, and optimized water resource allocation. Background Technology

[0002] Real-time water flow forecasting is a crucial task in the field of hydrological and water resources monitoring and forecasting. Its core requirement is to achieve high-precision, low-latency early warning capabilities, especially in responding to sudden hydrological events such as extreme floods, where forecast accuracy directly impacts the effectiveness of flood control and disaster reduction efforts. Current technologies generally face a bottleneck where it is difficult to simultaneously achieve both accuracy and real-time performance. While traditional physical mechanism models can analyze hydrological processes, they rely on a large number of parameter calibrations, resulting in high computational complexity and long processing time. This leads to a lag in short-term forecast response and often results in errors exceeding 20% ​​in capturing the peak value of extreme floods.

[0003] While single data-driven models (including Long Short-Term Memory networks, LSTM) significantly improve computational efficiency, they struggle to fully analyze the long-term dependence and extreme abrupt changes in water flow sequences, resulting in forecast Nash efficiency coefficients mostly below 0.85 and limited overall forecast accuracy. Furthermore, existing residual correction schemes lack adaptability, failing to effectively extract characteristic signals of extreme events from prediction errors, thus limiting the model's early warning capabilities in sudden flood scenarios.

[0004] Therefore, current real-time water flow forecasting technologies still suffer from insufficient accuracy, weak ability to capture extreme events, and significant response delays, failing to fully meet the rigid demand for high-precision short-term forecasts in flood control and disaster reduction. There is an urgent need for a comprehensive forecasting method that can balance forecast accuracy, real-time performance, and adaptability to extreme events to improve the reliability of dynamic monitoring and early warning of water flow in scenarios such as river basins and reservoirs. Summary of the Invention

[0005] This invention proposes a real-time water flow forecasting method based on LSTM time series prediction and residual analysis, which solves the problems of insufficient accuracy, weak ability to capture extreme events, and significant response delay in existing real-time water flow forecasting technologies.

[0006] The technical solution of this invention is implemented as follows: This invention provides a real-time water flow forecasting method based on LSTM time series prediction and residual analysis, comprising the following steps: S1, Data Preprocessing: Collect historical multi-source time-series hydrological and meteorological data of the target watershed, perform preprocessing, and construct a standardized feature dataset; S2, Construction and training of basic LSTM prediction model: Based on the standardized feature dataset, construct and train the basic LSTM prediction model; S3, Residual Sequence Generation and Analysis: The basic LSTM prediction model is used to predict the standardized feature dataset to obtain the historical basic prediction sequence; the difference between the historical basic prediction sequence and the historical measured sequence is calculated to obtain the historical residual sequence, and the historical residual sequence is subjected to stationarity verification, time-series dependency analysis, localization and classification. S4, Residual Prediction Model Construction and Training: For different types of residuals, select suitable prediction sub-models for fusion to construct a residual prediction model; use historical residual sequences and their corresponding multi-source feature data to train the residual prediction model; S5, Real-time Forecasting and Fusion Correction: Real-time hydrological and meteorological data of the target watershed up to the current time period are collected, preprocessed, and input into a trained basic LSTM prediction model to obtain the basic prediction sequence for the future time period; the latest residual sequence is calculated based on the recent basic prediction sequence and the measured sequence, and the latest residual sequence and real-time hydrological and meteorological data are input into a trained residual prediction model to obtain the residual prediction sequence for the corresponding future time period; the basic prediction sequence for the future time period is fused with the corresponding residual prediction sequence to obtain the final water flow forecast sequence.

[0007] Specifically, in step S1, the preprocessing includes outlier removal, missing value imputation, data standardization, and feature selection. The outlier removal method employs a dynamic threshold adjustment factor. Improvements The criteria and specific judgment methods are as follows: like Then remove outliers. ; in, This represents the first in a certain type of hydrological and meteorological data. i The multi-source time-series hydro-meteorological data includes at least a water flow sequence. Rainfall sequence Soil moisture sequence Weather radar echo intensity sequence Rainband movement speed sequence Multiple types of data; for The average value of the data series. for The standard deviation of the data series; This is a dynamic threshold adjustment factor that adaptively selects values ​​based on the historical flood frequency of the basin. The missing value imputation adopts a spatiotemporal collaborative strategy: for missing data in regular time periods, linear interpolation is used for imputation; for missing data in critical flood periods, forward imputation combined with correction using data from neighboring stations is used for imputation. The data standardization employs an adaptive Z-Score algorithm, segmenting the data for flood and non-flood periods. The standardization formula is as follows: ; in, These are the standardized sample values. , They represent the first k The mean and standard deviation of each data segment. k =1 corresponds to the flood season. k = 2 corresponds to the non-flood season; The feature selection method is as follows: calculate the water removal flow rate. External hydrological and meteorological characteristic variables x Corresponding water flow rate Mutual information entropy Filter out those that meet the requirements The feature variables constitute the final feature dataset.

[0008] Specifically, in step S2, the basic LSTM prediction model is a lightweight model that incorporates an attention mechanism. The model structure includes, in sequence, an input layer, an attention layer, a two-layer LSTM hidden layer, a fully connected layer, and an output layer. The attention layer is used to calculate the contribution weights of each multi-source feature in the input to the water flow forecast result. , i Number the feature type; The LSTM hidden layer captures the long-period dependence characteristics of hydrological sequences through a gating mechanism, and achieves gating adjustment through the Sigmoid activation function and element-wise product operation; The basic LSTM prediction model is trained to maximize the validation set Nash efficiency coefficient NSE. The formula for calculating the Nash efficiency coefficient NSE is as follows: ; in, N The total number of samples in the validation set of the standardized feature dataset; for t Actual measured water flow rate at any given time. for t Water flow rate model prediction at time 10:00 To verify the average value of all measured water flow rates in the dataset.

[0009] Specifically, in step S3, the historical residual sequence is calculated using the following formula: ; in, express t The residual value at time step, for t Actual measured water flow rate at any given time. for t Basic predicted water flow rate at any given time; The stationarity check specifically involves using the ADF unit root test method combined with a rolling window strategy to determine the stationarity of the historical residual sequence. When the P-value of the test output is less than a set threshold, the residual sequence is determined to be a stationary sequence; otherwise, the residual sequence is transformed into a stationary sequence by first-order differencing combined with trend term removal. The time-series dependency analysis specifically involves: calculating the autocorrelation function (ACF) and partial autocorrelation function (PACF) of the historical residual sequence, jointly determining the lag order d of the residual sequence, and using mutual information entropy (MI) for auxiliary verification to determine the coupling characteristics of long-term dependency and short-term fluctuation in the residual. The localization and classification employs a clustering-mutation point collaborative algorithm, including the following steps: Mutation point detection: An improved PELT algorithm is used to locate residual mutation points in the residual sequence corresponding to extreme floods. The loss function is: ; in, Weighting for hydrological scenarios; Clustering analysis: The historical residual sequence was classified using an adaptive K-means clustering algorithm. The number of clusters was set to K=3, corresponding to three categories: extreme flood residuals, normal fluctuation residuals, and random noise residuals. The clustering process dynamically initialized cluster centers based on the residual time series characteristics and hydrological scene labels, and the iteration termination condition was that the moving distance of the cluster center was less than a set threshold and the cluster purity was not less than 90%.

[0010] Specifically, in step S4, the prediction sub-model includes a GRU model, an LSTM model, and a GARCH model. The GRU model is adapted to extreme flood residuals with nonlinear and long-dependent characteristics; the LSTM model is adapted to conventional fluctuation residuals; and the GARCH model is used to capture the fluctuation clustering effect of the residuals. The prediction results of each prediction sub-model are fused using precision weighting to obtain the final residual prediction value. The fusion formula is as follows: , ; in, express t The final predicted residual value at time 10:00. Indicates the firstk The weights of each predictive sub-model. express t Time of the first k The residual prediction values ​​predicted by each prediction sub-model. Indicates the first k The Nash efficiency coefficients of each predictive sub-model on the validation set.

[0011] Preferably, in step S5, before fusion, the water level-flow conversion model is optimized using scene adaptive coefficients to provide an accurate initial reference for the basic LSTM prediction model, as shown in the following formula: ; in, The calculated water flow rate is a reference value. k These are the basic characteristic parameters of the watershed determined by fitting historical water level-discharge data. H Indicates water level value (unit: m). n This is a water level-discharge relationship index determined by fitting historical data; This is a scene adaptive correction function, based on terrain slope. S With rainfall intensity level Driven by the same principles, this approach adapts to the nonlinear changes in water level-flow relationship under different watershed scenarios.

[0012] Furthermore, for small watersheds in mountainous areas, the scene adaptive correction function is specifically optimized as follows: ; For urban flooding scenarios, the rainfall characteristics are corrected using the following formula: ; in, for t Time-corrected rainfall characteristic value for t The original rainfall observation value at that time; R This represents the percentage of the ground surface that is not permeable.

[0013] Specifically, in step S5, the fusion process adopts a dynamic fusion strategy, and the specific fusion formula is as follows: ; ; in, for t The final predicted water flow value at that time. for t The base forecast value at time, for t The predicted residual value at time step; To predict the confidence factor, the recent accuracy and stability of the residual prediction model are determined. This refers to the extreme flood risk level coefficient. This is an extreme correction option, which only takes effect when the flood risk level is ≥ the preset level. For safety reasons, These are the residual values ​​from historical extreme flood events; This is a time-lead time correction function, whose function value varies with the forecast lead time. It decreases as it increases.

[0014] Preferably, the prediction model is optimized and transferred using the following methods: A real-time feedback and iteration mechanism is constructed to collect forecast results and measured data daily to update the error sample library, and an incremental learning algorithm is used to update the model parameters periodically. For watersheds with scarce data, a transfer learning strategy is introduced. Based on a pre-trained model from a neighboring similar watershed, the parameters are fine-tuned using limited data from the local watershed to obtain a water flow prediction model adapted to the watershed with scarce data.

[0015] Compared with the prior art, the beneficial effects of the present invention are as follows: (1) This invention first ensures the core prediction capability and captures long-term dependence characteristics by incorporating an attention mechanism into the basic LSTM model; then, it performs in-depth analysis and classification of the prediction residuals, and adaptively fuses multiple models for specific predictions for different types of residuals; finally, it intelligently fuses the basic prediction value and the residual correction value, deeply mining the data features and the extreme event signals hidden in the residuals, thereby improving the forecast accuracy and the ability to capture extreme flood peaks while ensuring low-latency real-time response, effectively solving the core problems of insufficient accuracy, weak extreme event forecasting capability and difficulty in balancing real-time performance in the existing technology; (2) This invention utilizes the improved PELT algorithm combined with the scenario-weighted loss function to significantly improve the speed and accuracy of identifying residual mutation points corresponding to extreme floods; it also uses adaptive K-means clustering to accurately classify residuals, providing a clear scientific basis for the subsequent targeted construction of residual prediction models, making error correction more targeted. (3) The present invention adopts an adaptive fusion scheme of GRU, LSTM and GARCH models, which adapts the optimal sub-model to the residual type obtained by analysis and fuses them by precision weighting. This fully leverages the advantages of various models in predicting different residual characteristics, achieves accurate compensation for the basic prediction bias, and enhances the prediction ability for nonlinear and abrupt residuals. (4) The present invention designs a dynamic fusion scheme of “confidence-risk level dual drive”, which not only dynamically weights the real-time confidence of the basic prediction and residual prediction, but also introduces the risk level coefficient and extreme correction term linked to the recurrence period of extreme floods, realizing the intelligent adaptation of the forecast strategy and the real-time hydrological risk scenario, so that the model can maintain high accuracy under normal conditions and automatically enhance the amplitude and reliability of the early warning signal before the arrival of extreme floods, significantly improving the timeliness and accuracy of the early warning. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart illustrating a real-time water flow forecasting method based on LSTM time series prediction and residual analysis according to the present invention. Detailed Implementation

[0018] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0019] Reference Figure 1 This invention provides a real-time water flow forecasting method based on LSTM time series prediction and residual analysis, comprising the following steps: Step S1, Data Preprocessing: Collect historical multi-source time-series hydrological and meteorological data of the target watershed, perform preprocessing, and construct a standardized feature dataset; the multi-source time-series hydrological and meteorological data includes water flow sequences. (unit: Rainfall sequence (Unit: mm) Soil Moisture Sequence (Unit: %) Weather radar echo intensity sequence (Unit: dBZ) and rainband movement speed sequence (Unit: km / h) t This is a time variable that represents hourly time points.

[0020] Specifically, in this embodiment, the preprocessing includes outlier removal, missing value imputation, data standardization, and feature selection; To address the interference of impulsive outliers during flood season on model training, this embodiment introduces dynamic threshold adjustment. The outlier removal method uses a dynamic threshold adjustment factor. Improvements The criteria and specific judgment methods are as follows: like Then remove outliers. ; in, This represents the first in a certain type of hydrological and meteorological data. i Each sample value for The average value of the data series. for The standard deviation of the data series; The dynamic threshold adjustment factor is adaptively selected based on the historical flood frequency of the basin, with a value range of 0.8-1.2, and the upper limit of 1.2 is taken for flood-prone basins.

[0021] The missing data imputation employs a spatiotemporal collaborative strategy: for missing data in regular time periods, linear interpolation is used for imputation; for missing data during critical flood periods (especially in scenarios where radar data is missing), forward imputation combined with correction using data from neighboring stations is used; to balance the contribution of data from this station and data from neighboring stations, a weighting coefficient is introduced. (Values ​​range from 0.6 to 0.8), and priority should be given to ensuring the validity of the data from this site; neighboring sites must meet the screening criteria of a straight-line distance of no more than 5 kilometers and a terrain similarity of 0.6 or higher (values ​​range from 0 to 1) to ensure the accuracy of data correction.

[0022] To eliminate the impact of dimensional differences on model convergence, an adaptive Z-Score algorithm is used to standardize the data. For areas with dense extreme values, the mean and standard deviation are dynamically adjusted segmentally according to the flood season and non-flood season. The standardization formula is as follows: ; in, These are the standardized sample values. , They represent the first k The mean and standard deviation of each data segment. k =1 corresponds to the flood season. k = 2 corresponds to the non-flood period. This method can effectively avoid the excessive suppression of extreme features during the normalization process.

[0023] The feature selection method is as follows: calculate the water removal flow rate. External hydrological and meteorological characteristic variables x Corresponding water flow rate Mutual information entropy (Values ​​range from 0 to 1, used to quantify the degree of correlation between two variables), filter out those that meet the criteria. The feature variables (e.g., 0.3) constitute the final feature dataset. x To remove Other hydrological and meteorological variables, for , , , One of them.

[0024] Finally, the training set, validation set, and test set were divided in a ratio of 7:2:1, and the time step was set to 24-168 hours (24-72 hours for small watersheds and 72-168 hours for large watersheds) to construct multi-source feature input pairs, providing targeted and effective feature data for subsequent basic LSTM model training.

[0025] Step S2, Construction and training of the basic LSTM prediction model: Based on the standardized feature dataset, construct and train the basic LSTM prediction model; Specifically, in this embodiment, the basic LSTM prediction model is a lightweight model that incorporates an attention mechanism, which ensures prediction accuracy while taking into account computational efficiency, laying a reliable prediction foundation for subsequent residual analysis; the model structure includes, in sequence, an input layer, an attention layer, a two-layer LSTM hidden layer, a fully connected layer, and an output layer; The attention layer is used to calculate the contribution weights of each multi-source feature in the input to the water flow forecast result. , i Assign a feature type number, corresponding to the input multi-source features, and satisfy the normalization constraint: This enhances the ability to represent features related to extreme floods and effectively addresses the problem of insufficient attention to key features in traditional LSTM models. The number of neurons in the LSTM hidden layer is set to 64-256, which can be adjusted according to the amount of watershed data to ensure sufficient processing of various feature information. It captures the long-term dependency characteristics of hydrological sequences through a gating mechanism. The operations of the forget gate, input gate, cell state update, and output gate follow the core logic of temporal neural networks, and gating is achieved through the Sigmoid activation function (output range 0-1) and element-wise product operation. The weight matrix... , (dimensions are respectively) and ,in h This represents the number of neurons in the hidden layer. f (for input feature dimensions) and bias terms (The dimensions are consistent with the hidden layer dimensions) All dimensions are matched with the hidden layer dimensions, ensuring the model's fitting ability.

[0026] The basic LSTM prediction model is trained to maximize the validation set Nash efficiency coefficient NSE (ranging from -∞ to 1, with values ​​closer to 1 indicating higher prediction accuracy). The formula for calculating the Nash efficiency coefficient NSE is as follows: ; in, N The total number of samples in the validation set of the standardized feature dataset; for t Measured water flow rate at time (unit: ), for t Water flow rate model prediction at time (unit: ), To verify the average value of all measured water flow rates in the set (unit: During training, early stopping (by monitoring changes in the validation set NSE) is used to suppress overfitting, and hyperparameters such as learning rate and batch size are optimized to output preliminary prediction values. The inference time per sample is strictly controlled to be ≤2 minutes, laying an efficient foundation for subsequent real-time fusion correction.

[0027] Step S3, Residual Sequence Generation and Analysis: The standardized feature dataset is predicted using the basic LSTM prediction model to obtain the historical basic prediction sequence; the difference between the historical basic prediction sequence and the historical measured sequence is calculated to obtain the historical residual sequence, and the stationarity of the historical residual sequence is verified, time-series dependency analysis is performed, and the sequence is located and classified; the extreme flood characteristics hidden in the residual are mined through two-dimensional analysis to provide accurate modeling basis for the residual prediction model and make up for the prediction bias of the basic LSTM model. Specifically, in this embodiment, the historical residual sequence is calculated using the following formula: ; in, express t Residual value at time (unit: This reflects the deviation between the predicted values ​​of the basic model and the measured values. Its variation characteristics directly reveal the prediction defects of the basic model, especially the deviation performance under extreme flood conditions. for t Actual measured water flow rate at any given time. for t The basic predicted value of water flow at any given time.

[0028] Unlike traditional residual analysis, which only focuses on the overall error, this invention simultaneously records the hydrological scene labels (rainfall intensity level) corresponding to the residuals. The water level ratings are 1-5, with level 1 being light rain and level 5 being heavy rain; the water level ratings (H) are 1-3, with level 1 being a safe water level and level 3 being a warning water level. This provides a basis for scenario-based residual analysis.

[0029] The stationarity verification specifically involves using the ADF unit root test (which determines stationarity by outputting the P-value) combined with a rolling window strategy (the window size W is set to 1-3 hours, which can be adaptively adjusted according to the watershed confluence time) to determine the stationarity of the historical residual sequence. When the P-value output by the test is less than a set threshold (e.g., 0.05), the residual sequence is determined to be a stationary sequence; otherwise, the residual sequence is transformed into a stationary sequence by first-order differencing combined with trend term removal, effectively solving the problem of verification distortion in hydrological sequence abrupt change scenarios.

[0030] The time-series dependency analysis specifically involves: calculating the autocorrelation function (ACF) (range -1 to 1) and partial autocorrelation function (PACF) (range -1 to 1) of the historical residual sequence, jointly determining the lag order d of the residual sequence, and using mutual information entropy (MI) for auxiliary verification, thereby accurately locating the coupling characteristics of long-term dependency and short-term fluctuations in the residuals and clarifying the adaptation direction of the residual prediction model.

[0031] The localization and classification employs a clustering-mutation point collaborative algorithm, including the following steps: Mutation point detection: An improved PELT algorithm is used to locate residual mutation points in the residual sequence corresponding to extreme floods, enhancing the ability to identify mutation points in extreme scenarios. The loss function is: ; in, The weights for hydrological scenarios (ranging from 1.0 to 1.5, with 1.5 for extreme flood scenarios) reduce the mutation point location delay by more than 30% compared to the traditional PELT algorithm.

[0032] Cluster analysis: The historical residual sequence was classified using an adaptive K-means clustering algorithm, with K=3 clusters, corresponding to three categories: extreme flood residuals, regular fluctuation residuals, and random noise residuals. The clustering process dynamically initialized cluster centers based on residual time-series features and hydrological scene labels, and cluster centers were selected based on a minimum moving distance of less than a set threshold (e.g., ...). Furthermore, a cluster purity of no less than 90% is used as the termination condition for iteration, thereby achieving accurate classification of residuals and providing a basis for subsequent targeted residual prediction.

[0033] Step S4, Residual Prediction Model Construction and Training: For different types of residuals, select suitable prediction sub-models for fusion to construct a residual prediction model, improve the accuracy of residual prediction, and provide a reliable basis for bias compensation for the final fusion correction; use historical residual sequences and their corresponding multi-source feature data to train the residual prediction model. Specifically, in this embodiment, the prediction sub-model includes a GRU model, an LSTM model, and a GARCH model (generalized autoregressive conditional heteroscedasticity model). The GRU model is adapted to extreme flood residuals with nonlinear and long-dependent characteristics; the LSTM model is adapted to conventional volatile residuals; and the GARCH model is used to capture the volatile clustering effect of residuals. The three models work together to cover various residual characteristics. The GRU model simplifies the network structure through reset and update gates, improving computational efficiency while maintaining fitting ability to meet real-time forecasting requirements; the GARCH model satisfies the stationarity condition ( , and ,in For constant terms, For ARCH term coefficients, When the coefficients are GARCH terms (all are unitless parameters to be estimated), they can accurately characterize the memory and persistence of residual fluctuations.

[0034] Each branch model is trained using the maximization of NSE as the training criterion, and weights are dynamically allocated based on the prediction accuracy of each branch. The final residual prediction value is obtained through weighted fusion. The fusion formula is as follows: , ; in, express t Final residual prediction at time (unit: ), Indicates the first k The weights of each predictor sub-model (satisfying) ), express t Time of the first k The residual prediction values ​​of each prediction sub-model (unit: ), Indicates the first k The Nash efficiency coefficients of each prediction sub-model on the validation set; the reliability of the residual correction value after fusion is ensured through a precision weighting mechanism, providing accurate bias compensation for subsequent real-time fusion correction.

[0035] Step S5, Real-time Forecasting and Fusion Correction: Collect real-time hydrological and meteorological data of the target watershed up to the current time period, preprocess the data, and input it into the trained basic LSTM prediction model to obtain the basic prediction sequence for the future time period; calculate the latest residual sequence based on the recent basic prediction sequence and the measured sequence, input the latest residual sequence and real-time hydrological and meteorological data into the trained residual prediction model to obtain the residual prediction sequence for the corresponding future time period; fuse the basic prediction sequence for the future time period with the corresponding residual prediction sequence to obtain the final water flow forecast sequence.

[0036] In this step, based on the base forecast and residual forecast, real-time fusion correction is achieved through an edge-cloud collaborative architecture. This ensures forecast efficiency while improving accuracy, resulting in the final water flow forecast. The edge performs real-time data preprocessing (using the preprocessing algorithm from step S1, which, after pruning optimization, can complete outlier removal, spatiotemporal completion, and initial feature screening within one minute). Its computational efficiency is more than 40% higher than traditional solutions, while fully preserving the precursor features of extreme floods. The cloud employs a parallel model deployment strategy, deploying the base LSTM model and the residual prediction model on different computing nodes, simultaneously executing base and residual predictions, further reducing inference time.

[0037] Preferably, in this embodiment, before fusion, to address the nonlinear coupling problem of the water level-flow relationship, the water level-flow conversion model is optimized by introducing a scene adaptive coefficient, providing an accurate initial reference for the basic LSTM prediction model. The formula is as follows: ; in, The calculated water flow rate is a reference value (unit: ), k These are the basic characteristic parameters of the watershed determined by fitting historical water level-discharge data. H Indicates water level value (unit: m). n The water level-discharge relationship index (goodness of fit) determined by fitting historical data. ); This is a scene adaptive correction function (value range 0.8-1.2), based on terrain slope. S (Unit: degrees, extracted using GIS technology) and rainfall intensity level Driven by the same principle, it can effectively adapt to different watershed scenarios such as mountains, plains, and cities, thereby solving the problem of insufficient conversion accuracy of traditional fixed formulas.

[0038] Furthermore, for small watersheds in mountainous areas, supplementary topographic slope data should be provided. S The scene adaptive correction function is specifically optimized as follows: ; The optimized water level-flow conversion formula is: ; Among them, 0.01 is the slope correction coefficient, which reflects the influence of terrain slope on water flow velocity; the scene adaptive correction function considers the influence of terrain slope and rainfall intensity level in general scenarios, but only considers the influence of terrain slope and ignores the influence of rainfall intensity level in mountainous small watershed scenarios.

[0039] For urban flooding scenarios, the percentage of impermeable ground surface area has been supplemented. R (Unit: %) parameters, and embedding the impermeable area factor into the rainfall characteristics to correct the rainfall characteristics, thereby improving the forecast accuracy for urban scenarios; the correction formula is: ; in, for t Time-corrected rainfall characteristic value for t The original rainfall observation value at that time.

[0040] Meanwhile, the hyperparameters of the basic model are dynamically adjusted according to the watershed size, and the residual fusion stage is adjusted according to the proportion of residual types. Optimize branch weights: when the residual from extreme floods accounts for more than 30%, increase the GRU branch weight to 0.5 or higher; when the residual fluctuation clustering effect is significant, strengthen the GARCH branch weight to continuously adapt to changes in the hydrological characteristics of the watershed.

[0041] Specifically, in this embodiment, to adapt to the dynamic characteristics of extreme flood scenarios, a "confidence level-risk level dual-driven" weight fusion strategy is designed, and the fusion formula is optimized to achieve dynamic adaptation between basic prediction and residual correction. The specific fusion formula is as follows: ; ; in, for t Predicted final water flow rate at time (unit: ), for t The base forecast value at time, for t The predicted residual value at time step; To predict the confidence factor, it is determined based on the recent accuracy and stability of the residual prediction model (the value ranges from 0 to 1, calculated based on the rolling accuracy and real-time error stability of the residual model over the past 30 days; the closer it is to 1, the higher the confidence of the residual model). The extreme flood risk level coefficient is adjusted by a weighting factor of 1.0-1.8 based on the return period of 1-5, with higher weights for higher risks. Extreme correction term (unit: This only applies when the flood risk level is ≥ the preset level. For safety factors (values ​​ranging from 0.9 to 1.1), These are the residual values ​​from historical extreme flood events; This is a time-lead time correction function, whose function value varies with the forecast lead time. (Unit: hours) decreases as it increases, with a value range of 0.8-1.0, to compensate for the cumulative residual bias of long-term forecasts.

[0042] The fusion results were inversely normalized to restore the actual water flow units, and NSE and RMSE (root mean square error, unit: ...) were calculated. With three accuracy indicators—MAPE (mean absolute percentage error, unit: %), extreme peak capture accuracy, and strict control of response delay within 5 minutes throughout the process, it provides precise guidance for flood control emergency response in a three-in-one manner, encompassing accuracy, timeliness, and risk.

[0043] Preferably, the steps in this embodiment further include: Step S6, Model Optimization and Transfer: The prediction model is optimized and transferred using the following methods: A real-time feedback iteration mechanism is constructed to collect forecast results and observed data daily to update the error sample library, and an incremental learning algorithm is used to periodically update model parameters. Model performance is continuously optimized based on the deviation between daily forecast results and observed data to ensure the stability of long-term forecast accuracy while meeting the application needs of data-scarce watersheds. Daily forecast results and corresponding observed data are collected to update the error sample library, retaining valid data for the past 90 days and removing data with error values ​​greater than [a certain threshold]. ( Standard deviation of the error series in the sample library, unit: ( ) abnormal samples, in order to avoid abnormal data interfering with model optimization.

[0044] The model parameters are updated weekly using an incremental learning algorithm, and training is performed only on samples added in the last 7 days. A momentum term m (unitless, with a value of 0.9) is introduced to accelerate the convergence process. The weight matrix W and the bias term b are adjusted iteratively based on the gradient, thereby avoiding the efficiency loss caused by retraining on full data, so as to meet the dynamic optimization requirements of real-time forecasting.

[0045] For watersheds with scarce data (measured data duration < 3 years), a transfer learning strategy is introduced. The model is pre-trained on massive amounts of data from neighboring similar watersheds (hydrological feature similarity ≥ 0.7, unitless), and then fine-tuned using limited data from the local watershed (duration ≥ 1 year). During fine-tuning, the parameters of the LSTM / GRU bottom feature extraction layer are frozen, and only the parameters of the top fully connected layer and attention layer are updated, thereby reducing the dependence on long-sequence data from the local watershed.

[0046] To verify the feasibility and superiority of the method of this invention, the specific implementation process of the method is further explained below using a practical case of flood season flow forecasting in a small mountain watershed. This watershed has complex terrain, a short flood confluence time (approximately 1-2 hours), and is frequently affected by localized heavy rainfall ("sudden downpours"), leading to frequent extreme floods. Traditional forecasting methods in this watershed often have peak flow prediction errors exceeding 20% ​​and response delays exceeding 10 minutes. The specific steps of implementing the method of this invention are as follows: Step 1, Data Preprocessing: Hourly hydrological and meteorological data for the basin from 2018 to 2025 were collected, including water flow series. Rainfall sequence Soil moisture sequence Weather radar echo intensity sequence and rainband movement speed sequence And extract the terrain slope using GIS. S = 15° (accuracy ≥ 1°).

[0047] Outlier removal: Given the frequent flooding in this watershed, a dynamic threshold adjustment factor is set. Based on improvements The criteria exclude pulse anomalies during the flood season.

[0048] Missing value imputation: For periods of missing radar data, forward imputation (weighting coefficients) is used. The data was then corrected by combining it with data from three neighboring stations (straight-line distance ≤ 5 km, terrain similarity ≥ 0.7).

[0049] Standardization: The process is segmented into flood and non-flood periods, and an adaptive Z-Score algorithm is used. In this case, the flood period parameter is set as follows: , .

[0050] Feature selection: Selected features are obtained through mutual information entropy (MI) calculation. The four core features: , , , .

[0051] Dataset Construction: The training, validation, and test sets were divided in a 7:2:1 ratio. The time step was set to T=72 hours (to match the watershed's confluence characteristics). Features from the first 72 hours and the subsequent 1-6 hours were constructed. The sample pairs were used to enhance the initial attention weight of soil moisture features (set to 0.3).

[0052] Step 2, Building and training the basic LSTM prediction model: An attention-enhanced LSTM model was constructed with an input layer dimension of 4, an output dimension of 32 for the attention layer, 128 neurons in each of the two LSTM hidden layers, a fully connected layer dimension of 64, and an output layer dimension of 6 (corresponding to hourly forecasts from 1 to 6 hours).

[0053] Using the Adam optimizer, initial learning rate The batch size is B=64. The training is performed with the goal of maximizing the NSE on the validation set and early stopping is used (the training terminates when the NSE fluctuates by less than 0.001 for 20 consecutive rounds).

[0054] After training, the validation set metrics were: NSE = 0.86, RMSE = 8.2. The single-sample inference time is 1.2 minutes, which meets the real-time requirements.

[0055] Step 3, Residual Sequence Generation and Analysis: Calculate the historical residual sequence based on the historical baseline prediction sequence and the historical measured sequence; Stationarity check: The ADF test p-value was 0.008 (< 0.05 significance level), indicating that the residual series is stationary.

[0056] Temporal dependency analysis: The lag order d=12 was determined using ACF and PACF.

[0057] Mutation point detection and classification: The improved PELT algorithm was used to detect residual mutation points, which highly coincided with the peak times of three extreme floods on July 15, 2021, August 2, 2023, and June 18, 2024. Further K-means clustering (K=3) yielded the residual classification percentages as follows: extreme flood residuals 28%, normal fluctuation residuals 57%, and random noise residuals 15%.

[0058] Step 4, Residual Prediction Model Construction and Training: Construct a fusion residual prediction model of GRU, LSTM and GARCH(1,1) to adapt to the three types of residuals respectively; The GARCH(1,1) parameter is calibrated as follows: , , (satisfy Each branch model is trained using the same training parameters as the base LSTM prediction model. The validation set NSEs after training for each branch are as follows: GRU branch 0.82, LSTM branch 0.88, and GARCH branch 0.75. Precision-weighted fusion: Weighting each branch according to its precision. , , The RMSE of the fused residual model on the validation set was 2.1. Accurately capture residual patterns.

[0059] Step 5, Real-time Forecasting and Fusion Correction: Deploy an edge-cloud collaborative architecture, where real-time data preprocessing is completed at the edge (≤1 minute), and basic prediction and residual prediction are executed in parallel in the cloud.

[0060] Taking the extreme flood event of June 18, 2024 as an example: Baseline forecast (Exceeding the warning threshold by 200) ).

[0061] Real-time parameters: confidence factor Risk level coefficient (Corresponding to Level 4 flood), extreme correction item ( The historical extreme residual maximum value is 18.9. Forecast timeframe Hour( ).

[0062] Substitute into the dynamic fusion formula: ; Actual measured data shows that the actual water flow rate at that moment was 190.2. The prediction error is only 1.3%.

[0063] A total of 148 real-time forecasts were conducted throughout the 2024 flood season (June-August), with overall indicators of NSE=0.92 and RMSE=5.3. The MAPE was 4.8%, and the forecast error for the peak values ​​of the three extreme floods was controlled within 8%. The average response delay was 3.2 minutes, which was about 40 minutes earlier than the warning from the traditional model.

[0064] Step 6, Model Optimization: The error sample database is updated daily, and incremental learning is performed weekly (momentum term m=0.9, learning rate). The model parameters were updated based on the new samples (approximately 168 groups) added in the past 7 days.

[0065] After iterative optimization, the NSE of the base model validation set improved to 0.88, and the RMSE of the residual model decreased to 1.9. The model maintained stable accuracy and adaptability in subsequent flood forecasts.

[0066] This embodiment demonstrates that the method of the present invention can significantly improve the accuracy and timeliness of short-term forecasts in complex mountainous watersheds and scenarios with frequent extreme floods, verifying the effectiveness and engineering applicability of the technical solution.

[0067] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A real-time water flow forecasting method based on LSTM time series prediction and residual analysis, characterized in that, Includes the following steps: S1, Data Preprocessing: Collect historical multi-source time-series hydrological and meteorological data of the target watershed, perform preprocessing, and construct a standardized feature dataset; S2, Construction and training of basic LSTM prediction model: Based on the standardized feature dataset, construct and train the basic LSTM prediction model; S3, Residual Sequence Generation and Analysis: The basic LSTM prediction model is used to predict the standardized feature dataset to obtain the historical basic prediction sequence; the difference between the historical basic prediction sequence and the historical measured sequence is calculated to obtain the historical residual sequence, and the historical residual sequence is subjected to stationarity verification, time-series dependency analysis, localization and classification. S4, Residual Prediction Model Construction and Training: For different types of residuals, select suitable prediction sub-models for fusion to construct a residual prediction model; use historical residual sequences and their corresponding multi-source feature data to train the residual prediction model; S5, Real-time Forecasting and Fusion Correction: Real-time hydrological and meteorological data of the target watershed up to the current time period are collected, preprocessed, and input into a trained basic LSTM prediction model to obtain the basic prediction sequence for the future time period; the latest residual sequence is calculated based on the recent basic prediction sequence and the measured sequence, and the latest residual sequence and real-time hydrological and meteorological data are input into a trained residual prediction model to obtain the residual prediction sequence for the corresponding future time period; the basic prediction sequence for the future time period is fused with the corresponding residual prediction sequence to obtain the final water flow forecast sequence.

2. The real-time water flow forecasting method based on LSTM time series prediction and residual analysis as described in claim 1, characterized in that, In step S1, the preprocessing includes outlier removal, missing value imputation, data standardization, and feature selection. The outlier removal method employs a dynamic threshold adjustment factor. Improvements The criteria and specific judgment methods are as follows: like Then remove outliers. ; in, This represents the first in a certain type of hydrological and meteorological data. i The multi-source time-series hydro-meteorological data includes at least a water flow sequence. Rainfall sequence Soil moisture sequence Weather radar echo intensity sequence Rainband movement speed sequence Multiple types of data; for The average value of the data series. for The standard deviation of the data series; This is a dynamic threshold adjustment factor that adaptively selects values ​​based on the historical flood frequency of the basin. The missing value imputation adopts a spatiotemporal collaborative strategy: for missing data in regular time periods, linear interpolation is used for imputation; for missing data in critical flood periods, forward imputation combined with correction using data from neighboring stations is used for imputation. The data standardization employs an adaptive Z-Score algorithm, segmenting the data for flood and non-flood periods. The standardization formula is as follows: ； in, These are the standardized sample values. , They represent the first k The mean and standard deviation of each data segment. k = 1 corresponds to the flood season. k = 2 corresponds to the non-flood season; The feature selection method is as follows: calculate the water removal flow rate. External hydrological and meteorological characteristic variables x Corresponding water flow rate Mutual information entropy Filter out those that meet the requirements The feature variables constitute the final feature dataset.

3. The real-time water flow forecasting method based on LSTM time series prediction and residual analysis as described in claim 1, characterized in that, In step S2, the basic LSTM prediction model is a lightweight model that incorporates an attention mechanism. The model structure includes, in sequence, an input layer, an attention layer, a two-layer LSTM hidden layer, a fully connected layer, and an output layer. The attention layer is used to calculate the contribution weights of each multi-source feature in the input to the water flow forecast result. , i Number the feature type; The LSTM hidden layer captures the long-period dependence characteristics of hydrological sequences through a gating mechanism, and achieves gating adjustment through the Sigmoid activation function and element-wise product operation; The basic LSTM prediction model is trained to maximize the validation set Nash efficiency coefficient NSE. The formula for calculating the Nash efficiency coefficient NSE is as follows: ； in, N The total number of samples in the validation set of the standardized feature dataset; for t Actual measured water flow rate at any given time. for t Water flow rate model prediction at time 10:00 To verify the average value of all measured water flow rates in the dataset.

4. The real-time water flow forecasting method based on LSTM time series prediction and residual analysis as described in claim 1, characterized in that, In step S3, the historical residual sequence is calculated using the following formula: ； in, express t The residual value at time step, for t Actual measured water flow rate at any given time. for t Basic predicted water flow rate at any given time; The stationarity check specifically involves using the ADF unit root test method combined with a rolling window strategy to determine the stationarity of the historical residual sequence. When the P-value of the test output is less than a set threshold, the residual sequence is determined to be a stationary sequence; otherwise, the residual sequence is transformed into a stationary sequence by first-order differencing combined with trend term removal. The time-series dependency analysis specifically involves: calculating the autocorrelation function (ACF) and partial autocorrelation function (PACF) of the historical residual sequence, jointly determining the lag order d of the residual sequence, and using mutual information entropy (MI) for auxiliary verification to determine the coupling characteristics of long-term dependency and short-term fluctuation in the residual. The localization and classification employs a clustering-mutation point collaborative algorithm, including the following steps: Mutation point detection: An improved PELT algorithm is used to locate residual mutation points in the residual sequence corresponding to extreme floods. The loss function is: ； in, Weighting for hydrological scenarios; Clustering analysis: The historical residual sequence was classified using an adaptive K-means clustering algorithm. The number of clusters was set to K=3, corresponding to three categories: extreme flood residuals, normal fluctuation residuals, and random noise residuals. The clustering process dynamically initialized cluster centers based on the residual time series characteristics and hydrological scene labels, and the iteration termination condition was that the moving distance of the cluster center was less than a set threshold and the cluster purity was not less than 90%.

5. The real-time water flow forecasting method based on LSTM time series prediction and residual analysis as described in claim 1, characterized in that, In step S4, the prediction sub-model includes a GRU model, an LSTM model, and a GARCH model. The GRU model is adapted to extreme flood residuals with nonlinear and long-dependent characteristics; the LSTM model is adapted to conventional fluctuation residuals; and the GARCH model is used to capture the fluctuation clustering effect of the residuals. The prediction results of each prediction sub-model are fused using precision weighting to obtain the final residual prediction value. The fusion formula is as follows: ， ； in, express t The final predicted residual value at time step [time]. Indicates the first k The weights of each predictive sub-model. express t Time of the first k The residual prediction values ​​predicted by each prediction sub-model. Indicates the first k The Nash efficiency coefficients of each predictive sub-model on the validation set.

6. The real-time water flow forecasting method based on LSTM time series prediction and residual analysis as described in claim 1, characterized in that, In step S5, before fusion, the water level-flow conversion model is optimized using scene adaptive coefficients to provide an accurate initial reference for the basic LSTM prediction model. The formula is as follows: ； in, The calculated water flow rate is a reference value. k These are the basic characteristic parameters of the watershed determined by fitting historical water level-discharge data. H Indicates water level value (unit: m). n This is a water level-discharge relationship index determined by fitting historical data; This is a scene adaptive correction function, based on terrain slope. S With rainfall intensity level Driven by the same principles, this approach adapts to the nonlinear changes in water level-flow relationship under different watershed scenarios.

7. The real-time water flow forecasting method based on LSTM time series prediction and residual analysis as described in claim 6, characterized in that, For small watersheds in mountainous areas, the scene adaptive correction function is specifically optimized as follows: ； For urban flooding scenarios, the rainfall characteristics are corrected using the following formula: ； in, for t Time-corrected rainfall characteristic value for t The original rainfall observation value at that time; R This represents the percentage of the ground surface that is not permeable.

8. The real-time water flow forecasting method based on LSTM time series prediction and residual analysis as described in claim 1, characterized in that, In step S5, the fusion process employs a dynamic fusion strategy, and the specific fusion formula is as follows: ； ； in, for t The final predicted water flow value at that time. for t The base forecast value at time, for t The predicted residual value at time step; To predict the confidence factor, the recent accuracy and stability of the residual prediction model are determined. This refers to the extreme flood risk level coefficient. This is an extreme correction option, which only takes effect when the flood risk level is ≥ the preset level. For safety reasons, These are the residual values ​​from historical extreme flood events; This is a time-lead time correction function, whose function value varies with the forecast lead time. It decreases as it increases.

9. The real-time water flow forecasting method based on LSTM time series prediction and residual analysis as described in claim 1, characterized in that, The prediction model is optimized and transferred using the following methods: A real-time feedback and iteration mechanism is constructed to collect forecast results and measured data daily to update the error sample library, and an incremental learning algorithm is used to update the model parameters periodically. For watersheds with scarce data, a transfer learning strategy is introduced. Based on a pre-trained model from a neighboring similar watershed, the parameters are fine-tuned using limited data from the local watershed to obtain a water flow prediction model adapted to the watershed with scarce data.

Images