A runoff prediction model considering extreme runoff and a construction method and system thereof
By using feature screening and model optimization, and combining XGBoost, SHAP and BO-BiLSTM models, a runoff forecasting model constructed using historical hydrological and meteorological factors and teleconnection climate factors has been developed. This has solved the problem of low accuracy in runoff forecasting in existing technologies and enabled accurate forecasting of long-term and extreme runoff.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- HUAZHONG UNIV OF SCI & TECH
- Filing Date
- 2023-09-22
- Publication Date
- 2026-06-26
AI Technical Summary
Existing technologies for runoff forecasting suffer from low long-term forecast accuracy and large deviations in extreme runoff forecasts. They are unable to accurately characterize the mathematical and physical relationship between runoff and climate elements, and the application of teleconnection climate factors varies in different regions.
The feature screening method based on XGBoost and SHAP models was used to select the forecasting factors with the highest contribution. The conventional model and the extreme value model were constructed by combining the BO-BiLSTM model. The comprehensive runoff forecasting model was formed by optimizing the contribution and fit. The forecast was made using historical hydrological and meteorological data and teleconnection climate factors.
It improves the accuracy and reliability of runoff forecasting, effectively takes into account both long-term runoff and extreme flood forecasting, has a wide range of applications, and the model does not rely on future information, making it suitable for practical long-term monthly forecasts.
Smart Images
Figure CN117391131B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of hydrological forecasting technology, and more specifically, relates to a runoff forecasting model that considers extreme runoff and its construction method and system. Background Technology
[0002] Water resources are an essential resource for human survival and development, and high-precision runoff forecasting is of great reference value for water resource planning, management, and intensive utilization. However, accurate long-term runoff forecasting has always been a research hotspot in this field. This is because runoff changes are the result of the combined effects of many factors, and their mechanisms are extremely complex. Furthermore, current technologies and forecasting methods still have certain limitations, failing to clearly characterize the mathematical and physical relationships between various factors and runoff. Therefore, integrating deep learning models to explore runoff driving mechanisms and perform long-term forecasting is particularly important.
[0003] Runoff is one of the most critical processes in the global water cycle. Currently, climate and human activities are considered the main factors influencing runoff changes. Quantifying human activities is complex and lacks a unified understanding; therefore, most current runoff forecasting studies select climate elements as independent variables in the runoff forecasting system. For the selected climate variables, measured meteorological data from the watershed are the primary source. While these variables are widely used, they suffer from problems such as varying data lengths across regions and inaccessibility in areas without data. Teleconnected climate factor indices can effectively address these issues, and the generation and changes of long-term runoff are often more closely related to these global climate indices. From a causal perspective, teleconnected climate factors are more suitable for long-term runoff forecasting.
[0004] The formation mechanism of long-term runoff is complex and difficult to describe directly using mathematical and physical relationships. Deep learning models are black-box models that can directly generate results without considering the process. In recent years, this technology has been widely used in the field of runoff forecasting; however, it also has problems such as low accuracy in long-term runoff prediction models and large deviations in extreme runoff forecasts. Summary of the Invention
[0005] In view of the above-mentioned defects or improvement needs of the existing technology, the present invention provides a runoff forecasting model that takes into account extreme runoff, as well as its construction method and system. The purpose is to effectively take into account both long-term runoff forecasting and future extreme flood forecasting, and improve the accuracy and reliability of forecasting.
[0006] To achieve the above objectives, according to a first aspect of the present invention, a method for constructing a runoff forecasting model that considers extreme runoff is proposed, comprising the following steps:
[0007] S1. Collect and preprocess historical hydrological and meteorological data and teleconnection climate factor data of the study area to obtain multiple forecast factors and form a forecast factor set;
[0008] S2. Determine the contribution of each forecast factor and initially screen out the N forecast factors with the highest contribution; sort these N forecast factors from high to low contribution to obtain the key forecast factor set.
[0009] S3. Input the first N, the first N-1, ..., the first 1 forecast factors in the set of key forecast factors into the preliminary forecast model for training and testing, thereby obtaining N forecast schemes. Each forecast scheme includes the input forecast factors and the corresponding trained preliminary forecast model.
[0010] For N forecast schemes, calculate their Nash efficiency coefficients during testing, and select the set of forecast schemes with the highest Nash efficiency coefficients as the conventional model;
[0011] Meanwhile, for N forecast schemes, within the range exceeding the runoff extreme threshold, the forecast runoff and actual runoff output by the preliminary forecast model during the test are fitted respectively, and the set of forecast schemes with the best fit is selected as the extreme value model.
[0012] S4. Couple the conventional model and the extreme value model to obtain a comprehensive runoff forecasting model.
[0013] As a further preferred embodiment, the contribution of each forecast factor is determined by a feature screening model; the feature screening model includes an XGBoost model and a SHAP model, wherein the XGBoost model is used to predict runoff based on the forecast factors, and the XGBoost model is trained using the forecast factor set; the SHAP model is used to calculate the contribution of each forecast factor to the runoff prediction result based on the runoff prediction result output by the XGBoost model.
[0014] As a further preferred option, the preliminary forecast model is constructed based on the BO-BiLSTM model.
[0015] As a further preferred embodiment, the optimization parameters in the BO-BiLSTM model include L2 regularization terms.
[0016] As a further preferred method, the method for determining the runoff extreme threshold is as follows: calculate the mean and standard deviation of historical runoff data, and according to the 3σ principle, take the sum of the mean and three times the standard deviation as the runoff extreme threshold.
[0017] As a further preferred embodiment, the historical hydrological and meteorological data includes monthly precipitation, monthly evapotranspiration, and monthly runoff; the teleconnection climate factor data includes atmospheric circulation factors, sea surface temperature index, and other relevant indices.
[0018] As a further preferred embodiment, the preprocessing includes: obtaining initial forecast factors based on historical hydrological and meteorological data and teleconnection climate factor data; deleting and interpolating forecast factors according to time-related missing data; and then performing time-delay processing on the forecast factors to form a forecast factor set.
[0019] According to a second aspect of the present invention, a system for constructing a runoff forecasting model that considers extreme runoff is provided, including a processor for executing the above-described method for constructing a runoff forecasting model that considers extreme runoff.
[0020] According to a third aspect of the present invention, a runoff forecasting model is provided, characterized in that it is constructed using the above-described method for constructing a runoff forecasting model that considers extreme runoff.
[0021] According to a fourth aspect of the present invention, a runoff forecasting method based on the above-mentioned runoff forecasting model is provided, characterized by comprising the following steps: acquiring each forecasting factor of the study area in real time, and inputting the corresponding forecasting factors into the conventional model and the extreme value model respectively to obtain conventional runoff data and extreme runoff data.
[0022] In summary, compared with the prior art, the above-described technical solutions conceived by this invention mainly possess the following technical advantages:
[0023] 1. This invention performs initial screening of forecasting factors according to their contribution, and then uses different standards to construct conventional models and extreme value models to obtain a comprehensive runoff forecasting model. This model can effectively take into account both real-time long-term runoff forecasting and future extreme flood forecasting, thereby improving forecast accuracy and reliability and having a wide range of applications.
[0024] 2. This invention combines the XGBoost model and the SHAP model to obtain the contribution of each forecasting factor to the runoff forecast results, thereby selecting forecasting factors with higher contributions for subsequent analysis, which can improve the efficiency of model construction and the accuracy of the final forecast model.
[0025] 3. The model of this invention is based on historical watershed measured data and teleconnection climate factors to predict runoff, without involving any future information, and can be used for actual long-term monthly forecasts. Attached Figure Description
[0026] Figure 1 A flowchart illustrating the method for constructing a runoff forecasting model considering extreme runoff in an embodiment of the present invention;
[0027] Figure 2 This is a forecast factor-driven graph according to an embodiment of the present invention;
[0028] Figure 3 This is a measured-predicted flow process line according to an embodiment of the present invention. Detailed Implementation
[0029] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.
[0030] This invention provides a method for constructing a runoff forecasting model that considers extreme runoff, such as... Figure 1 As shown, it includes the following steps:
[0031] S1. Collect and preprocess historical hydrological and meteorological data and teleconnection climate factor data of the study area to obtain multiple forecast factors and form a forecast factor set; the forecast factor set is divided into training period and test period according to the time series length of historical hydrological and meteorological data.
[0032] Furthermore, the preprocessing includes deleting forecast factors with many missing values, imputing forecast factors with few missing values, and performing time-delay processing on the forecast factors to form a forecast factor set.
[0033] Furthermore, historical hydrological and meteorological data include monthly precipitation, monthly evapotranspiration, and monthly runoff; teleconnection climate factor data include atmospheric circulation factors, sea surface temperature index, and other related indices.
[0034] Furthermore, the forecast factor set after time lag processing includes monthly precipitation, monthly evapotranspiration, and monthly runoff with time lags of 1 to 3 months, and teleconnection climate factor set with time lags of 1 to 12 months, with the output being the runoff for the current month.
[0035] S2. Determine the contribution of each forecast factor through the feature screening model, and obtain the N forecast factors with the highest contribution (preferably N≥20); sort the N forecast factors from high to low contribution to obtain the key forecast factor set.
[0036] Furthermore, the feature screening model is constructed based on the XGBoost-SHAP model, where the XGBoost model is used to predict runoff based on forecast factors, and the SHAP model is used to calculate the contribution of each forecast factor to the runoff prediction results.
[0037] The main principles of the XGBoost model are as follows:
[0038]
[0039] In the formula: Let x be the predicted value for the i-th sample, k be the number of regression trees, and x be the predicted value for the i-th sample. iLet a(x) be the feature vector set of the i-th sample, F be the function space of the regression tree, and a(x) be the feature vector set of the i-th sample. i ) is the corresponding x i The index function mapped to the leaf node. Let be the weight parameters for the i-th sample.
[0040] The main principles of the SHAP model are as follows:
[0041]
[0042]
[0043] In the formula: F represents the sample x i The complete set of all influencing factors, J represents the sample x. i Let ν(J) be the contribution of the combined effect of any subset of influencing factors in subset J, and ν(J∪{i,j})-ν(J) be the contribution of influencing factor j to this combined effect; base This serves as the baseline for the entire model.
[0044] Furthermore, the XGBoost model is pre-trained using a set of predictor factors and its parameters are optimized using 10-fold cross-validation.
[0045] S3. Input the top N, top N-1, ..., top 1 forecast factors from the key forecast factor set into the preliminary forecast model for training and testing. This can be understood as training and testing the forecast factors selected in S2 N times, eliminating them one by one in order of increasing contribution each time. That is, the first time all N forecast factors are input, the second time the forecast factor with the smallest contribution is eliminated, and the remaining N-1 forecast factors are input, and so on. The Nth time only the top 1 forecast factor with the largest contribution is input. This results in N sets of training and testing results (forecast schemes), each set containing selected input forecast factors and the corresponding trained preliminary forecast model.
[0046] For the above N sets of training and testing results, two judgments are made respectively:
[0047] (1) Using the Nash efficiency coefficient (NSE) during the test period as the standard, select the forecast scheme with the highest NSE during the test period as the conventional model.
[0048] (2) Within the range exceeding the extreme threshold of runoff, the predicted runoff and actual runoff output by the preliminary forecast model are fitted, and the best-fit forecast scheme is selected as the extreme value model.
[0049] Furthermore, the method for determining the extreme threshold of runoff is as follows: based on historical runoff data, according to the 3σ principle, the sum of the mean and three times the standard deviation is taken as the extreme threshold of runoff.
[0050] Furthermore, the preliminary forecast model is built on the BO-BiLSTM model, which optimizes the hyperparameters of the BiLSTM model using the Bayesian optimization algorithm (BO).
[0051] Furthermore, the BO-BiLSTM model is improved by adding an L2 regularization term to the hyperparameters to be optimized in the BiLSTM model. In this embodiment, the names and value ranges of the hyperparameters of the BO-BiLSTM model are shown in Table 1.
[0052] Table 1. Range of values for parameters to be optimized
[0053] Parameter name Parameter meaning Range of values Learning rate Base learning rate [0.001,0.1] lstmLayer Number of LSTM neurons [20,200] L2 L2 regularization term [0.00001,0.1] MaxEpochs Maximum number of training iterations [20,200]
[0054] S4. Couple the conventional model and the extreme value model to obtain a comprehensive runoff forecasting model.
[0055] S5. Utilize the established runoff forecasting model and scheme to make forecasts and evaluate the model's accuracy; introduce the high flow rate evaluation index KGE, and the calculation methods for each evaluation index are as follows:
[0056]
[0057]
[0058] In the formula: Q o (t) represents the measured monthly runoff, in meters. 3 Q p (t) represents the predicted monthly runoff, in meters. 3 n is the number of months in the testing period; r is the correlation coefficient; α is the ratio of the standard deviation of the predicted monthly runoff to the measured monthly runoff; β is the ratio of the predicted monthly runoff to the average measured monthly runoff.
[0059] When using the runoff forecasting model obtained through the above method for runoff forecasting, the process includes: acquiring each forecasting factor for the study area in real time, inputting the corresponding forecasting factors into the conventional model and the extreme value model respectively, to obtain conventional runoff data and extreme runoff data. When the obtained extreme runoff data is less than the runoff extreme threshold, the conventional runoff data is used as the runoff forecast result; when the obtained extreme runoff data is not less than the runoff extreme threshold, the extreme runoff data is used as the runoff forecast result.
[0060] The model of this invention has the characteristics of solid theoretical foundation, clear physical action process mechanism, and ability to predict extreme floods. It is suitable for widespread application and provides technical support for water resource planning and rational allocation.
[0061] The following are specific examples:
[0062] For the upper reaches of the Han River, a runoff forecasting model is constructed. The construction method includes the following steps:
[0063] S1. Data collection and preprocessing;
[0064] Hydrological and meteorological data for the basin from January 1987 to December 2020 were collected, including monthly precipitation data, monthly pan evaporation observations, and monthly runoff data. Monthly pan evaporation observations were used as monthly evapotranspiration data. Teleconnection climate factor data from January 1986 to December 2020 were also collected, including 88 atmospheric circulation factors, 26 sea surface temperature indices, and 16 other indices.
[0065] Missing data in the teleconnection climate factor dataset were processed, including deleting and imputing missing values. This resulted in 79 atmospheric circulation factors, 26 sea surface temperature indices, and 9 other indices. Furthermore, the extreme runoff threshold of 4050 m for historical runoff sequences was determined based on the 3σ principle. 3 / s, and time lag processing was applied to all forecast factors. The settings were as follows: the time lag for precipitation, evapotranspiration and runoff data was set to 1-3 months, and the time lag for each index of teleconnection factors was set to 1-12 months, respectively. A total of 1371 forecast factor features were generated.
[0066] S2. Construct the XGBoost-SHAP model and perform initial screening of forecast factors;
[0067] The XGBoost-SHAP model algorithm program was developed based on the Python 3.7 platform.
[0068] The machine learning algorithm used for feature selection employs an approximate 8:2 ratio between its training and test sets. The training period (February 1987 to February 2014, accumulating 325 months) and the test period (March 2014 to December 2020, accumulating 82 months) are used. The predicted factor set is input into the XGBoost-SHAP model for initial feature screening, selecting the top 20 predicted factors by contribution. The initial screening results are shown in Table 2. The SHAP model then generates... Figure 2 ,according to Figure 2 It can analyze the driving mechanism of forecast factors on runoff.
[0069] Table 2 Predicted Factor Information After Initial Screening
[0070] Forecasting factors Time lag (months) Western Pacific subtropical high ridge position index 5 North American subtropical high ridge position index 12 North American polar vortex intensity index 2 precipitation 1 Western Pacific Teleconnection Index 7 Northern Hemisphere Subtropical High Area Index 1 Western Pacific subtropical high ridge position index 9 run-off 1 North American polar vortex area index 7 Northern Hemisphere Subtropical High Northern Boundary Location Index 6 Eastern Pacific subtropical high ridge position index 1 Asian polar vortex area index 12 North Pacific subtropical high ridge position index 11 200hPa Zonal Wind Index of the Central and Eastern Equatorial Pacific 5 Western Pacific subtropical high intensity index 1 ENSO-like index 12 Oyashio Sea Temperature Index 12 30hPa Zonal Wind Index 2 Solar radiation flux index 1 North Atlantic-European Circulation W-type Index 5
[0071] S3. Construct a BO-BiLSTM model to determine the conventional model and the extreme value model;
[0072] Write the BO-BiLSTM model algorithm program, with the same ratio of training and testing phases in the model as above.
[0073] Conventional model: Factors were eliminated one by one after the initial screening. When the number of forecast factors was 9, the NSE was the highest during the model test period. The model with this combination of forecast factors was selected as the conventional model.
[0074] Extreme value model: After repeated elimination of factors after initial screening, when the forecast factor is 19, although the overall NSE of the model during the test period is low, the model fits the extreme values well. The model under this combination of forecast factors is selected as the extreme value model.
[0075] S4, Integrated Forecasting Model;
[0076] Simultaneously running both the conventional model and the extreme value model, combined with an extreme value flow threshold of 4050m. 3 / s, when the prediction result of the extreme value model is not determined to be extreme data, the conventional model is used; otherwise, the extreme value model is used.
[0077] S5. Model Accuracy Evaluation: The model is evaluated using test period data. The relevant accuracy indices KGE and NSE of this invention are calculated, and the simulated and measured flow process lines are compared to evaluate accuracy. Simulation results are as follows: Figure 3 As shown in Table 3, the comparison results with the conventional model are as follows:
[0078] Table 3 Comparison of model simulation results accuracy
[0079] Model KGE NSE This invention model 0.84 0.76 Conventional model 0.76 0.73
[0080] During the testing period, the KGE and NSE of the model of this invention were 0.84 and 0.76, respectively, both higher than those of conventional models, with the KGE exceeding that of conventional models by a significant margin. This simulation demonstrates high accuracy. According to the "Hydrological Information Forecasting Standard," an NSE greater than 0.75 indicates excellent performance, demonstrating the overall superior performance of the model of this invention.
[0081] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. A method for constructing a runoff forecasting model that considers extreme runoff, characterized in that, Includes the following steps: S1. Collect and preprocess historical hydrological and meteorological data and teleconnection climate factor data of the study area to obtain multiple forecast factors and form a forecast factor set; S2. Determine the contribution of each forecast factor and initially screen out the N forecast factors with the highest contribution; sort these N forecast factors from high to low contribution to obtain the key forecast factor set. S3. Input the first N, the first N-1, ..., the first 1 forecast factors in the set of key forecast factors into the preliminary forecast model for training and testing, thereby obtaining N forecast schemes. Each forecast scheme includes the input forecast factors and the corresponding trained preliminary forecast model. For N forecast schemes, calculate their Nash efficiency coefficients during testing, and select the set of forecast schemes with the highest Nash efficiency coefficients as the conventional model; Meanwhile, for N forecast schemes, within the range exceeding the runoff extreme threshold, the forecast runoff and actual runoff output by the preliminary forecast model during the test are fitted respectively, and the set of forecast schemes with the best fit is selected as the extreme value model. S4. Couple the conventional model and the extreme value model to obtain a comprehensive runoff forecasting model.
2. The method for constructing a runoff forecasting model considering extreme runoff as described in claim 1, characterized in that, The contribution of each forecast factor is determined by a feature screening model. The feature screening model includes an XGBoost model and a SHAP model. The XGBoost model is used to predict runoff based on the forecast factors and is trained using the forecast factor set. The SHAP model is used to calculate the contribution of each forecast factor to the runoff prediction result based on the runoff prediction result output by the XGBoost model.
3. The method for constructing a runoff forecasting model considering extreme runoff as described in claim 1, characterized in that, The preliminary forecast model is constructed based on the BO-BiLSTM model.
4. The method for constructing a runoff forecasting model considering extreme runoff as described in claim 3, characterized in that, The optimization parameters in the BO-BiLSTM model include L2 regularization terms.
5. The method for constructing a runoff forecasting model considering extreme runoff as described in claim 1, characterized in that, The method for determining the extreme threshold of runoff is as follows: calculate the mean and standard deviation of historical runoff data, and based on 3 The principle is to take the sum of the mean and three times the standard deviation as the extreme threshold for runoff.
6. The method for constructing a runoff forecasting model considering extreme runoff as described in any one of claims 1-5, characterized in that, The historical hydrological and meteorological data include monthly precipitation, monthly evapotranspiration, and monthly runoff; the teleconnection climate factor data include atmospheric circulation factors, sea surface temperature index, and other related indices.
7. The method for constructing a runoff forecasting model considering extreme runoff as described in claim 6, characterized in that, The preprocessing includes: obtaining initial forecast factors based on historical hydrological and meteorological data and teleconnection climate factor data; deleting and imputing forecast factors based on time-related missing data; and then performing time-lag processing on the forecast factors to form a forecast factor set.
8. A system for constructing a runoff forecasting model that considers extreme runoff, characterized in that, Includes a processor for executing the runoff forecasting model construction method considering extreme runoff as described in any one of claims 1-7.
9. A runoff forecasting method, characterized in that, The process includes the following steps: real-time acquisition of various forecast factors in the study area; construction of a runoff forecast model using the runoff forecast model construction method considering extreme runoff as described in any one of claims 1-7; inputting the corresponding forecast factors into the conventional model and the extreme value model respectively to obtain conventional runoff data and extreme runoff data.