Runoff process correction method coupling machine learning and physical process model

Abstract

The invention discloses a runoff process correction method coupling machine learning and a physical process model, and belongs to the technical field of hydrological forecasting, and the method comprises the following steps: collecting historical hydrometeorological data of a research area, examining the reliability, consistency and representativeness of the data, and carrying out the normalization processing to obtain a normalized sample; constructing a coupling model; and on the basis of the trained coupling model, watershed runoff forecasting is carried out by adopting hydro-meteorological data. According to the method, the physical process model and the machine learning model are coupled, the runoff production stage and the confluence stage in the runoff forming process are corrected through the machine learning model, the runoff simulation precision is improved, and the method has great significance in improving the drainage basin runoff simulation precision and achieving efficient management of water resources; and the runoff forecasting precision of the drainage basin and the physical consistency of the correction process are improved.

Description

Technical Field

[0001] The invention belongs to the technical field of hydrological forecasting, and in particular relates to a runoff process correction method coupling machine learning and a physical process model. Background Art

[0002] With the rapid development of society, the problems caused by floods and water shortages are becoming increasingly prominent. In order to reduce the uncertainty brought by floods to property safety, my country's current flood control and disaster reduction measures can be mainly divided into two categories: flood control engineering measures and non-engineering flood control measures. As an important part of non-engineering flood control measures, hydrological forecasting plays an increasingly important role in social and economic construction and development, flood control and waterlogging prevention. Hydrological forecasting is a work with strong theoretical requirements and high application requirements. Runoff prediction is of great significance in basin water resources planning and design and flood control and waterlogging prevention of water conservancy projects. However, due to the influence of climate change and human activities, the rainfall runoff process shows a high degree of nonlinearity and instability, and accurate runoff prediction is difficult.

[0003] At present, runoff prediction models are mainly divided into two categories: physical process models and data-driven models. The hydrological model of the physical process has certain physical meanings, but due to the lack of understanding of the mechanism of the hydrological process, the high complexity of the model and the large number of parameters, the model faces challenges in practical application. In contrast, the data-driven hydrological model only predicts by mining the correlation between the input and output data, and does not need to consider the actual physical meaning of the runoff formation process. Among them, the machine learning model has certain advantages in the analysis and prediction of runoff sequences. However, due to the black box properties of the machine learning model, it is difficult to explain from the perspective of physical principles, so when using this method for prediction, there may be credibility issues.

[0004] The present invention couples the physical process model and the machine learning model to solve the limitations of the two types of models. At present, the research on using machine learning models to correct runoff in physical process models is mainly based on the terminal correction method, that is, only the final output of the physical process model is corrected, without considering the model's own description of the runoff generation and runoff process, failing to correct the runoff generation and runoff process, and not considering the physical consistency of the correction process. Summary of the invention

[0005] The object of the present invention is to provide a runoff process correction method that couples machine learning and physical process models to solve the problems raised in the above background technology.

[0006] To achieve the above object, the present invention provides the following technical solution: a runoff process correction method coupling machine learning and physical process model, comprising the following steps:

[0007] S1. Collect historical hydrological and meteorological data in the study area, review the reliability, consistency and representativeness of the data, and perform normalization to obtain normalized samples;

[0008] S2, construction of coupling model;

[0009] S3. Based on the trained coupling model, hydrological and meteorological data are used to carry out basin runoff forecast.

[0010] The above schemes are adopted. Since the physical process model simplifies the hydrological process to varying degrees, and the model is highly complex, has many parameters, and has high requirements for data quality, there are great challenges in the actual application process. When the machine learning model is used for runoff prediction, although it has high accuracy in use, it still has process unexplainability and does not consider the consistency of the actual physical process. Coupling the two types of models can improve the accuracy of basin runoff simulation on the basis of considering the physical meaning of the actual runoff generation and confluence of the basin, which is of great significance to improving the accuracy of basin runoff simulation and realizing efficient management of water resources. The two types of models are coupled in a way that: the method of using machine learning models to correct the runoff of the physical process model is mainly based on the terminal correction method, that is, only the final output result of the physical process model is corrected, without considering the model's own description of the runoff generation and confluence process, without correcting the runoff generation and confluence process, and without considering the physical consistency of the correction process. A runoff process correction method that couples machine learning and physical process models proposed in the present invention takes into account the actual physical significance of runoff formation, corrects the runoff from two aspects, and improves the runoff forecast accuracy of the basin and the physical consistency of the correction process.

[0011] As a preferred implementation, step S1 specifically includes the following steps:

[0012] S101. Division of training period and validation period: Taking years as units, divide the data of the research area into training period and validation period according to a certain ratio;

[0013] S102, normalize the data of the training period and the validation period respectively:

[0014]

[0015] Where: x ′ represents the normalized data; x represents the observed data; x min and x max Represent the minimum and maximum values ​​in the observed data respectively.

[0016] As a preferred implementation, step S2 specifically includes the following steps:

[0017] S201, calibration of physical process model and division of runoff generation and confluence modules;

[0018] S202, construction of the runoff correction framework (PEC-1);

[0019] S203, increasing the amount of training data;

[0020] S204, calculation of correction factor K;

[0021] S205. Construction of the confluence correction framework (PEC-2).

[0022] As a preferred implementation, in step S201, a genetic algorithm is used to calibrate the physical process model, and the objective function of parameter calibration is the Nash coefficient (NSE); according to the actual physical meaning of runoff formation, the model is divided into a flow generation module and a flow confluence module.

[0023] As a preferred implementation, in step S202, according to the divided runoff modules, the machine learning model is used as the correction model, and the prediction factors and prediction targets of the runoff correction framework are constructed according to the following scheme. According to different prediction targets, the runoff correction schemes can be roughly divided into two categories: PECG-D and PECG-R; Scheme PECG-D is based on the observed annual runoff. As the prediction target, the PECG-R scheme uses the annual flow residual of the physical process model as a prediction target.

[0024] Solution PECG-D-1: As a predictor, As output, As a prediction target;

[0025] Solution PECG-D-2: As a predictor, As output, As a prediction target;

[0026] Solution PECG-D-3: As a predictor, As output, As a prediction target;

[0027] Solution PECG-R-1: As a predictor, As output, As a prediction target;

[0028] Solution PECG-R-2: As a predictor, As output, As a prediction target;

[0029] Solution PECG-R-3: As a predictor, As output, as a prediction target.

[0030] in: They represent the annual precipitation and annual evaporation at time T respectively;

[0031] and They represent the simulated annual flow of the physical process model at time T, the annual flow after correction, and the observed annual flow; Represents the residual value of the annual flow rate simulated by the physical process model at time T, It represents the residual value of annual flow after correction at time T.

[0032] As a preferred implementation, in step S203, in order to reduce the impact of too little data on the model training effect, the training data is magnified in time sequence of the original data to increase the amount of training data; the amount of training data of the best solution obtained in step S202 is amplified, and the amount of data amplified by different multiples is used as input to explore the optimal amount of training data.

[0033] As a preferred implementation, in step S204, the best annual corrected flow rate Wsim is obtained based on the best training scheme obtained in step S202 and the best training data volume obtained in step S203; at the same time, the annual correction factor is calculated based on this value and the simulated flow rate WPBM before correction. The correction factor is used to correct the flow generation process of the flow generation module, namely:

[0034] As a preferred implementation, in step S205, the As the input of the physical process model confluence module, it is used for confluence calculation. After the simulation of the confluence module, the output result is the simulated runoff process. The runoff process was corrected to construct the PEC-2 framework, and the correction results of the entire model were obtained; the Bayesian optimization algorithm was used to optimize the hyperparameters of the machine learning model with the objective function as NSE; similar to the scheme setting of the PEC-1 framework, the confluence results were corrected according to two types of correction schemes (PECC-D and PECC-R), as follows:

[0035] Scheme PECC-D-1: As a predictor, As output, As a training goal;

[0036] Scheme PECC-D-2: As a predictor, As output, As a training goal;

[0037] Scheme PECC-R-1: As a predictor, As output, As a training goal;

[0038] Scheme PECC-R-2: As a predictor, As output, as a training goal.

[0039] in, as well as They represent the simulated runoff value after only runoff correction, the simulated runoff value after runoff correction, and the observed runoff value at time t respectively; represents the residual value of runoff after only runoff correction at time t, It represents the simulated residual value of runoff after runoff correction at time t.

[0040] Compared with the prior art, the present invention has the following beneficial effects:

[0041] This runoff process correction method that couples machine learning and physical process models faces great challenges in practical application because the physical process model simplifies the hydrological process to varying degrees, and the model is highly complex, has many parameters, and has high requirements for data quality. Although the machine learning model has high accuracy in runoff prediction, it still has process unexplainability and does not consider the consistency of the actual physical process. Coupling the two types of models can improve the accuracy of basin runoff simulation on the basis of considering the physical meaning of the actual runoff generation and confluence of the basin, which is of great significance to improving the accuracy of basin runoff simulation and achieving efficient management of water resources;

[0042] The runoff process correction method coupled with machine learning and physical process models utilizes two types of model coupling methods: the method of using machine learning models to correct runoff for physical process models is mainly based on the terminal correction method, that is, only the final output result of the physical process model is corrected, without considering the model's own description of the runoff generation and confluence process, without correcting the runoff generation and confluence process, and without considering the physical consistency of the correction process. The runoff process correction method coupled with machine learning and physical process models proposed in the present invention takes into account the actual physical significance of runoff formation, corrects the runoff from two aspects, and improves the runoff forecast accuracy of the basin and the physical consistency of the correction process. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a schematic diagram of the process of the present invention;

[0044] Figure 2 (a) is a flow correction result diagram of the present invention without enlarging the amount of training data;

[0045] Figure 2 (b) is a flow correction result diagram after the training data volume is enlarged according to the present invention;

[0046] Figure 3 This is a graph showing the result of the current confluence correction of the present invention;

[0047] Figure 4 It is a comparison diagram of the monthly runoff process lines before and after the correction of the present invention. DETAILED DESCRIPTION

[0048] See also Figure 1-4 The present invention provides a runoff process correction method coupling machine learning and physical process model, comprising the following steps:

[0049] S1: Collect historical hydrological and meteorological data of the study area, conduct data reliability, consistency and representativeness review and normalization processing, and obtain normalized samples. In the embodiment of the present invention, step S1 specifically includes the following steps:

[0050] S101: Division of training period and validation period: Divide the data into years, with the first 60% as the training period and the last 40% as the validation period.

[0051] S102: Normalizing the data of the training period and the validation period respectively to reduce the influence of the difference between the data on the weight of the runoff input variable.

[0052]

[0053] Where: x ′ represents the normalized data; x represents the observed data; x min and xmax Represent the minimum and maximum values ​​in the observed data respectively.

[0054] Step S2: Construction of the coupling model. It mainly includes five steps: calibration of the physical process model and division of the runoff generation and confluence modules, construction of the runoff generation correction framework (PEC-1), amplification of the training data volume, calculation of the correction factor K, and construction of the runoff correction framework (PEC-2).

[0055] S201: Calibration of the physical process model and division of the runoff generation and confluence modules. The genetic algorithm is used as the optimization algorithm for the physical process model parameters, and the maximum Nash coefficient (NSE) is used as the objective function to calibrate the model. The calibrated physical process model is divided into a runoff generation module and a confluence module according to the actual physical meaning of runoff formation.

[0056] S202: Construction of the runoff correction framework (PEC-1). The runoff result of the physical process model, the runoff, is calculated in years, and the prediction factors and prediction targets of the machine learning model are constructed according to Table 1. The schemes can be roughly divided into two categories: PECG-D and PECG-R. The former is based on the observed annual runoff. As the training target, the latter uses the annual flow residual of the physical process model as a training goal.

[0057] Table 1 PEC-1 framework scheme settings

[0058]

[0059]

[0060] Note: They represent the annual precipitation and annual evaporation at time T respectively; and They represent the simulated annual flow of the physical process model at time T, the annual flow after correction, and the observed annual flow; Represents the residual value of the annual flow rate simulated by the physical process model at time T, It represents the residual value of annual flow after correction at time T.

[0061] When different schemes are used to construct the PEC-1 framework, the adaptive moment estimation algorithm is used to train and optimize the machine learning model, and the mean square error is used as the objective function of the optimization process. For scheme PECG-D, the output of this type of scheme is the simulated value of the annual flow; for scheme PECG-R, the output of this type of scheme is added to the output of the physical process model in the flow generation module to obtain the simulated value of the annual flow.

[0062] S203: Amplify the amount of training data. In order to avoid the impact of too small data on the simulation effect, the training data is amplified in the order of the original data time to increase the amount of training data. The training length of the best solution obtained in the previous step is amplified by 10, 20, 30, 40, 50, 60, 70, 80, 90, 100 times respectively, and the data amplified by different times is used as input to build machine learning models with different training lengths.

[0063] S204: Calculation of correction factor K. Based on the best training scheme obtained in S202 and the best training data obtained in S203, the best annual corrected flow Wsim is obtained. At the same time, the annual correction factor is calculated based on this value and the simulated flow WPBM before correction. The correction factor is used to correct the flow generation process of the flow generation module, that is,

[0064] S205: Construction of the confluence correction framework (PEC-2). As the input of the physical process model confluence module, the confluence calculation is performed on the flow produced after flow correction. After simulation by the confluence module, the output result is Similar to the runoff correction, the runoff results were corrected according to two correction schemes (PECC-D and PECC-R), as shown in Table 2.

[0065] Table 2PEC-2 framework scheme settings

[0066]

[0067] Note: as well as They represent the simulated runoff value after only runoff correction, the simulated runoff value after runoff correction, and the observed runoff value at time t respectively; represents the residual value of runoff after only runoff correction at time t, It represents the simulated residual value of runoff after runoff correction at time t.

[0068] Since the outputs of the two schemes are different, the processing methods for the output results are different. For the PECC-D scheme, the output result of the model is the correction result; for the PECC-R scheme, the output result of the model is added to the output result of the physical process model to obtain the correction result.

[0069] The model runoff forecast results were evaluated using NSE, root mean square error (RMSE), mean absolute error (MAE), flood error (REV) and modified Kling-Gupta coefficient (KGEmod).

[0070]

[0071] In the formula, x t , and They represent the measured value, simulated value and mean of the measured value of the flow rate respectively; N represents the length of the data; R and Represent the total runoff of measured runoff and simulated runoff, respectively. For KGEmod, r represents the correlation coefficient between simulated runoff and observed runoff; α represents the ratio of the mean of simulated runoff to measured runoff; β represents the ratio of the coefficient of variation of simulated runoff to measured runoff. The closer the values ​​of NSE and KGEmod are to 1, the smaller the values ​​of RMSE and MAE are, indicating that the simulated values ​​of the model are closer to the measured values, and the better the simulation effect of the model; the closer the value of REV is to 0, the smaller the error between the simulated runoff and the total amount of measured runoff.

[0072] In order to effectively verify the effectiveness of the process correction method proposed in the present invention, the following examples are needed for demonstration. Example 1: The runoff process of Waizhou Station, Xiajiang Station and Dongbei Station in the Ganjiang River Basin is taken as the research object.

[0073] The daily runoff data observed at the Waizhou, Xiajiang, and Dongbei hydrological stations and the daily precipitation and evaporation data of the control section are used as basic data. For the Waizhou and Xiajiang stations, the data length is 1961–2013, of which 1961–1992 is the training period and 1993–2013 is the validation period; for the Dongbei station, the data length is 1961–2009, of which 1961–1990 is the training period and 1991–2009 is the validation period. The daily precipitation, evaporation, and runoff data of the three stations are normalized according to formula (1).

[0074] This example selects the Xin'anjiang (XAJ) model, which is widely used in my country's humid areas, as the physical process model, and selects the long short-term memory neural network (LSTM) model, which has a good effect on time series prediction problems, as the machine learning model. Therefore, this example couples the XAJ model with the LSTM model. The genetic algorithm is used to calibrate the parameters of the XAJ model, and according to the physical meaning of the XAJ model, the evapotranspiration, runoff generation, and water source calculation parts of the model are divided into runoff generation modules, and the confluence part of the model is divided into confluence modules, and the PEC-1 framework is constructed according to Table 1. Figure 2 (a) shows the runoff correction results obtained by training the LSTM model according to the six schemes. It can be found that the best scheme for the three stations is PECG-D-2, that is, annual precipitation, annual evaporation and XAJ simulated annual runoff are used as prediction factors, and the measured annual runoff is used as the prediction target.

[0075] In order to avoid the impact of small amount of training data on simulation effect, according to the best runoff correction scheme PECG-D-2, the data volume of different stations is enlarged by 10 times, 20 times, 30 times, 40 times, 50 times, 60 times, 70 times, 80 times, 90 times and 100 times respectively. The correction effect after enlargement is shown in Figure 2. Figure 2 (b) According to Figure 2 (b) It can be found that the three stations have good correction effects when the magnification is 80 times, so 80 times is taken as the optimal magnification of the training data volume of the three stations. According to the results simulated by the above optimal runoff correction scheme and the optimal training data volume, the ratio of the annual optimal runoff Wsim obtained by the XAJ model to the annual runoff WXAJ simulated by the XAJ model is calculated. It is used to correct the flow generation process. And the corrected flow generation process results are input into the XAJ model confluence module for confluence calculation.

[0076] Similarly, the confluence correction process was set up according to the PEC-2 framework scheme (Table 2), in which the hyperparameter optimization of the LSTM model used the Bayesian optimization algorithm and the K-fold cross-validation method, and the objective function for optimizing the hyperparameters was NSE. The correction results are shown in Figure 3 . It can be found that for the three stations, the four schemes all have certain correction effects, among which PECR-1 has the best correction effect (NSE and KGEmod values ​​are the largest, and RMSE, MAE and REV values ​​are closest to 0). Compared with the XAJ model, NSE and KGEmod increased by 17.721% to 21.376% and 16.114% to 16.824%, respectively, and RMSE, MAE and REV values ​​decreased by 33.295% to 52.511%, 37.661% to 59.479% and 73.509% to 90.144%, respectively, indicating that the proposed process correction method has a good correction effect. In order to fully compare the simulation effects of the XAJ model and the PEC framework on runoff, Figure 4 Monthly runoff hydrographs before and after correction for the three stations are shown.

Claims

1. A runoff process correction method that couples machine learning and physical process models, characterized by: The following steps are involved: S1. Collect historical hydrological and meteorological data in the study area, review the reliability, consistency and representativeness of the data, and perform normalization to obtain normalized samples; S2, construction of coupling model; S3. Based on the trained coupling model, hydrological and meteorological data are used to carry out basin runoff forecast.

2. The runoff process correction method of coupling machine learning and physical process model according to claim 1 is characterized in that: Step S1 specifically includes the following steps: S101. Division of training period and validation period: Taking years as units, divide the data of the research area into training period and validation period according to a certain ratio; S102, normalize the data of the training period and the validation period respectively: In the formula: x′ represents the normalized data; x represents the observed data; x min and x max Represent the minimum and maximum values ​​in the observed data respectively.

3. The runoff process correction method of coupling machine learning and physical process model according to claim 1 is characterized in that: Step S2 specifically includes the following steps: S201, calibration of physical process model and division of runoff generation and confluence modules; S202, construction of the runoff correction framework (PEC-1); S203, increasing the amount of training data; S204, calculation of correction factor K; S205. Construction of the confluence correction framework (PEC-2).

4. The runoff process correction method of coupling machine learning and physical process model according to claim 3 is characterized by: In step S201, a genetic algorithm is used to calibrate the physical process model, and the objective function of parameter calibration is the Nash coefficient (NSE); according to the actual physical meaning of runoff formation, the model is divided into a runoff generation module and a runoff confluence module.

5. The runoff process correction method of coupling machine learning and physical process model according to claim 3 is characterized by: In step S202, according to the divided runoff modules, the machine learning model is used as the correction model, and the prediction factors and prediction targets of the runoff correction framework are constructed according to the following scheme; according to different prediction targets, the runoff correction schemes can be roughly divided into two categories: PECG-D and PECG-R; Scheme PECG-D is based on the observed annual runoff As the prediction target, the PECG-R scheme uses the annual flow residual of the physical process model As a prediction target; Solution PECG-D-1: As a predictor, As output, As a prediction target; Solution PECG-D-2: As a predictor, As output, As a prediction target; Solution PECG-D-3: As a predictor, As output, As a prediction target; Solution PECG-R-1: As a predictor, As output, As a prediction target; Solution PECG-R-2: As a predictor, As output, As a prediction target; Solution PECG-R-3: As a predictor, As output, As a prediction target; in: They represent the annual precipitation and annual evaporation at time T respectively; and They represent the simulated annual flow of the physical process model at time T, the annual flow after correction, and the observed annual flow; Represents the residual value of the annual flow rate simulated by the physical process model at time T, It represents the residual value of annual flow after correction at time T.

6. The runoff process correction method of coupling machine learning and physical process model according to claim 3 is characterized by: In step S203, in order to reduce the impact of too little data on the model training effect, the training data is magnified in time sequence of the original data to increase the amount of training data; the amount of training data of the best solution obtained in step S202 is amplified, and the amount of data amplified by different multiples is used as input to explore the optimal amount of training data.

7. The runoff process correction method of coupling machine learning and physical process modeling according to claim 3 is characterized by: In step S204, the optimal annual corrected flow rate Wsim is obtained based on the optimal training scheme obtained in step S202 and the optimal training data volume obtained in step S203; at the same time, the annual correction factor is calculated based on this value and the simulated flow rate WPBM before correction. The correction factor is used to correct the flow generation process of the flow generation module, namely:

8. The runoff process correction method of coupling machine learning and physical process model according to claim 3 is characterized by: In step S205, the As the input of the physical process model confluence module, it is used for confluence calculation; after the simulation of the confluence module, the output result is the simulated runoff process The runoff process was corrected to construct the PEC-2 framework, and the correction results of the entire model were obtained; the Bayesian optimization algorithm was used to optimize the hyperparameters of the machine learning model with the objective function as NSE; similar to the scheme setting of the PEC-1 framework, the confluence results were corrected according to two types of correction schemes (PECC-D and PECC-R), as follows: Scheme PECC-D-1: As a predictor, As output, As a training goal; Scheme PECC-D-2: As a predictor, As output, As a training goal; Scheme PECC-R-1: As a predictor, As output, As a training goal; Scheme PECC-R-2: As a predictor, As output, As a training goal; in, as well as They represent the simulated runoff value after only runoff correction, the simulated runoff value after runoff correction, and the observed runoff value at time t respectively; represents the residual value of runoff after only runoff correction at time t, It represents the simulated residual value of runoff after runoff correction at time t.

Images