Hydrological model real-time correction method based on reinforcement learning A2C algorithm

Abstract

The application discloses a hydrological model real-time correction method based on a reinforcement learning A2C algorithm, and the method comprises the following steps: step 1, obtaining and preprocessing basic data of a research area; step 2, constructing a real-time correction model based on the reinforcement learning A2C algorithm; step 3, training and verifying the real-time correction model; and step 4, correcting rainfall in real time by using the real-time correction model. By coupling the reinforcement learning A2C algorithm and the hydrological model, the application solves the problems of slow convergence and easy falling into local optimization of a traditional method, and realizes high-precision and self-adaptive real-time correction of flood forecasting. The method disclosed by the application significantly improves the accuracy and dynamic response capability of flood forecasting, and provides reliable technical support for flood control and disaster reduction and water resource management.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of hydrology and water resources and artificial intelligence, and particularly relates to a hydrological model real-time correction method based on a reinforcement learning A2C algorithm. BACKGROUND

[0002] With the influence of global climate change and human activities, the frequency and intensity of extreme hydrological events such as floods are increasing, and accurate flood forecasting is crucial for flood control and disaster reduction, rational allocation of water resources, and protection of people's lives and property. As the core tool for flood forecasting, hydrological models play an important role in flood simulation and prediction. However, traditional hydrological models face many challenges in practical application.

[0003] On the one hand, hydrological processes are highly complex and uncertain, influenced by meteorological conditions, underlying surface conditions, human activities and other factors, making it difficult to accurately determine model parameters, resulting in deviations between simulation results and actual observation data. On the other hand, existing real-time correction methods for hydrological models are mostly based on traditional optimization algorithms such as least squares, Kalman filtering, etc. These methods often have slow convergence speed and are prone to local optimization when dealing with nonlinear and dynamic hydrological systems, making it difficult to adapt to complex and variable flood processes and achieving high-precision real-time flood forecasting.

[0004] In recent years, the application of artificial intelligence technology in the field of hydrology has gradually emerged, providing new ideas and methods for real-time correction of hydrological models. Reinforcement learning, as an important branch of artificial intelligence, can learn through the interaction between agents and the environment, constantly optimizing decision strategies to maximize long-term rewards. However, the current application of reinforcement learning to real-time correction of hydrological models is still in the exploratory stage, and how to effectively combine reinforcement learning algorithms with hydrological models to improve the real-time and accuracy of flood forecasting remains a problem to be solved. Therefore, a new real-time correction method for hydrological models based on reinforcement learning is needed to improve the accuracy and reliability of flood forecasting. SUMMARY

[0005] The purpose of the present application is to provide a hydrological model real-time correction method based on reinforcement learning A2C algorithm to solve the above technical problems.

[0006] To achieve the above purpose, the present application provides the following technical solutions:

[0007] The present application discloses a hydrological model real-time correction method based on reinforcement learning A2C algorithm, which comprises the following steps:

[0008] Step 1, obtaining basic data of the study area and preprocessing: selecting a small and medium-sized watershed with more than ten years of continuous hydro-meteorological monitoring data as the study area, obtaining the historical multi-year basic data of the study area, including daily-scale area-averaged precipitation data, area-averaged forecast precipitation data, and area-averaged observed flow data; then, the obtained basic data is preprocessed, specifically, the missing values of the area-averaged precipitation data, the area-averaged forecast precipitation data, and the area-averaged observed flow data are filled by using linear interpolation method to ensure the continuity of the time series;

[0009] Step 2, constructing a real-time correction model based on the reinforcement learning A2C algorithm: constructing a real-time correction model based on the reinforcement learning A2C algorithm, the real-time correction model is composed of a conceptual hydrological model M and a policy network π θ , the conceptual hydrological model M is used to receive the area-averaged precipitation data and output the simulated flow for constructing the state vector h; the policy network π θ is composed of an observation decision network π o and a forecast decision network π f , which respectively receive the observation state h o and the forecast state h f , generate the area-averaged precipitation correction amount δp o and the area-averaged forecast precipitation correction amount δp f for correcting the rainfall data, and the parameters θ thereof are updated by the A2C algorithm;

[0010] The construction process of the real-time correction model specifically includes the following steps:

[0011] Step 2.1, parameter and data initialization: initializing the parameters and data, the parameters include: prediction step τ, τ ∈ N + ; time window length T, T ∈ N + ; current time t, t ∈ N + , t ≤ t max , t max is set according to the time length of the data obtained in step 1; past time i, i ∈ N + ; discount factor γ, 0 < γ < 1, γ ∈ R; experience pool D, experience pool capacity d, d ∈ N + ; batch size B, B ∈ N + and B ≤ d; the data includes area-averaged precipitation: area-averaged forecast precipitation: area-averaged observed flow:

[0012] Step 2.2, calculating the simulated flow: using the conceptual hydrological model M to operate p o to obtain the simulated flow

[0013] q s =M(p o )

[0014] Step 2.3, Construct the current state and calculate the correction action: Construct the current state h = (h o ,h f ), calculate the correction action a = (δp) o ,δp f The value of π and its probability distribution θ (a∣h); where δp o δp is the correction for the average groundfall rainfall from time i to time t. f h is the area average forecast rainfall correction from time t+1 to time t+τ. o It is the observation state, h f Forecast status, π θ For policy networks;

[0015] Step 2.4: Correct the average rainfall received and the average forecast rainfall data: Correct the average rainfall from time i to time t. The predicted rainfall p from time t+1 to time t+τ f :

[0016]

[0017] p f ←p f +δp f

[0018] Step 2.5: Update the policy network π θ The parameter θ: The policy network π based on the A2C algorithm is updated. θ The parameter θ;

[0019] Then repeat steps 2.2 to 2.5 until t reaches t_t. max ;

[0020] Step 3: Train and validate the real-time correction model: First, divide the preprocessed data from Step 1 into a training set and a validation set according to the proportions; then train the real-time correction model using the training set, and finally validate the real-time correction model using the validation set; until the model passes validation, it indicates that the real-time correction model training is complete.

[0021] Step 4: Correct rainfall in real time using a real-time correction model: Acquire real-time data and preprocess it, including real-time average rainfall per unit area, average forecast rainfall per unit area, and average observed flow rate per unit area; initialize parameters and set t. max =+∞, initialize the average groundfall rainfall p using real-time data. o Average rainfall forecast p f Observed flow rate q o; strategy network π of loading real-time correction model of training completion θ , then according to steps 2.2 to 2.5, the real-time corrected areal rainfall can be obtained

[0022] Further, the areal rainfall data in step 1 is derived from the measured values of the ground rainfall stations in the study area, the areal forecast rainfall data is derived from the output results of the CMA-MESO mesoscale model of the meteorological bureau, and the areal observed flow data is derived from the observed values of the flow observation stations in the study area.

[0023] Further, the conceptual hydrological model M in step 2.2 is the Xin'anjiang model.

[0024] Further, the current state h = (h o , h f ) in step 2.3 is constructed, the value of the correction action a = (δp o , δp f ) and its probability distribution π θ (a | h) are calculated, which specifically includes the following steps:

[0025] Step 2.3.1, constructing the observation state h o for calculating the areal rainfall correction amount δp o from i to t:

[0026] The areal rainfall from i to t , the simulated flow and the areal observed flow are spliced to construct the observation state h o :

[0027]

[0028] h o is sent to the observation decision network π o , and the probability distribution of δp o is obtained:

[0029] π o (δp o | h o )

[0030] The specific value of δp o is obtained by sampling π o (δp o | h o );

[0031] Wherein, the observation decision network π o is a multi-layer perception (MLP), and δp oThe specific value is obtained through reparameterization, specifically: δp o =μ+σ⊙z, where μ and σ are π o The output Gaussian distribution parameters, z is the generated random number sequence that follows a standard normal distribution;

[0032] Step 2.3.2: Construct the predicted state h f The area average forecast rainfall correction δp is used to calculate the rainfall correction from time t+1 to time t+τ. f :

[0033] By splicing the average groundfall rainfall from time i to t Average forecast rainfall p f Correction for average rainfall on the ground δp o Construct the forecast state h f :

[0034]

[0035] h f Input into the forecast decision network π f Average rainfall forecast correction δp f probability distribution:

[0036] π f (δp f |h f )

[0037] By analyzing π f (δp f |h f )Sampling to obtain δp f The specific value;

[0038] Among them, the forecast decision network π f For Long Short-Term Memory (LSTM) networks, δp f The specific value is obtained through reparameterization, the principle of which is the same as δp. o The specific sampling process;

[0039] Step 2.3.3: Calculate the correction action a = (δp) o ,δp f In the current state h = (h o ,h f The probability distribution under () is:

[0040] π θ (a|h)=π o (δp o |h o )·π f (δp f |h f (δp o).

[0041] Further, the step 2.5 updates the parameters θ of the policy network π θ based on the A2C algorithm, which includes the following steps:

[0042] Step 2.5.1, simulate the flow with the corrected rainfall data: use the conceptual hydrological model M to simulate the flow o with the corrected areal rainfall p f and the areal forecast rainfall p

[0043]

[0044] Step 2.5.2, delayed reward calculation: at time t, temporarily store the current simulated flow in buffer area A, and mark its time range as [t+1, t+τ]; when the system actually arrives at time t+τ, obtain the true flow from the observation data, and calculate the delayed reward r:

[0045]

[0046] Step 2.5.3, update the experience pool: store the experience (h, a, None, h') at time t in temporary buffer B, and mark it as a reward to be updated; when time t+τ arrives, take out the experience at time t from B, fill in the reward r; check the capacity of the experience pool D, if |D| = d, remove the oldest experience; store the complete experience (h, a, r, h') in the experience pool D; where h' is the state vector at the next time;

[0047] Step 2.5.4, update the parameters θ: when there are at least B complete experiences in the experience pool D, randomly select B samples from the experience pool D Then update the parameters ω of the value network V ω : through the gradient of the loss function and the update rate α ω , update ω to: Then update the parameters θ of the policy network π θ : through the policy gradient and the update rate α θ , update θ to:

[0048] Step 2.5.5, update the next frame data: time update: i ← i + 1, t ← t + 1; areal rainfall p o and areal observed flow q o are updated to the current time t, and areal forecast rainfall p f is updated to t+τ.

[0049] Furthermore, the specific process of training the model using the training set in step 3 is as follows: initialize parameters, set t max Initialize the average groundfall rainfall p using data from the training set. o Average rainfall forecast p f Observed flow rate q o Then repeat steps 2.2 to 2.5 until t reaches t. max Set the value;

[0050] The specific process of validating the model using a validation set is as follows: initialize parameters, set t max Initialize the average groundfall rainfall p using data from the validation set. o Average rainfall forecast p f Observed flow rate q o Then, repeat steps 2.2 to 2.5 repeatedly, recording the simulated flow rate during each iteration. Observed flow rate q f , respectively denoted as and Used to calculate the Nash efficiency coefficient (NSE) for each cycle. t Root Mean Square Error (RMSE) t :

[0051]

[0052]

[0053] Then calculate the average Nash efficiency coefficient. and root mean square error As the final evaluation indicator:

[0054]

[0055]

[0056] If the final evaluation index meets the following conditions: Average Nash efficiency coefficient And the root mean square error If the value is less than 10% of the standard deviation of historical flow observations, it indicates that the model has passed validation; otherwise, the parameters should be reset, the training data optimized, and the training and validation iteratively repeated until the final evaluation index meets the requirements.

[0057] The beneficial effects of this invention are as follows: By coupling the reinforcement learning A2C algorithm with a hydrological model, this invention solves the problems of slow convergence and susceptibility to local optima in traditional methods, achieving high-precision, adaptive, and real-time correction of flood forecasts. The method described in this invention significantly improves the accuracy and dynamic response capability of flood forecasts, providing reliable technical support for flood control, disaster reduction, and water resource management.

[0058] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments. Attached Figure Description

[0059] Figure 1 This is a flowchart of the method described in this invention;

[0060] Figure 2 This is a schematic diagram illustrating the construction process of the real-time correction model in Example 1;

[0061] Figure 3 The area average rainfall correction amount δp in Example 1 O And forecast rainfall correction δp f A schematic diagram of the calculation;

[0062] Figure 4 In Example 1, the average rainfall p o Observed flow rate q o and forecast rainfall p f The updated diagram is shown below. Detailed Implementation

[0063] Example 1

[0064] This embodiment discloses a real-time correction method for hydrological models based on the reinforcement learning A2C algorithm, such as... Figure 1 As shown, it includes the following steps:

[0065] Step 1: Obtain basic data for the study area and perform preprocessing: Select small and medium-sized watersheds (area of ​​100–10,000 km²) with more than ten years of continuous hydrological and meteorological monitoring data. 2 This example selects a tributary of the Yangtze River (approximately 5000 km²) as the study area. 2 As the study area, the basin includes 10 rain gauge stations and 3 flow observation stations. It has the characteristics of a humid and semi-humid climate, with concentrated rainfall during the flood season (June-September) and frequent flood disasters.

[0066] The basic data of the research area from 2015 to 2024 is obtained, including daily scale surface precipitation data, surface forecast precipitation data and surface observation flow data. Among them, the surface precipitation data is obtained from the measured value of the ground rainfall station in the research area, the surface forecast precipitation data is obtained from the output result of the mesoscale model CMA-MESO of the meteorological bureau, and the surface observation flow data is obtained from the observation value of the flow observation station in the research area. Then the obtained basic data is pretreated, specifically, the missing values of the surface precipitation data, the surface forecast precipitation data and the surface observation flow data are filled by using linear interpolation method to ensure the continuity of time series.

[0067] Step 2, constructing a real-time correction model based on reinforcement learning A2C algorithm: constructing a real-time correction model based on reinforcement learning A2C algorithm, the real-time correction model is composed of a conceptual hydrological model M and a policy network π θ , the conceptual hydrological model M is used to receive surface precipitation data and output simulated flow to construct state vector h, the conceptual hydrological model M of the embodiment adopts Xin'anjiang model; the policy network π θ is composed of an observation decision network π o and a forecast decision network π f , which respectively receive observation state h o and forecast state h f , generate surface precipitation (historical rainfall) correction amount δp o and surface forecast rainfall (future rainfall) correction amount δp f for correcting rainfall data, and the parameters θ thereof are updated by A2C algorithm.

[0068] As shown in Figure 2 , the construction process of the real-time correction model specifically includes the following steps:

[0069] Step 2.1, parameter and data initialization: specifically including:

[0070] Prediction step: τ=3, representing the length of the future prediction period, τ∈N + ;

[0071] Time window length: T=4, representing the range of historical data used for model calculation, T∈N + ;

[0072] Current time: t=4, t∈N + , t≤t max ; t max is set according to the time length of the data obtained in step 1;

[0073] Past time: i=1, i∈N + ;

[0074] Discount factor: γ = 0.99, used to balance the weights of future rewards, 0 < γ < 1, γ ∈ R;

[0075] Experience Pool: Experience pool capacity: d = 100, defining the maximum number of experiences stored in the experience pool, d ∈ N. + ;

[0076] Batch size: B = 32, the number of samples drawn from the experience pool each time during model training, B ∈ n + And B≤d;

[0077] Average rainfall per unit area:

[0078] Average rainfall forecast:

[0079] Average flow rate per unit area:

[0080] Step 2.2, Calculate the simulated flow: Use the conceptual hydrological model M (Xin'anjiang model) to calculate p o The simulated flow rate is obtained through calculation.

[0081] q s =M(p o )

[0082] Step 2.3, Construct the current state and calculate the correction action: Construct the current state h = (h o ,h f ), calculate the correction action a = (δp) o ,δp f The value of π and its probability distribution θ (a∣h). Where δp o δp is the correction for the average groundfall rainfall from time i to time t. f h is the area average forecast rainfall correction from time t+1 to time t+τ. o It is the observation state, h f Forecast status, π θ For policy networks, such as Figure 3 As shown, the specific steps include:

[0083] Step 2.3.1: Construct the observation state h o Used to calculate the correction amount δp for average groundfall rainfall from time i to t. o :

[0084] By splicing the average groundfall rainfall from time i to t Simulated traffic Observed flow rate of the surface Construct the observation state h o :

[0085]

[0086] h o Input into the observation-decision network π o δp o probability distribution:

[0087] π o (δp o |h o )

[0088] By analyzing π o (δp o |h o )Sampling to obtain δp o The specific value.

[0089] Among them, the observation decision network π o For a multilayer perceptron (MLP), δp o The specific value can be obtained through reparameterization, specifically: δp o =μ+σ⊙z, where μ and σ are π o The output Gaussian distribution parameters, z is the generated random number sequence that follows a standard normal distribution.

[0090] Step 2.3.2: Construct the predicted state h f The area average forecast rainfall correction δp is used to calculate the rainfall correction from time t+1 to time t+τ. f :

[0091] By splicing the average groundfall rainfall from time i to t Average forecast rainfall p f Correction for average rainfall on the ground δp o Construct the forecast state h f :

[0092]

[0093] h f Input into the forecast decision network π f Average rainfall forecast correction δp f probability distribution:

[0094]

[0095] By analyzing π f (δp f |h f )Sampling to obtain δp f The specific value.

[0096] Among them, the forecast decision network π fis a long short-term memory network (LSTM). The specific value of δp f can be obtained by reparameterization, which is the same principle as δp o .

[0097] Step 2.3.3, calculate the probability distribution of the modified action a = (δp o , δp f ) under the current state h = (h o , h f ):

[0098] π θ (a | h) = π o (δp o | h o ) · π f (δp f | h f (δp o ))

[0099] In the logic of real-time correction of the hydrological model, first, the correction amount δp o of the surface rainfall in the current period (i to t time) needs to be determined according to the existing observation data (including the surface rainfall p s , the simulated flow q o and the surface observed flow q o , which is the correction of historical data. Then, based on the correction result of the historical data, combined with the surface forecast rainfall p f , the correction amount δp f of the surface forecast rainfall in the future period (t+1 to t+τ time) is determined. Such sequence conforms to the logic process from known to unknown and from history to future, which can more reasonably correct the hydrological model in real time.

[0100] Step 2.4, correct the surface rainfall and the surface forecast rainfall data: correct the surface rainfall p in i to t time and the forecast rainfall p f in t+1 to t+τ time:

[0101]

[0102] p f ← p f + δp f

[0103] Step 2.5, update the parameters θ of the policy network π θ : update the parameters θ of the policy network π θ based on the A2C algorithm; specifically including the following steps:

[0104] Step 2.5.1: Simulate flow rate using corrected rainfall data: Use the conceptual hydrological model M to simulate the corrected average groundfall rainfall p. o Average rainfall forecast p f To simulate traffic

[0105]

[0106] Step 2.5.2, Delayed Reward Calculation: At time t, temporarily store the current simulated traffic. The data is transferred to buffer A, and its time range is marked as [t+1, t+τ]. When the system actually arrives at time t+τ, the actual traffic flow is obtained from the observed data. Calculate the delayed reward r:

[0107]

[0108] Step 2.5.3: Update the experience pool: Store the experience (h, a, None, h′) at time t into a temporary buffer B, marking it as a reward to be updated. When time t+τ arrives, retrieve the experience at time t from B and fill it with reward r (calculated in step 2.5.2). Check the capacity of the experience pool D; if |D|=d, remove the oldest experience. Store the complete experience (h, a, r, h′) into the experience pool D. Here, h′ is the state vector for the next time step.

[0109] Step 2.5.4: Update parameter θ: When there are at least B complete experiences in the experience pool D, randomly draw B samples from the experience pool D. Then update the value network V ω The parameter ω: through the loss function gradient and update rate α ω Update ω to: Then update the policy network π. θ The parameter θ: obtained through the policy gradient and update rate α θ Update θ to:

[0110] Step 2.5.5, Update the data in the next frame: Time-based update: i←i+1, t←t+1; Average groundfall rainfall p o Observed flow rate q o Updated to the current time t, average forecast rainfall p f Update to t+τ, such as Figure 4 As shown.

[0111] Then repeat steps 2.2 to 2.5 until t reaches t_t. max .

[0112] Step 3, training and verification of real-time correction model: first, the data preprocessed in step 1 is divided into training set and verification set according to a certain proportion. In this embodiment, the data is divided into training set and verification set according to 9:1, that is, the data from 2015 to 2023 is taken as the training set, and the data in 2024 is taken as the verification set, so as to ensure that the time range of the training set and the verification set does not overlap.

[0113] Then the model is trained using the training set, and the specific process is as follows: initialize the parameters, set t max = 9x12x30; initialize the surface mean rainfall p o , the surface mean predicted rainfall p f , and the surface mean observed flow q o in the training set. At this time, t=4 in step 2.1 corresponds to the actual time of January 4, 2015, and i=1 corresponds to the actual time of January 1, 2015. Then steps 2.2 to 2.5 are executed in a loop until t reaches t max .

[0114] Finally, the model is verified using the verification set, and the specific process is as follows: initialize the parameters, set t max = 1x12x30; initialize the surface mean rainfall p o , the surface mean predicted rainfall p f , and the surface mean observed flow q o in the verification set. At this time, t=4 in step 2.1 corresponds to the actual time of January 4, 2024, and i=1 corresponds to the actual time of January 1, 2024.

[0115] Then steps 2.2 to 2.5 are executed in a loop, and the simulated flow and the surface mean observed flow q f in each loop are recorded, respectively denoted as and , which are used to calculate the Nash efficiency coefficient NSE t and the root mean square error RMSE t in each loop:

[0116]

[0117]

[0118] Further, the average Nash efficiency coefficient and the average root mean square error are calculated as the final evaluation indicators:

[0119]

[0120]

[0121] If the final evaluation indicators meet the following two conditions at the same time, it indicates that the model verification is passed: 1) the average Nash efficiency coefficient Indicates that the model prediction results have high consistency with the measured data; 2) the average root mean square error Lower than 10% of the standard deviation of historical flow observations, indicating that the model prediction error is within a reasonable range.

[0122] If any indicator does not meet the requirements, the following improvement measures need to be taken: 1) re-set the model parameters (including discount factor, time window length, etc.); (2) optimize the training data, check the data quality and process outliers, and if necessary, introduce more historical data to enhance the generalization ability; (3) iterate training and verification until the final evaluation indicators meet the requirements.

[0123] Step 4, real-time correction of rainfall using real-time correction model: obtain real-time data and preprocess, including real-time area mean rainfall, area mean forecast rainfall and area mean observed flow. Initialize parameters and set t max = + ∞, initialize the area mean rainfall p o , the area mean forecast rainfall p f , and the area mean observed flow q o . At this time, t = 4 in step 2.1 corresponds to the actual time 2025-01-04, and i = 1 corresponds to the actual time 2025-01-01. Load the strategy network π θ of the real-time correction model trained, then execute steps 2.2 to 2.5, that is, the real-time corrected area mean rainfall p

[0124] Finally, it should be noted that the above description is only to illustrate the technical solutions of the present application and not to limit. Although the present application has been described in detail with reference to the preferred arrangement, those skilled in the art should understand that the technical solutions of the present application can be modified or replaced equivalently without departing from the spirit and scope of the technical solutions of the present application.

Claims

1. A hydrological model real-time correction method based on a reinforcement learning A2C algorithm, characterized in that, The method comprises the following steps: Step 1, obtaining research area basic data and preprocessing: selecting a small and medium-sized river basin with more than ten years of continuous hydro-meteorological monitoring data as the research area, obtaining historical multi-year basic data of the research area, including daily scale area mean precipitation data, area mean forecast precipitation data and area mean observed flow data; then, the obtained basic data is preprocessed, specifically, the missing values of the area mean precipitation data, the area mean forecast precipitation data and the area mean observed flow data are filled by using linear interpolation method to ensure the continuity of the time series; Step 2, Constructing the real-time correction model based on the reinforcement learning A2C algorithm: Constructing the real-time correction model based on the reinforcement learning A2C algorithm, the real-time correction model is composed of a conceptual hydrological model M and a policy network π θ , the conceptual hydrological model M is used to receive the areal rainfall data to output the simulated flow for constructing the state vector h; the policy network π θ is composed of an observation decision network π o and a prediction decision network π f , which respectively receive the observation state h o and the prediction state h f , and generate the areal rainfall correction amount δp o and the areal prediction rainfall correction amount δp f for correcting the rainfall data, the parameters θ of which are updated by the A2C algorithm; The construction process of the real-time correction model specifically comprises the following steps: Step 2.1, parameter and data initialization: initialize parameters and data, the parameters including: prediction step size τ, τ ∈ N + ; time window length T, T ∈ N + ; current time t, t ∈ N + , t ≤ t max , t max according to the time length of the data obtained in step 1; past time i, i ∈ N + ; discount factor γ, 0 < γ < 1, γ ∈ R; experience pool D, experience pool capacity d, d ∈ N + ; batch size B, B ∈ N + and B ≤ d; the data including: area-averaged rainfall on the ground: area-averaged predicted rainfall: area-averaged observed flow: Step 2.2, Calculate simulated flow: Using the conceptual hydrological model M on p o The simulated flow is calculated as q s = M(p o ) Step 2.3, build the current state calculation correction action: build the current state h = (h o ,h f ), calculate the value and its probability distribution π o (a∣h) of the correction action a = (δp f ,δp θ ); wherein δp o is the i to t time surface mean precipitation correction amount, δp f is the t+1 to t+τ time surface mean forecast rainfall correction amount, h o is the observation state, h f is the forecast state, and π θ is the policy network; Step 2.

4. Correction of the areal rainfall and the areal forecast rainfall data: Correction of the areal rainfall from i to t time and the forecast rainfall p from t+1 to t+τ time f : p f ←p f +δp f Step 2.5, updating the policy network p with parameters 0: updating the policy network p with parameters 0 based on the A2C algorithm θ Step 2.5, updating the policy network p with parameters 0: updating the policy network p with parameters 0 based on the A2C algorithm θ Step 2.5, updating the policy network p with Then steps 2.2 to 2.5 are repeated until t reaches t max ; Step 3, training and verifying the real-time correction model: first, the data preprocessed in step 1 is divided into a training set and a verification set in proportion; then, the training set is used to train the real-time correction model, and finally, the verification set is used to verify the real-time correction model; until the model is verified, indicating that the training of the real-time correction model is completed; Step 4, real-time correction of rainfall: obtain real-time data and pre-process, including real-time areal rainfall, areal forecast rainfall and areal observed flow; initialize parameters and set t max = +∞, initialize areal rainfall p o , areal forecast rainfall p f , and areal observed flow q o with real-time data; load the strategy network π θ of the real-time correction model trained, and then perform steps 2.2 to 2.5, to obtain the real-time corrected areal rainfall p 2. The hydrological model real-time correction method based on the reinforcement learning A2C algorithm according to claim 1, characterized in that, The area mean precipitation data in step 1 is derived from the measured values of the ground precipitation stations in the research area, the area mean forecast precipitation data is derived from the output results of the mesoscale model CMA-MESO of the meteorological bureau, and the area mean observed flow data is derived from the observed values of the flow observation stations in the research area.

3. The hydrological model real-time correction method based on the reinforcement learning A2C algorithm according to claim 1, characterized in that, The conceptual hydrological model M in step 2.2 is the Xin'anjiang model.

4. The hydrological model real-time correction method based on the reinforcement learning A2C algorithm according to claim 1, characterized in that, Step 2.

3. Construct the current state h = (h o ,h f ) and compute the value and its probability distribution π o (a | h) of the correction action a = (δp f , δp θ ) according to the following steps: Step 2.3.1, constructing observation state h o for calculating the correction amount of the area-averaged rain amount on the ground δp at time i to t o : By stitching the areal rainfall at i to t time Simulated flow And the observed areal flow Constructing the observed state h o : h o sent into the observation decision network π o , obtaining the probability distribution of δp o . π o (δp o ∣h o ) By sampling the value of δp o (δp o ∣h o ) to obtain the specific value of δp o ; where the observation policy network π o is a multi-layer perceptron (MLP), and δp o is a specific value obtained by reparameterization, specifically: δp o = μ + σ ⊙ z, where μ and σ are Gaussian distribution parameters of the output of π o , and z is a generated random number sequence subject to a standard normal distribution. Step 2.3.2, constructing the predicted state h f for calculating the areal forecast rainfall correction δp from time t+1 to t+τ f : By splicing the surface mean rainfall at i to t time Surface mean forecast rainfall p f And surface mean rainfall correction amount δp o , Construct a forecast state h f : h f sent into the prediction decision network π f , the probability distribution of the face mean precipitation correction amount δp f : π f (δp f ∣h f ) By sampling the value of δp f (δp f ∣h f ) to obtain the specific value of δp f ; where the prediction policy network p f is a long short-term memory network (LSTM), and f the specific value of is obtained by reparameterization, which is the same principle as the specific sampling process of o ​ Step 2.3.

3. Compute the probability distribution of the correction action a = (δp o , δp f ) given the current state h = (h o , h f ): π θ (a | h) = π o (δp o | h o ) · π f (δp f | | h f (δp o )) 5. The hydrological model real-time correction method based on the reinforcement learning A2C algorithm according to claim 4, characterized in that, Step 2.5 updates the policy network p based on the A2C algorithm with parameters 0, specifically including the following steps: θ Step 2.5 updates the policy network p based on the A2C algorithm with parameters 0, specifically including the following steps: Step 2.5.

1. Simulating runoff with the revised rainfall data: Runoff is simulated with the conceptual hydrological model M using the revised areal rainfall p o and the areal forecast rainfall p f ​ q c = M(p o ⊕p f ) Step 2.5.2, Delayed reward computation: At time t, store the current simulated traffic to buffer A, marking its time range as [t+1, t+τ]; when the system actually arrives at time t+τ, obtain the real traffic from observation data Compute delayed reward r: Step 2.5.3, Update experience pool: store the experience (h, a, None, h') at time t in a temporary buffer B, marked as to be updated reward; when time t + τ arrives, retrieve the experience at time t from B, fill in the reward r; check the capacity of the experience pool D, if |D| = d, remove the oldest experience; store the complete experience (h, a, r, h ′ ) in the experience pool D; where h ′ is the state vector at the next time step. Step 2.5.4, update parameters θ: when there are at least B complete experiences in experience pool D, randomly sample B samples from experience pool D Then update the parameters ω of the value network V ω : by the gradient of the loss function and the update rate α ω , update ω to be: Then update the parameters θ of the policy network π θ : by the policy gradient and the update rate α θ , update θ to be: Step 2.5.5, update next frame data: time update: i <- i + 1, t <- t + 1; surface mean precipitation p o and surface mean observed flow q o update to current time t, surface mean forecasted precipitation p f update to t + τ.

6. The hydrological model real-time correction method based on the reinforcement learning A2C algorithm according to claim 1, characterized in that, The specific process of training the model with the training set in step 3 is as follows: initializing parameters, setting t max ; initializing the areal mean rainfall p o , the areal mean predicted rainfall p f , and the areal mean observed flow q o in the training set; and then cyclically executing steps 2.2 to 2.5 until t reaches t max ; The specific process of verifying the model by using the verification set is: initializing parameters, setting t max ; initializing the surface mean rainfall p o , the surface mean predicted rainfall p f , and the surface mean observed flow q o in the data in the verification set; Then steps 2.2 to 2.5 are executed in a loop, recording the simulated flow rate during each loop and the observed flow rate q f , respectively and are used to calculate the Nash-Sutcliffe efficiency coefficient NSE t and the root mean square error RMSE t for each loop Further, the average Nash efficiency coefficient is calculated and the average root mean square error as the final evaluation index: If the final evaluation index satisfies the following conditions: the average Nash efficiency coefficient and the average root mean square error is less than 10% of the standard deviation of historical flow observations, it indicates that the model verification is passed; otherwise, by resetting the parameters, optimizing the training data, iterating the training and verification, until the final evaluation index meets the requirements.

Images