Hydrological model real-time correction method based on reinforcement learning A2C algorithm

Abstract

The invention discloses a hydrological model real-time correction method based on a reinforcement learning A2C algorithm. The method comprises the following steps that 1, basic data of a research area are obtained and preprocessed; 2, constructing a real-time correction model based on a reinforcement learning A2C algorithm; 3, training and verifying the real-time correction model; and 4, correcting the rainfall in real time by using the real-time correction model. By coupling the reinforcement learning A2C algorithm and the hydrological model, the problems that a traditional method is slow in convergence and prone to falling into local optimum are solved, and high-precision and self-adaptive flood forecast real-time correction is achieved. According to the method, the accuracy and the dynamic response capability of flood forecasting are remarkably improved, and reliable technical support is provided for flood control and disaster reduction and water resource management.

Description

Technical Field

[0001] The present invention belongs to the field of hydrology, water resources and artificial intelligence technology, and in particular relates to a real-time correction method for a hydrological model based on a reinforcement learning A2C algorithm. Background Art

[0002] With the impact of global climate change and human activities, the frequency and intensity of extreme hydrological events, such as floods, are increasing. Accurate flood forecasting is crucial for flood prevention and disaster reduction, the rational allocation of water resources, and protecting people's lives and property. As a core tool for flood forecasting, hydrological models play a vital role in flood simulation and prediction. However, traditional hydrological models face numerous challenges in practical application.

[0003] On the one hand, hydrological processes are highly complex and uncertain, influenced by multiple factors such as meteorological conditions, underlying surface conditions, and human activities. Model parameters are difficult to accurately determine, leading to discrepancies between simulation results and actual observations. On the other hand, existing real-time correction methods for hydrological models are mostly based on traditional optimization algorithms, such as least squares and Kalman filtering. These methods often suffer from slow convergence and proneness to local optimality when dealing with nonlinear and dynamically changing hydrological systems. This makes them difficult to adapt to complex and changing flood processes and incapable of achieving high-precision real-time flood forecasting.

[0004] In recent years, the application of artificial intelligence (AI) technology in the field of hydrology has gradually gained momentum, providing new ideas and methods for the real-time correction of hydrological models. Reinforcement learning, a key branch of AI, can continuously optimize decision-making strategies to maximize long-term rewards through interactive learning between an agent and its environment. However, research on the application of reinforcement learning to the real-time correction of hydrological models is still in its exploratory stage. How to effectively combine reinforcement learning algorithms with hydrological models to improve the real-time and accuracy of flood forecasting remains an urgent challenge. Therefore, a new reinforcement learning-based real-time correction method for hydrological models is urgently needed to enhance the accuracy and reliability of flood forecasting. Summary of the Invention

[0005] The purpose of the present invention is to provide a real-time correction method for a hydrological model based on the reinforcement learning A2C algorithm to solve the above technical problems.

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] The present invention discloses a real-time correction method for a hydrological model based on a reinforcement learning A2C algorithm, the method comprising the following steps:

[0008] Step 1: Obtain basic data for the study area and preprocess it: Select small and medium-sized watersheds with more than ten years of continuous hydrological and meteorological monitoring data as the study area, and obtain historical basic data for the study area over many years, including daily-scale average rainfall data, average forecast rainfall data, and average observed flow data. Then, preprocess the obtained basic data by using linear interpolation to fill in missing values ​​in the average rainfall data, average forecast rainfall data, and average observed flow data to ensure time series continuity.

[0009] Step 2: Construct a real-time correction model based on reinforcement learning A2C algorithm: Construct a real-time correction model based on reinforcement learning A2C algorithm. The real-time correction model consists of the conceptual hydrological model M and the policy network π θ The conceptual hydrological model M is used to receive the average rainfall data and output the simulated flow to construct the state vector h; the strategy network π θ By observing the decision network π o and forecast decision network π f Composition, respectively receive the observation state h o and the forecast state h f , generating the correction value of the average rainfall amount on the surface δp o and the correction value of the average rainfall forecast δp f Used to correct rainfall data, its parameter θ is updated by the A2C algorithm;

[0010] The construction process of the real-time correction model specifically includes the following steps:

[0011] Step 2.1, parameter and data initialization: Initialize parameters and data, the parameters include: prediction step length τ, τ∈N + ; Time window length T, T∈N + ; Current time t, t∈N + , t≤t max , t max Set according to the time length of the data obtained in step 1; past time i, i∈N + ; Discount factor γ, 0<γ<1, γ∈R; Experience pool D, Experience pool capacity d, d∈N + ; Batch size B, B∈N + And B≤d; the data include the average rainfall per area: Average rainfall forecast per area: Average observed flow rate:

[0012] Step 2.2: Calculate the simulated flow: Use the conceptual hydrological model M to calculate the p o Calculate the simulated flow

[0013] q s =M(p o )

[0014] Step 2.3: Construct the current state and calculate the correction action: Construct the current state h = (h o ,h f ), calculate the correction action a=(δp o ,δp f ) and its probability distribution π θ (a|h); where δp o is the correction value of the average rainfall amount from time i to time t, δp f is the correction value of the surface average forecast rainfall from t+1 to t+τ, h o is the observation state, h f Forecast state, π θ For strategic networks;

[0015] Step 2.4: Correct the surface average rainfall and surface average forecast rainfall data: Correct the surface average rainfall from time i to time t And the predicted rainfall p from t+1 to t+τ f :

[0016]

[0017] p f ←p f +δp f

[0018] Step 2.5: Update the policy network π θ Parameter θ: Update the policy network π based on the A2C algorithm θ Parameter θ;

[0019] Then repeat steps 2.2 to 2.5 until t reaches t max ;

[0020] Step 3: Training and validating the real-time correction model: First, the data pre-processed in step 1 is divided into a training set and a validation set in proportion; then, the real-time correction model is trained using the training set, and finally, the real-time correction model is validated using the validation set; until the model validation passes, the real-time correction model training is completed;

[0021] Step 4: Use the real-time correction model to correct rainfall in real time: obtain real-time data and preprocess it, including real-time average rainfall, average forecast rainfall, and average observed flow; initialize parameters and set t max =+∞, use real-time data to initialize the average surface rainfall p o , average surface rainfall forecast p f , average observed flow rate q o; Load the policy network π of the trained real-time correction model θ Then follow steps 2.2 to 2.5 to get the real-time corrected average rainfall.

[0022] Furthermore, the area-averaged fallen rainfall data described in step 1 are derived from the measured values ​​of the ground rain gauges in the study area, the area-averaged forecast rainfall data are derived from the output results of the Meteorological Bureau's mesoscale model CMA-MESO, and the area-averaged observed flow data are derived from the observed values ​​of the flow observation stations in the study area.

[0023] Furthermore, the conceptual hydrological model M in step 2.2 is the Xin'an River model.

[0024] Furthermore, the current state h=(h o ,h f ), calculate the correction action a=(δp o ,δp f ) and its probability distribution π θ (a|h), specifically including the following steps:

[0025] Step 2.3.1. Construct the observation state h o Used to calculate the correction value δp of the average rainfall on the surface from time i to t o :

[0026] By splicing the average surface rainfall from time i to time t Simulated traffic Average observed flow Construct observation state h o :

[0027]

[0028] h o Send to the observation decision network π o , we get δp o The probability distribution of :

[0029] π o (δp o ∣h o )

[0030] Through π o (δp o ∣h o ) sampling to obtain δp o The specific value of

[0031] Among them, the observation decision network π o is a multi-layer perceptron (MLP), δp oThe specific value of is obtained by reparameterization, specifically: δp o =μ+σ⊙z, where μ and σ are π o The output Gaussian distribution parameters, z is the generated random number sequence that obeys the standard normal distribution;

[0032] Step 2.3.2: Construct the predicted state h f Used to calculate the correction value δp of the average rainfall forecast from t+1 to t+τ f :

[0033] By splicing the average surface rainfall from time i to time t Average surface rainfall forecast p f and the correction value of average rainfall on the surface δp o , construct the forecast state h f :

[0034]

[0035] h f Send to the forecast decision network π f , correction value of average rainfall forecast δp f The probability distribution of :

[0036] π f (δp f ∣h f )

[0037] Through π f (δp f ∣h f ) sampling to obtain δp f The specific value of

[0038] Among them, the forecast decision network π f is the long short-term memory network LSTM, δp f The specific value of is obtained by reparameterization, the principle is the same as δp o Specific sampling process;

[0039] Step 2.3.3, calculate the correction action a = (δp o ,δp f ) in the current state h=(h o ,h f ) is the probability distribution under:

[0040] π θ (a|h)=π o (δp o ∣h o )·π f (δp f ∣∣h f (δp o)).

[0041] Furthermore, the strategy network π is updated based on the A2C algorithm described in step 2.5. θ The parameters θ include the following steps:

[0042] Step 2.5.1: Use the corrected rainfall data to simulate flow: Use the conceptual hydrological model M to simulate the corrected average rainfall p o and the average surface rainfall p f To simulate traffic

[0043]

[0044] Step 2.5.2, Delayed Reward Calculation: At time t, temporarily store the current simulated traffic Go to cache area A and mark its time range as [t+1,t+τ]; when the system actually reaches time t+τ, obtain the real flow from the observation data Calculate the delayed reward r:

[0045]

[0046] Step 2.5.3. Update the experience pool: Store the experience (h, a, None, h′) at time t into a temporary buffer B and mark it as a reward to be updated. When time t+τ arrives, take the experience at time t from B and fill it with reward r. Check the capacity of experience pool D. If |D| = d, remove the oldest experience. Store the complete experience (h, a, r, h′) into experience pool D. Where h′ is the state vector at the next time.

[0047] Step 2.5.4, update parameter θ: when there are at least B complete experiences in the experience pool D, randomly extract B samples from the experience pool D Then update the value network V ω Parameter ω: through the loss function Gradient and update rate α ω , update ω to: Then update the policy network π θ Parameters θ: through policy gradient and update rate α θ , update θ to:

[0048] Step 2.5.5, update the next frame of data: update time: i←i+1, t←t+1; average rainfall p o and the average observed flow q o Updated to the current time t, the average surface rainfall forecast p f Update to t+τ.

[0049] Furthermore, the specific process of training the model using the training set in step 3 is as follows: initializing the parameters, setting t max ; Use the data in the training set to initialize the average rainfall amount p o , average surface rainfall forecast p f , average observed flow rate q o ; Then loop through steps 2.2 to 2.5 until t reaches t max Set value;

[0050] The specific process of using the validation set to validate the model is as follows: initializing the parameters, setting t max ; Use the data in the validation set to initialize the average rainfall amount p o , average surface rainfall forecast p f , average observed flow rate q o Then repeat steps 2.2 to 2.5 and record the simulated flow rate during each cycle. and the average observed flow q f , respectively and Used to calculate the Nash efficiency coefficient NSE for each cycle t and root mean square error RMSE t :

[0051]

[0052]

[0053] Then calculate the average Nash efficiency coefficient and the average root mean square error As the final evaluation indicator:

[0054]

[0055]

[0056] If the final evaluation index meets the following conditions: Average Nash efficiency coefficient Average Nash efficiency coefficient And the average root mean square error If it is lower than 10% of the standard deviation of historical flow observations, it indicates that the model validation has passed; otherwise, the parameters are reset, the training data is optimized, and training and validation are iterated until the final evaluation indicators meet the requirements.

[0057] The present invention achieves the following beneficial effects: By coupling the reinforcement learning A2C algorithm with a hydrological model, it overcomes the slow convergence and local optima of traditional methods, enabling highly accurate, adaptive, and real-time correction of flood forecasts. This method significantly improves the accuracy and dynamic response capabilities of flood forecasts, providing reliable technical support for flood prevention, disaster reduction, and water resource management.

[0058] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 This is a flow chart of the method of the present invention;

[0060] Figure 2 Schematic diagram of the construction process of the real-time correction model in Example 1;

[0061] Figure 3 is the correction value of average rainfall in Example 1, δp O and the forecast rainfall correction δp f Schematic diagram of the calculation;

[0062] Figure 4 is the average rainfall p in Example 1 o , observed flow q o and the predicted rainfall p f Update diagram of . DETAILED DESCRIPTION

[0063] Example 1

[0064] This embodiment discloses a real-time correction method for a hydrological model based on a reinforcement learning A2C algorithm, such as Figure 1 As shown, the following steps are included:

[0065] Step 1: Obtain basic data of the study area and pre-process it: Select small and medium-sized watersheds (with an area of ​​100 to 10,000 km2) with more than ten years of continuous hydrological and meteorological monitoring data. 2 ) as the research area. This example selects a tributary of the Yangtze River (with an area of ​​about 5000km 2 ) is the study area. The basin includes 10 rainfall stations and 3 flow observation stations. It has the characteristics of humid and semi-humid climate. Precipitation is concentrated during the flood season (June-September) and flood disasters occur frequently.

[0066] Basic data for the study area from 2015 to 2024 were obtained, including daily average surface rainfall data, area-averaged forecast rainfall data, and area-averaged observed flow data. The area-averaged surface rainfall data were derived from measured values ​​at surface rain gauges within the study area, the area-averaged forecast rainfall data were derived from the output of the Meteorological Administration's mesoscale model, CMA-MESO, and the area-averaged observed flow data were derived from observations at flow observation stations within the study area. The obtained basic data were then preprocessed by using linear interpolation to fill in missing values ​​in the area-averaged surface rainfall data, area-averaged forecast rainfall data, and area-averaged observed flow data to ensure time series continuity.

[0067] Step 2: Construct a real-time correction model based on reinforcement learning A2C algorithm: Construct a real-time correction model based on reinforcement learning A2C algorithm. The real-time correction model consists of the conceptual hydrological model M and the policy network π θ The conceptual hydrological model M is used to receive the average rainfall data and output the simulated flow to construct the state vector h. The conceptual hydrological model M of this embodiment adopts the Xin'an River model; the strategy network π θ By observing the decision network π o and forecast decision network π f Composition, respectively receive the observation state h o and the forecast state h f , generate the correction value δp of the average rainfall (historical rainfall) o and the correction value of the average surface rainfall forecast (future rainfall) δp f Used to correct rainfall data, its parameter θ is updated by the A2C algorithm.

[0068] like Figure 2 As shown in FIG, the construction process of the real-time correction model specifically includes the following steps:

[0069] Step 2.1, parameter and data initialization: specifically including:

[0070] Prediction step: τ = 3, representing the length of the future prediction period, τ∈N + ;

[0071] Time window length: T = 4, indicating the range of historical data used for model calculation, T∈N + ;

[0072] Current time: t = 4, t∈N + , t≤t max ;t max Set according to the length of time to obtain data in step 1;

[0073] Past time: i=1, i∈N + ;

[0074] Discount factor: γ = 0.99, used to balance the weight of future rewards, 0 < γ < 1, γ ∈ R;

[0075] Experience Pool: Experience pool capacity: d = 100, defines the maximum amount of experience stored in the experience pool, d∈N + ;

[0076] Batch size: B = 32, the number of samples drawn from the experience pool each time the model is trained, B∈n + And B≤d;

[0077] Average rainfall per surface area:

[0078] Average rainfall forecast per area:

[0079] Average observed flow rate:

[0080] Step 2.2: Calculate the simulated flow: Use the conceptual hydrological model M (Xin'anjiang model) to calculate the p o Calculate the simulated flow

[0081] q s =M(p o )

[0082] Step 2.3: Construct the current state and calculate the correction action: Construct the current state h = (h o ,h f ), calculate the correction action a=(δp o ,δp f ) and its probability distribution π θ (a|h). Among them, δp o is the correction value of the average rainfall amount from time i to time t, δp f is the correction value of the surface average forecast rainfall from t+1 to t+τ, h o is the observation state, h f Forecast state, π θ For the policy network, Figure 3 As shown, the specific steps include:

[0083] Step 2.3.1. Construct the observation state h o Used to calculate the correction value δp of the average rainfall on the surface from time i to t o :

[0084] By splicing the average surface rainfall from time i to time t Simulated traffic Average observed flow Construct observation state h o :

[0085]

[0086] h o Send to the observation decision network π o , we get δp o The probability distribution of :

[0087] π o (δp o ∣h o )

[0088] Through π o (δp o ∣h o ) sampling to obtain δp o The specific value of .

[0089] Among them, the observation decision network π o is a multi-layer perceptron (MLP), δp o The specific value of can be obtained by reparameterization, specifically: δp o =μ+σ⊙z, where μ and σ are π o The output Gaussian distribution parameters, z is a random number sequence generated that obeys the standard normal distribution.

[0090] Step 2.3.2: Construct the predicted state h f Used to calculate the correction value δp of the average rainfall forecast from t+1 to t+τ f :

[0091] By splicing the average surface rainfall from time i to time t Average surface rainfall forecast p f and the correction value of average rainfall on the surface δp o , construct the forecast state h f :

[0092]

[0093] h f Send to the forecast decision network π f , correction value of average rainfall forecast δp f The probability distribution of :

[0094]

[0095] Through π f (δp f ∣h f ) sampling to obtain δp f The specific value of .

[0096] Among them, the forecast decision network π fis a long short-term memory network (LSTM). δp f The specific value of can be obtained by reparameterization, the principle is the same as δp o The specific sampling process.

[0097] Step 2.3.3, calculate the correction action a = (δp o ,δp f ) in the current state h=(h o ,h f ) is the probability distribution under:

[0098] π θ (a|h)=π o (δp o ∣h o )·π f (δp f ∣∣h f (δp o ))

[0099] In the logic of real-time correction of hydrological models, we first need to use the existing observation data (including the average surface rainfall p o , simulated flow q s and the average observed flow q o ) to determine the correction value δp of the average rainfall in the current period (i to t) o , which is a correction of historical data. Then, based on the correction results of historical data, combined with the average surface rainfall forecast p f , and then determine the correction amount δp of the average surface rainfall forecast in the future period (t+1 to t+τ) f This sequence conforms to the logical process from known to unknown, from history to the future, and can more reasonably make real-time corrections to the hydrological model.

[0100] Step 2.4: Correct the average surface rainfall and the average surface rainfall forecast data: Correct the average surface rainfall from time i to time t And the predicted rainfall p from t+1 to t+τ f :

[0101]

[0102] p f ←p f +δp f

[0103] Step 2.5: Update the policy network π θ Parameter θ: Update the policy network π based on the A2C algorithm θ Parameter θ; specifically including the following steps:

[0104] Step 2.5.1: Use the corrected rainfall data to simulate flow: Use the conceptual hydrological model M to simulate the corrected average rainfall p o and the average surface rainfall p f To simulate traffic

[0105]

[0106] Step 2.5.2, Delayed Reward Calculation: At time t, temporarily store the current simulated traffic Go to cache area A and mark its time range as [t+1,t+τ]. When the system actually reaches time t+τ, the real flow is obtained from the observation data. Calculate the delayed reward r:

[0107]

[0108] Step 2.5.3. Update the experience pool: Store the experience (h, a, None, h′) at time t in a temporary buffer B and mark it as a pending reward. When time t + τ arrives, extract the experience at time t from B and fill it with reward r (calculated in step 2.5.2). Check the capacity of experience pool D. If |D| = d, remove the oldest experience. Store the complete experience (h, a, r, h′) in experience pool D. Where h′ is the state vector at the next time.

[0109] Step 2.5.4, update parameter θ: when there are at least B complete experiences in the experience pool D, randomly extract B samples from the experience pool D Then update the value network V ω Parameter ω: through the loss function Gradient and update rate α ω , update ω to: Then update the policy network π θ Parameters θ: through policy gradient and update rate α θ , update θ to:

[0110] Step 2.5.5, update the next frame of data: update time: i←i+1, t←t+1; average rainfall p o and the average observed flow q o Updated to the current time t, the average surface rainfall forecast p f Update to t+τ, such as Figure 4 shown.

[0111] Then repeat steps 2.2 to 2.5 until t reaches t max .

[0112] Step 3: Train and validate the real-time correction model: First, divide the data preprocessed in Step 1 into a training set and a validation set according to a certain ratio. In this embodiment, the data is divided into training and validation sets at a ratio of 9:1, that is, the data from 2015 to 2023 is used as the training set, and the data from 2024 is used as the validation set, ensuring that the time ranges of the training set and validation set do not overlap.

[0113] Then use the training set to train the model. The specific process is: initialize the parameters, set t max =9×12×30; initialize the average surface rainfall p using the data in the training set o , average surface rainfall forecast p f , average observed flow rate q o At this time, the actual time corresponding to t=4 in step 2.1 is 2015-01-04, and the actual time corresponding to i=1 is 2015-01-01. Then, steps 2.2 to 2.5 are executed repeatedly until t reaches t max Set the value.

[0114] Finally, the model is verified using the validation set. The specific process is as follows: initialize the parameters, set t max =1×12×30; initialize the average surface rainfall p using the data in the validation set o , average surface rainfall forecast p f , average observed flow rate q o At this time, the actual time corresponding to t=4 in step 2.1 is 2024-01-04, and the actual time corresponding to i=1 is 2024-01-01.

[0115] Then repeat steps 2.2 to 2.5 and record the simulated flow rate during each cycle. and the average observed flow q f , respectively and Used to calculate the Nash efficiency coefficient NSE for each cycle t and root mean square error RMSE t :

[0116]

[0117]

[0118] Then calculate the average Nash efficiency coefficient and the average root mean square error As the final evaluation indicator:

[0119]

[0120]

[0121] If the final evaluation index meets the following two conditions at the same time, it means that the model verification has passed: 1) Average Nash efficiency coefficient This shows that the model prediction results are highly consistent with the measured data; 2) Average root mean square error It is lower than 10% of the standard deviation of historical flow observations, indicating that the model prediction error is within a reasonable range.

[0122] If any indicator fails to meet the standard, the following improvement measures should be taken: (1) reset the model parameters (including discount factor, time window length, etc.); (2) optimize the training data, check the data quality and deal with outliers, and introduce more historical data to enhance the generalization ability when necessary; (3) iterate the training and verification until the final evaluation indicator meets the requirements.

[0123] Step 4: Use the real-time correction model to correct rainfall in real time: obtain real-time data and preprocess it, including real-time average rainfall, average forecast rainfall, and average observed flow. Initialize the parameters and set t max =+∞, use real-time data to initialize the average surface rainfall p o , average surface rainfall forecast p f , average observed flow rate q o At this time, the actual time corresponding to t=4 in step 2.1 is 2025-01-04, and the actual time corresponding to i=1 is 2025-01-01. Load the policy network π of the trained real-time correction model θ Then follow steps 2.2 to 2.5 to get the real-time corrected average rainfall.

[0124] Finally, it should be noted that the above description is only used to illustrate the technical solution of the present invention and is not intended to limit it. Although the present invention has been described in detail with reference to the preferred arrangement scheme, those skilled in the art should understand that the technical solution of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention.

Claims

1. A real-time correction method for a hydrological model based on reinforcement learning A2C algorithm, characterized in that: The method comprises the following steps: Step 1: Obtain basic data for the study area and preprocess it: Select small and medium-sized watersheds with more than ten years of continuous hydrological and meteorological monitoring data as the study area, and obtain historical basic data for the study area over many years, including daily-scale average rainfall data, average forecast rainfall data, and average observed flow data. Then, preprocess the obtained basic data by using linear interpolation to fill in missing values ​​in the average rainfall data, average forecast rainfall data, and average observed flow data to ensure time series continuity. Step 2: Construct a real-time correction model based on reinforcement learning A2C algorithm: Construct a real-time correction model based on reinforcement learning A2C algorithm. The real-time correction model consists of the conceptual hydrological model M and the policy network π θ The conceptual hydrological model M is used to receive the average rainfall data and output the simulated flow to construct the state vector h; the strategy network π θ By observing the decision network π o and forecast decision network π f Composition, respectively receive the observation state h o and the forecast state h f , generating the correction value of the average rainfall amount on the surface δp o and the correction value of the average rainfall forecast δp f Used to correct rainfall data, its parameter θ is updated by the A2C algorithm; The construction process of the real-time correction model specifically includes the following steps: Step 2.1, parameter and data initialization: Initialize parameters and data, the parameters include: prediction step length τ, τ∈N + ; Time window length T, T∈N + ; Current time t, t∈N + , t≤t max , t max Set according to the time length of the data obtained in step 1; past time i, i∈N + ; Discount factor γ, 0<γ<1, γ∈R; Experience pool D, Experience pool capacity d, d∈N + ; Batch size B, B∈N + And B≤d; the data include the average rainfall per area: Average rainfall forecast per area: Average observed flow rate: Step 2.2: Calculate the simulated flow: Use the conceptual hydrological model M to calculate the p o Calculate the simulated flow q s =M(p o ) Step 2.3: Construct the current state and calculate the correction action: Construct the current state h = (h o ,h f ), calculate the correction action a=(δp o ,δp f ) and its probability distribution π θ (a|h); where δp o is the correction value of the average rainfall amount from time i to time t, δp f is the correction value of the surface average forecast rainfall from t+1 to t+τ, h o is the observation state, h f Forecast state, π θ For strategic networks; Step 2.4: Correct the surface average rainfall and surface average forecast rainfall data: Correct the surface average rainfall from time i to time t And the predicted rainfall p from t+1 to t+τ f : p f ←p f +δp f Step 2.5: Update the policy network π θ Parameter θ: Update the policy network π based on the A2C algorithm θ Parameter θ; Then repeat steps 2.2 to 2.5 until t reaches t max ; Step 3: Training and validating the real-time correction model: First, the data pre-processed in step 1 is divided into a training set and a validation set in proportion; then, the real-time correction model is trained using the training set, and finally, the real-time correction model is validated using the validation set; until the model validation passes, the real-time correction model training is completed; Step 4: Use the real-time correction model to correct rainfall in real time: obtain real-time data and preprocess it, including real-time average rainfall, average forecast rainfall, and average observed flow; initialize parameters and set t max =+∞, use real-time data to initialize the average surface rainfall p o , average surface rainfall forecast p f , average observed flow rate q o ; Load the policy network π of the trained real-time correction model θ Then follow steps 2.2 to 2.5 to get the real-time corrected average rainfall.

2. A real-time correction method for a hydrological model based on reinforcement learning A2C algorithm according to claim 1, characterized in that: The average surface rainfall data mentioned in step 1 are derived from the measured values ​​of the surface rain gauges in the study area, the average surface forecast rainfall data are derived from the output results of the CMA-MESO mesoscale model of the Meteorological Administration, and the average surface observed flow data are derived from the observed values ​​of the flow observation stations in the study area.

3. The real-time correction method of the hydrological model based on the reinforcement learning A2C algorithm according to claim 1 is characterized in that: The conceptual hydrological model M described in step 2.2 is the Xin'an River model.

4. The real-time correction method of the hydrological model based on the reinforcement learning A2C algorithm according to claim 1 is characterized in that: Step 2.3 constructs the current state h=(h o ,h f ), calculate the correction action a=(δp o ,δp f ) and its probability distribution π θ (a|h), specifically including the following steps: Step 2.3.

1. Construct the observation state h o Used to calculate the correction value δp of the average rainfall on the surface from time i to t o : By splicing the average surface rainfall from time i to time t Simulated traffic Average observed flow Construct observation state h o : h o Send to the observation decision network π o , we get δp o The probability distribution of : p o (δp o ∣h o ) Through π o (δp o ∣h o ) sampling to obtain δp o The specific value of Among them, the observation decision network π o is a multi-layer perceptron (MLP), δp o The specific value of is obtained by reparameterization, specifically: δp o =μ+σ⊙z, where μ and σ are π o The output Gaussian distribution parameters, z is the generated random number sequence that obeys the standard normal distribution; Step 2.3.2: Construct the predicted state h f Used to calculate the correction value δp of the average rainfall forecast from t+1 to t+τ f : By splicing the average surface rainfall from time i to time t Average surface rainfall forecast p f and the correction value of average rainfall on the surface δp o , construct the forecast state h f : h f Send to the forecast decision network π f , correction value of average rainfall forecast δp f The probability distribution of : p f (δp f ∣h f ) Through π f (δp f ∣h f ) sampling to obtain δp f The specific value of Among them, the forecast decision network π f is the long short-term memory network LSTM, δp f The specific value of is obtained by reparameterization, the principle is the same as δp o Specific sampling process; Step 2.3.3, calculate the correction action a = (δp o ,δp f ) in the current state h=(h o ,h f ) is the probability distribution under: p θ (a∣h)=π o (δp o ∣h o )·p f (δp f ∣∣h f (δp o ))。 5. The real-time correction method of the hydrological model based on the reinforcement learning A2C algorithm according to claim 4 is characterized in that: Step 2.5: Update the policy network π based on the A2C algorithm θ The parameters θ include the following steps: Step 2.5.1: Use the corrected rainfall data to simulate flow: Use the conceptual hydrological model M to simulate the corrected average rainfall p o and the average surface rainfall p f To simulate traffic q c =M(p o ⊕p f ) Step 2.5.2, Delayed Reward Calculation: At time t, temporarily store the current simulated traffic Go to cache area A and mark its time range as [t+1,t+τ]; when the system actually reaches time t+τ, obtain the real flow from the observation data Calculate the delayed reward r: Step 2.5.3, update the experience pool: store the experience (h, a, None, h′) at time t into the temporary buffer B and mark it as the reward to be updated; when time t+τ arrives, take the experience at time t from B and fill it with reward r; check the capacity of the experience pool D, if |D| = d, remove the oldest experience; store the complete experience (h, a, r, h ′ ) is stored in the experience pool D; where h ′ is the state vector at the next moment; Step 2.5.4, update parameter θ: when there are at least B complete experiences in the experience pool D, randomly extract B samples from the experience pool D Then update the value network V ω Parameter ω: through the loss function Gradient and update rate α ω , update ω to: Then update the policy network π θ Parameters θ: through policy gradient and update rate α θ , update θ to: Step 2.5.5, update the next frame of data: update time: i←i+1, t←t+1; average rainfall p o and the average observed flow q o Updated to the current time t, the average surface rainfall forecast p f Update to t+τ.

6. The real-time correction method of the hydrological model based on the reinforcement learning A2C algorithm according to claim 1 is characterized in that: The specific process of training the model using the training set in step 3 is as follows: Initialize the parameters, set t max ; Use the data in the training set to initialize the average rainfall amount p o , average surface rainfall forecast p f , average observed flow rate q o ; Then loop through steps 2.2 to 2.5 until t reaches t max Set value; The specific process of using the validation set to validate the model is as follows: initializing the parameters, setting t max ; Use the data in the validation set to initialize the average rainfall amount p o , average surface rainfall forecast p f , average observed flow rate q o ; Then repeat steps 2.2 to 2.5 and record the simulated flow rate during each cycle. and the average observed flow q f , respectively and Used to calculate the Nash efficiency coefficient NSE for each cycle t and root mean square error RMSE t : Then calculate the average Nash efficiency coefficient and the average root mean square error As the final evaluation indicator: If the final evaluation index meets the following conditions: Average Nash efficiency coefficient Average Nash efficiency coefficient And the average root mean square error If it is lower than 10% of the standard deviation of historical flow observations, it indicates that the model validation has passed; otherwise, the parameters are reset, the training data is optimized, and training and validation are iterated until the final evaluation indicators meet the requirements.

Images