Karst basin runoff prediction method based on reinforcement learning and deep learning

Abstract

The invention discloses a karst basin runoff prediction method based on reinforcement learning and deep learning, and the method comprises the steps: carrying out the convolution dimension reduction and feature extraction through a two-dimensional convolution neural network by employing a two-dimensional rainfall cumulative map and a two-dimensional temperature average map of each hour, and the fixed earth surface and underground features of a basin; inputting the extracted hour scale time sequence into a long short-term memory network to construct a 2D-CNN-LSTM deep learning network, optimizing hyper-parameters of the 2D-CNN-LSTM network based on a DQN algorithm of an improved reward function, configuring the optimized hyper-parameters to the 2D-CNN-LSTM network, training the 2D-CNN-LSTM network through a training data set, and obtaining a training result of the 2D-CNN-LSTM network. And verifying the prediction performance of the trained 2D-CNN-LSTM network by using the verification data set, and performing runoff prediction by using the verified 2D-CNN-LSTM network. The method can improve the prediction precision of the model on the runoff of the karst basin.

Description

Technical Field

[0001] The present invention relates to the technical field of hydrological prediction, and in particular to a karst basin runoff prediction method based on reinforcement learning and deep learning. Background Art

[0002] Karst areas face many challenges in rainfall runoff prediction due to their unique geological structure and hydrological characteristics. Karst landforms have complex underground drainage systems and porous rock structures, which make the rainfall runoff process extremely complex and nonlinear. Traditional runoff prediction models, such as physical models and statistical models, are usually difficult to accurately simulate the hydrological process in karst areas, especially when extreme weather events occur frequently, and the prediction accuracy is significantly reduced.

[0003] With the rapid development of deep learning and reinforcement learning technologies, data-driven methods have shown great potential in the field of hydrological prediction. Deep learning, especially convolutional neural networks (CNNs) and long short-term memory networks (LSTMs), has significant advantages in processing high-dimensional spatial data and time series data. CNNs can effectively extract important features from two-dimensional spatial data, while LSTMs are good at capturing long-term dependencies in time series data. However, the performance of deep learning models is highly dependent on the choice of hyperparameters, and how to automatically and effectively optimize these hyperparameters has become a key issue.

[0004] Reinforcement learning, especially the deep Q network (DQN), provides a method for policy optimization in dynamic environments. By defining a suitable reward function, DQN can reduce prediction errors while enhancing the model's ability to respond to extreme weather events. Introducing DQN into the hyperparameter optimization of deep learning models can automatically adjust model parameters and improve the prediction accuracy and robustness of the model in complex environments.

[0005] The karst basin runoff prediction method based on reinforcement learning and deep learning proposed in this paper uses two-dimensional rainfall maps, temperature maps, and surface and underground features of the basin to extract features through CNN, and performs time series prediction through LSTM. Through DQN optimization of hyperparameters, comprehensive consideration of prediction errors and extreme weather event responses, an efficient and accurate runoff prediction model is constructed. This model can not only accurately capture the rainfall runoff characteristics in karst areas, but also effectively cope with complex meteorological and geological conditions, provide reliable hydrological forecasts, and provide technical support for disaster prevention and mitigation and water resources management. Summary of the invention

[0006] In view of the unique geological and hydrological characteristics of karst areas, this paper proposes a karst basin runoff prediction method based on reinforcement learning and deep learning, aiming to solve the problem that existing models are difficult to accurately simulate the complex runoff process in karst areas. By combining reinforcement learning and deep learning techniques and using a large amount of historical data to train the model, accurate simulation and prediction of rainfall runoff processes in karst areas can be achieved.

[0007] The present invention achieves the above-mentioned purpose through the following technical solutions:

[0008] A karst basin runoff prediction method based on reinforcement learning and deep learning includes the following steps:

[0009] Step 1: Obtain a two-dimensional rainfall accumulation map, a two-dimensional temperature average map, and fixed surface and underground features of the basin every hour, wherein the fixed surface and underground features of the basin include a two-dimensional land use map, a two-dimensional soil type map, and a two-dimensional geological map;

[0010] Step 2: Construct model training and validation data sets based on the data obtained in step 1. The training set is used for model parameter training, and the validation set is used to evaluate model performance.

[0011] Step 3: Take the data obtained in step 1 as input, perform convolution dimension reduction and feature extraction through the two-dimensional convolutional neural network 2D-CNN, input the extracted hourly time series into the long short-term memory network LSTM, and build a 2D-CNN-LSTM deep learning network for runoff prediction;

[0012] Step 4: Redesign the reward function R in the DQN algorithm by introducing flood prediction accuracy, improve the DQN algorithm, and obtain a DQN algorithm based on the improved reward function;

[0013] Step 5: Optimize the hyperparameters of the 2D-CNN-LSTM network constructed in step 3 based on the DQN algorithm with improved reward function;

[0014] Step 6: Configure the optimized hyperparameters to the 2D-CNN-LSTM network, train the 2D-CNN-LSTM network using the training data set, and save the trained 2D-CNN-LSTM network;

[0015] Step 7: Use the validation data set to verify the prediction performance of the trained 2D-CNN-LSTM network; Step 8: Use the validated 2D-CNN-LSTM network to perform runoff prediction.

[0016] Furthermore, step 1 specifically includes:

[0017] Step 11: Rainfall data collection: Collect rainfall data from each station in the basin and generate a two-dimensional rainfall accumulation map per hour by interpolation;

[0018] Step 12: Temperature data collection: Obtain temperature data in the basin from the meteorological station, calculate the average temperature per hour, and generate a two-dimensional temperature average map by interpolation;

[0019] Step 13: Land use data collection: Obtain land use data in the watershed and generate a two-dimensional land use map;

[0020] Step 14: Soil type data collection: Collect soil type data in the watershed and generate a two-dimensional soil type map;

[0021] Step 15: Geological data collection: Obtain geological data within the basin and generate a two-dimensional geological map.

[0022] Furthermore, step 2 specifically includes:

[0023] Step 21: Data integration: The two-dimensional rainfall accumulation map, the two-dimensional temperature average map, the two-dimensional land use map, the two-dimensional soil type map and the two-dimensional geological map obtained in step 1 are unified in resolution to form a multidimensional data set;

[0024] Step 22: Data cleaning: Clean the integrated data to remove missing values ​​and outliers to ensure data integrity and consistency;

[0025] Step 23: Data standardization: Standardize the data and convert data of different dimensions into unified dimensions, including one-hot encoding of land use maps, soil type maps, and geological maps;

[0026] Step 24: Data segmentation: divide the integrated data set into a training data set and a validation data set;

[0027] Step 25: Data enhancement: Use data enhancement technology to improve the robustness of the model. The data enhancement technology includes noise addition and data smoothing.

[0028] Furthermore, step 3 specifically includes:

[0029] Step 31: Design a 2D-CNN-LSTM architecture that combines a convolutional neural network (CNN) and a long short-term memory (LSTM) network to extract both spatial features and capture temporal dynamics;

[0030] Step 32: Convolutional layer design: Design multiple convolutional layers to extract spatial features of the 2D rainfall accumulation map, 2D temperature average map, 2D land use map, 2D soil type map, and 2D geological map;

[0031] Step 33: Pooling layer design: Add a pooling layer after the convolution layer to reduce dimensionality and computational complexity;

[0032] Step 34: LSTM layer design: Add LSTM layer after convolution layer and pooling layer to capture timing features and dynamic changes;

[0033] Step 35: Fully connected layer design: Add a fully connected layer after the LSTM layer to integrate the extracted features and output the rainfall runoff prediction results;

[0034] Step 36: Network parameter initialization: Initialize the network parameters, including weights and biases;

[0035] Step 37: Loss function design: Select a suitable loss function to measure the difference between the predicted result and the true value. The loss function includes mean square error (MSE).

[0036] Step 38: Optimization algorithm selection: Select a suitable optimization algorithm for optimizing network parameters, wherein the optimization algorithm includes the Adam optimizer.

[0037] Furthermore, in step 4, the reward function R in the DQN algorithm is redesigned by introducing the flood prediction accuracy, including:

[0038] The model prediction accuracy R t and flood prediction accuracy R e Superposition gives the reward function R:

[0039] R = a × R t +b×R e

[0040] Where a and b are the weight coefficients of model prediction accuracy and flood prediction accuracy respectively; model prediction accuracy R t The calculation formula is:

[0041] R t =1-MSE t

[0042]

[0043] Where MSE is the mean square error of the model prediction, n is the number of data points, and y i is the true value of the i-th data point; is the model’s predicted value for the i-th data point;

[0044] Flood prediction accuracy R e The calculation formula is:

[0045] R e =1-MSEe

[0046]

[0047] In the formula, MSE e is the mean square error of flood prediction, n e is the number of flood data points exceeding the warning level, y ei is the true value of the i-th super-warning flood data point; is the model's predicted value for the i-th super-warning flood data point.

[0048] Furthermore, step 5 optimizes the hyperparameters of the 2D-CNN-LSTM deep learning network based on the DQN algorithm with improved reward function, specifically including:

[0049] Step 51: state space definition: define the state space of the DQN algorithm, including the hyperparameters of the current 2D-CNN-LSTM network, including the learning rate, convolution kernel size, number of LSTM units, and pooling layer size;

[0050] Step 52: Action Space Definition: Define the action space of the DQN algorithm, including the adjustment of the 2D-CNN-LSTM network hyperparameters:

[0051] Increase or decrease the learning rate;

[0052] Increase or decrease the number of convolution kernels;

[0053] Increase or decrease the number of LSTM units;

[0054] Adjust the size of the pooling layer;

[0055] Step 53: Q value update: Improve the DQN algorithm through the reward function R, use the Q learning algorithm to update the Q value, and guide the action selection of DQN. The Q value update formula is as follows:

[0056] Q(S,A i )←Q(S,A i )+α(R+γmax Q(S',A i ')-Q(S,A i ))

[0057] Where S is the number of 2D-CNN-LSTM network layers, A i is the hyperparameter of the 2D-CNN-LSTM network, R is the reward function value, α is the learning rate, and γ is the discount factor;

[0058] Step 54: Obtaining the optimal hyperparameters of the 2D-CNN-LSTM network: Use the ε-greedy strategy to balance exploration and utilization, set the initial ε to 1.0, gradually decay to 0.1, select random actions with a probability of ε, and select the action with the largest current Q value with a probability of 1-ε. Through continuous trial and error and ε-greedy strategy adjustment, the optimal hyperparameters of the 2D-CNN-LSTM network with the maximum Q value are obtained.

[0059] Furthermore, step 6 specifically includes:

[0060] Step 61: Model training: Configure the optimized hyperparameters to the 2D-CNN-LSTM network, use the training data set to train the 2D-CNN-LSTM network, set the upper limit of iterations to 200, and set the early stopping mechanism;

[0061] Early stopping mechanism: When the loss function does not reach a new minimum value within several consecutive iterations, it is considered that the model may have entered the overfitting stage, triggering early stopping, thereby stopping model training to prevent overfitting;

[0062] Step 62: Model saving: Save the trained 2D-CNN-LSTM network.

[0063] Furthermore, step 7 uses a validation data set to verify the prediction performance of the trained 2D-CNN-LSTM network, and the verification method is to evaluate the prediction performance of the trained 2D-CNN-LSTM network through mean square error and determination coefficient.

[0064] Furthermore, step 8 uses the verified 2D-CNN-LSTM network to perform runoff prediction, including:

[0065] The spatial features of the 2D rainfall accumulation map, 2D temperature average map, 2D land use map, 2D soil type map and 2D geological map are extracted through the convolution layer of 2D-CNN;

[0066] The spatial features are reduced in dimension through the pooling layer of 2D-CNN, and then the one-dimensional feature vector is converted into a numerical value through the fully connected layer until all data are extracted to obtain a time series containing spatial features;

[0067] The time series output by 2D-CNN is passed through the input gate, forget gate, output gate and unit state update mechanism of the long short-term memory network LSTM to predict the runoff value.

[0068] The present invention combines reinforcement learning and deep learning technology to construct a rainfall runoff model suitable for karst areas. This method can accurately simulate the complex rainfall runoff process in karst areas and has high prediction accuracy and stability. Compared with traditional methods, the present invention combines a two-dimensional convolutional neural network (2D-CNN) with a long short-term memory network (LSTM) to extract the spatial and temporal characteristics of complex geological and hydrological conditions in the karst basin, and uses the DQN algorithm improved by the reward function to optimize the hyperparameters of the 2D-CNN-LSTM model and train and verify the model, solving the problem that the existing model is difficult to accurately simulate the complex runoff process in the karst area. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 This is a diagram showing the simulation prediction results of the runoff in the Yangji Chong karst basin in Guizhou by the 2D-CNN-LSTM network of an embodiment of the present invention;

[0070] Figure 2 This is the simulation prediction result of the traditional LSTM network for the runoff of the Yangji Chong Karst Basin in Guizhou

[0071] Figure 3 It is a flowchart of a karst basin runoff prediction method based on reinforcement learning and deep learning in an embodiment of the present invention. DETAILED DESCRIPTION

[0072] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0073] The embodiment of the present invention takes the Yangji Chong karst basin in Guizhou as an example, introduces the DQN technology into the rainfall-runoff simulation, and provides a karst basin runoff prediction method based on reinforcement learning and deep learning. The Yangji Chong karst basin in Guizhou is located in the eastern suburbs of Longli County, Guizhou Province, and its geographical location is between 106°59′58″~107°4′19″ east longitude and 26°25′4″~26°24′18″ north latitude. The overall terrain of the basin is high in the south and low in the north, with an elevation range of 1080~1670m. The terrain is undulating and belongs to the karst medium and low mountain and hilly landform. The climate type is the northern subtropical humid monsoon climate, with mild climate and abundant rainfall. The average temperature for many years is 14.8℃, and the average rainfall for many years is 1100mm. The rainfall is unevenly distributed throughout the year, mainly concentrated in April-July, accounting for more than 60% of the annual rainfall. The main vegetation types are subtropical limestone evergreen oak forest and evergreen deciduous broad-leaved mixed forest. The main soil types are lime soil, yellow loam, yellow-brown soil and paddy soil.

[0074] The total land area of ​​Guizhou Yangjichong Karst Basin is 7.68km 2 , including 5.60% cultivated land (including 2.73% sloping cultivated land and 2.87% paddy field), 0.33% orchard, 28.33% forest land (including 26.57% tree forest land and 1.75% shrub forest land), 7.70% other grassland and 1.88% land for transportation.

[0075] This embodiment uses the runoff measured at the basin outlet control station from 2019 to 2022 as the basis for model training and verification.

[0076] See also Figure 3 The embodiment of the present invention provides a karst basin runoff prediction method based on reinforcement learning and deep learning, comprising the following steps:

[0077] Step 1: Obtain a two-dimensional rainfall accumulation map, a two-dimensional temperature average map, and fixed surface and underground features of the basin every hour. The fixed surface and underground features of the basin include a two-dimensional land use map, a two-dimensional soil type map, and a two-dimensional geological map.

[0078] According to step 1 of the method of the present invention, a two-dimensional rainfall accumulation map, a two-dimensional temperature average map, and a fixed surface and underground characteristic map of the Yangji Chong karst basin in Guizhou are obtained every hour. Specifically, as follows:

[0079] Rainfall data collection (step 11): Collect hourly rainfall data measured by rain gauges in the Yangji Chong karst basin in Guizhou from 2019 to 2022, and use the Kriging interpolation method to generate a two-dimensional rainfall accumulation map of the basin every hour.

[0080] Temperature data collection (step 12): Obtain hourly temperature data for the same period from the meteorological station, calculate the average temperature per hour, and generate a two-dimensional average temperature map of the basin per hour using the Kriging interpolation method.

[0081] Land use data collection (step 13): The land use data in the watershed are obtained by interpreting the remote sensing images of the watershed using geographic information system (GIS) technology, and a two-dimensional land use map is generated, which specifically includes the classification of cultivated land, orchards, woodlands, grasslands, and land for transportation.

[0082] Soil type data collection (step 14): Through soil surveys and soil databases, soil type data in the basin are obtained to generate a two-dimensional soil type map. Soil types include lime soil, yellow loam, yellow-brown soil, and paddy soil.

[0083] Geological data collection (step 15): Using local geological survey data and geological maps, obtain geological data in the basin and generate a two-dimensional geological map.

[0084] Step 2: Build model training and validation datasets based on the data obtained in step 1

[0085] According to step 2, the data obtained in step 1 are integrated, cleaned, standardized, segmented and enhanced to construct model training and verification data sets. Specifically, it includes:

[0086] Data integration (step 21): The 2D rainfall accumulation map, 2D temperature average map, 2D land use map, 2D soil type map and 2D geological map are unified into a spatial resolution of 100 m × 100 m and aligned in the time dimension to form a multidimensional data set.

[0087] Data cleaning (step 22): Check and remove missing values ​​and outliers in the data. For example, missing rainfall data can be filled by interpolating data from neighboring stations; abnormal temperature data can be reasonably corrected or removed.

[0088] Data standardization (step 23): Standardize data of different dimensions. Perform Min-Max standardization on rainfall and temperature data, and perform One-Hot Encoding on land use, soil type, and geological maps to convert them into binary vectors.

[0089] Data segmentation (step 24): The integrated dataset is divided into a training set (80%) and a validation set (20%) in a ratio of 8:2.

[0090] Data enhancement (step 25): Data enhancement techniques are applied to add Gaussian noise to the rainfall data and smooth the temperature data to increase data diversity and improve the robustness of the model.

[0091] Step 3: Build a 2D-CNN-LSTM deep learning network for runoff prediction

[0092] Step 3 specifically includes:

[0093] Network architecture design (step 31): Design a 2D-CNN-LSTM architecture that combines a convolutional neural network (CNN) with a long short-term memory network (LSTM). The architecture first extracts spatial features through multiple layers of convolution and pooling, then captures temporal dynamics through the LSTM layer, and finally outputs the runoff prediction results through a fully connected layer.

[0094] Convolutional layer design (step 32): Set up three convolutional layers, each with a 3×3 kernel size and 32 kernels to extract spatial features of rainfall, temperature, land use, soil type, and geological maps. Each convolution layer is followed by a ReLU activation function.

[0095] Pooling layer design (step 33): A 2×2 maximum pooling layer is added after each convolutional layer to reduce the size and computation of the feature map while retaining the main features.

[0096] LSTM layer design (step 34): After the convolution and pooling layers, a two-layer LSTM is added, each layer contains 256 LSTM units to capture timing features and dynamic changes.

[0097] Fully connected layer design (step 35): After the LSTM layer, two fully connected layers are set. The first layer contains 512 neurons and uses the ReLU activation function. The second layer is the output layer, which outputs the predicted runoff.

[0098] Network parameter initialization (step 36): The He initialization method is used to initialize the weights of the convolutional layer and the fully connected layer, and the bias is initialized to zero.

[0099] Loss function design (step 37): The mean square error (MSE) is selected as the loss function to measure the difference between the predicted runoff and the actual runoff.

[0100] Optimization algorithm selection (step 38): Select the Adam optimizer, and the learning rate is initially set to 0.001 to optimize the network parameters.

[0101] Step 4: Introduce flood prediction accuracy to redesign the reward function R in the DQN algorithm, thereby improving the DQN algorithm.

[0102] The reward function design method is: the model prediction accuracy R t and flood prediction accuracy R e Superposition gives the reward function R:

[0103] R = a × R t +b×R e

[0104] In the formula, a and b are the weight coefficients of model prediction accuracy and flood prediction accuracy, respectively. Model prediction accuracy R t The calculation formula is:

[0105] R t =1-MSE t

[0106]

[0107] Where MSE is the mean square error of the model prediction, n is the number of data points, and y i is the true value of the i-th data point; is the model's prediction for the ith data point.

[0108] Flood prediction accuracy R e The calculation formula is:

[0109] R e =1-MSE e

[0110]

[0111] In the formula, MSE e is the mean square error of flood prediction, n e is the number of flood data points exceeding the warning level, y ei is the true value of the i-th super-warning flood data point; is the model's predicted value for the i-th super-warning flood data point.

[0112] Step 5: Based on the DQN algorithm with improved reward function, optimize the hyperparameters (learning rate, convolution kernel size, number of LSTM units, and pooling layer size) of the 2D-CNN-LSTM network constructed in step 3.

[0113] Step 5 specifically includes:

[0114] State space definition (step 51): Define the state space of the DQN algorithm, including the hyperparameters of the current 2D-CNN-LSTM network (learning rate, convolution kernel size, number of LSTM units, pooling layer size).

[0115] Action space definition (step 52): Define the action space of the DQN algorithm, including the adjustment of the hyperparameters of the 2D-CNN-LSTM network:

[0116] Increase or decrease the learning rate (e.g. ±0.0001)

[0117] Increase or decrease the number of convolution kernels (such as ±16)

[0118] Increase or decrease the number of LSTM units (e.g. ±64)

[0119] Adjust the pooling layer size (e.g. 2×2, 3×3)

[0120] Q value update (step 53): The DQN algorithm is improved by the reward function R, and the Q value is updated using the Q learning algorithm to guide the action selection of DQN. The Q value update formula is as follows:

[0121] Q(S,A i )←Q(S,A i )+α(R+γmax Q(S',A i ')-Q(S,A i ))

[0122] Where S is the number of 2D-CNN-LSTM network layers, A i is the hyperparameter of the 2D-CNN-LSTM network, R is the reward function value, α is the learning rate, and γ is the discount factor.

[0123] Optimal hyperparameters of 2D-CNN-LSTM network (step 54): Use the ε-greedy strategy to balance exploration and exploitation. Set the initial ε to 1.0 and gradually decay to 0.1. Select a random action with a probability of ε, and select the action with the maximum current Q value with a probability of 1-ε. Through continuous trial and error and ε-greedy strategy adjustment, the optimal hyperparameters of the 2D-CNN-LSTM network with the maximum Q value are obtained.

[0124] Step 6: Configure the optimized hyperparameters to the 2D-CNN-LSTM network, train the 2D-CNN-LSTM network using the training data set, and save the trained 2D-CNN-LSTM network.

[0125] Step 6 specifically includes:

[0126] Step 61: Model training: Configure the optimized hyperparameters for the 2D-CNN-LSTM network, train the 2D-CNN-LSTM network using the training data set, set the upper limit of iterations to 200, and set an early stopping mechanism.

[0127] Early stopping mechanism: When the loss function does not reach a new minimum value within several consecutive iterations (such as 10 consecutive iterations), it is considered that the model may have entered the overfitting stage, triggering early stopping, thereby stopping model training to prevent overfitting.

[0128] Step 62: Model saving: Save the trained 2D-CNN-LSTM network.

[0129] Step 7: Model Validation and Evaluation

[0130] The validation data set is used to validate the trained 2D-CNN-LSTM network, and the mean square error (MSE), determination coefficient (R 2 )Evaluate the prediction performance of the model after training.

[0131] Comparative analysis:

[0132] Comparison with the traditional LSTM model: The rainfall runoff simulation results of the 2D-CNN-LSTM model are compared with those of the traditional rainfall runoff model (LSTM). Figure 1 and Figure 2 The specific results are as follows:

[0133] Prediction Accuracy:

[0134] Mean square error (MSE): The MSE of the traditional rainfall-runoff model (LSTM) is 0.24, while the MSE of the optimized 2D-CNN-LSTM model is significantly reduced to 0.046, indicating that the prediction accuracy of the model is significantly improved.

[0135] Coefficient of determination (R 2 ): R of the traditional rainfall-runoff model (LSTM) 2 is 0.72, while the R 2 The value reaches 0.94, indicating that the model can simulate the changes in runoff volume well.

[0136] Flood Forecast:

[0137] Under extreme rainfall events, the optimized 2D-CNN-LSTM model can successfully capture the sudden increase trend of runoff, and the prediction error is controlled within a reasonable range, which has high application value.

[0138] Step 8: Use the verified 2D-CNN-LSTM network to predict runoff, including:

[0139] The spatial features of the 2D rainfall accumulation map, 2D temperature average map, 2D land use map, 2D soil type map and 2D geological map are extracted through the convolution layer of 2D-CNN;

[0140] The spatial features are reduced in dimension through the pooling layer of 2D-CNN, and then the one-dimensional feature vector is converted into a numerical value through the fully connected layer until all data are extracted to obtain a time series containing spatial features;

[0141] The time series output by 2D-CNN is passed through the input gate, forget gate, output gate and unit state update mechanism of the long short-term memory network LSTM to predict the runoff value.

[0142] Through the specific application in the embodiment, the effectiveness and superiority of the method for constructing a rainfall runoff model in karst areas based on reinforcement learning and deep learning are verified. The hyperparameter optimization of the 2D-CNN-LSTM network using the DQN algorithm with an improved reward function significantly improves the prediction accuracy of the model, and successfully solves the problem that traditional models are difficult to accurately simulate under complex geological and hydrological conditions. This method has broad application prospects and can be extended to rainfall runoff simulation and prediction in other karst areas and similar complex geological and geomorphic areas.

[0143] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A karst basin runoff prediction method based on reinforcement learning and deep learning, characterized in that: The following steps are involved: Step 1: Obtain a two-dimensional rainfall accumulation map, a two-dimensional temperature average map, and fixed surface and underground features of the basin every hour, wherein the fixed surface and underground features of the basin include a two-dimensional land use map, a two-dimensional soil type map, and a two-dimensional geological map; Step 2: Construct model training and validation data sets based on the data obtained in step 1. The training set is used for model parameter training, and the validation set is used to evaluate model performance. Step 3: Take the data obtained in step 1 as input, perform convolution dimension reduction and feature extraction through the two-dimensional convolutional neural network 2D-CNN, input the extracted hourly time series into the long short-term memory network LSTM, and build a 2D-CNN-LSTM deep learning network for runoff prediction; Step 4: Redesign the reward function R in the DQN algorithm by introducing flood prediction accuracy, improve the DQN algorithm, and obtain a DQN algorithm based on the improved reward function; Step 5: Optimize the hyperparameters of the 2D-CNN-LSTM network constructed in step 3 based on the DQN algorithm with improved reward function; Step 6: Configure the optimized hyperparameters to the 2D-CNN-LSTM network, train the 2D-CNN-LSTM network using the training data set, and save the trained 2D-CNN-LSTM network; Step 7: Use the validation data set to verify the prediction performance of the trained 2D-CNN-LSTM network; Step 8: Use the verified 2D-CNN-LSTM network to perform runoff prediction.

2. The karst basin runoff prediction method based on reinforcement learning and deep learning according to claim 1 is characterized in that: Step 1 specifically includes: Step 11: Rainfall data collection: Collect rainfall data from each station in the basin and generate a two-dimensional rainfall accumulation map per hour by interpolation; Step 12: Temperature data collection: Obtain temperature data in the basin from the meteorological station, calculate the average temperature per hour, and generate a two-dimensional temperature average map by interpolation; Step 13: Land use data collection: Obtain land use data in the watershed and generate a two-dimensional land use map; Step 14: Soil type data collection: Collect soil type data in the watershed and generate a two-dimensional soil type map; Step 15: Geological data collection: Obtain geological data within the basin and generate a two-dimensional geological map.

3. The karst basin runoff prediction method based on reinforcement learning and deep learning according to claim 1 is characterized in that: Step 2 specifically includes: Step 21: Data integration: The two-dimensional rainfall accumulation map, the two-dimensional temperature average map, the two-dimensional land use map, the two-dimensional soil type map and the two-dimensional geological map obtained in step 1 are unified in resolution to form a multidimensional data set; Step 22: Data cleaning: Clean the integrated data to remove missing values ​​and outliers to ensure data integrity and consistency; Step 23: Data standardization: Standardize the data and convert data of different dimensions into unified dimensions, including one-hot encoding of land use maps, soil type maps, and geological maps; Step 24: Data segmentation: divide the integrated data set into a training data set and a validation data set; Step 25: Data enhancement: Use data enhancement technology to improve the robustness of the model. The data enhancement technology includes noise addition and data smoothing.

4. The karst basin runoff prediction method based on reinforcement learning and deep learning according to claim 1, characterized in that: Step 3 specifically includes: Step 31: Design a 2D-CNN-LSTM architecture that combines a convolutional neural network (CNN) and a long short-term memory (LSTM) network to extract both spatial features and capture temporal dynamics; Step 32: Convolutional layer design: Design multiple convolutional layers to extract spatial features of the 2D rainfall accumulation map, 2D temperature average map, 2D land use map, 2D soil type map, and 2D geological map; Step 33: Pooling layer design: Add a pooling layer after the convolution layer to reduce dimensionality and computational complexity; Step 34: LSTM layer design: Add LSTM layer after convolution layer and pooling layer to capture timing features and dynamic changes; Step 35: Fully connected layer design: Add a fully connected layer after the LSTM layer to integrate the extracted features and output the rainfall runoff prediction results; Step 36: Network parameter initialization: Initialize the network parameters, including weights and biases; Step 37: Loss function design: Select a suitable loss function to measure the difference between the predicted result and the true value. The loss function includes mean square error (MSE). Step 38: Optimization algorithm selection: Select a suitable optimization algorithm for optimizing network parameters, wherein the optimization algorithm includes the Adam optimizer.

5. The karst basin runoff prediction method based on reinforcement learning and deep learning according to claim 1, characterized in that: In step 4, the reward function R in the DQN algorithm is redesigned by introducing the flood prediction accuracy, including: The model prediction accuracy R t and flood prediction accuracy R e Superposition gives the reward function R: R=a×R t +b×R e Where a and b are the weight coefficients of model prediction accuracy and flood prediction accuracy respectively; model prediction accuracy R t The calculation formula is: R t =1-MSE t Where MSE is the mean square error of the model prediction, n is the number of data points, and y i is the true value of the i-th data point; is the model’s predicted value for the i-th data point; Flood prediction accuracy R e The calculation formula is: R e =1-MSE e In the formula, MSE e is the mean square error of flood prediction, n e is the number of flood data points exceeding the warning level, y ei is the true value of the i-th super-warning flood data point; is the model's predicted value for the i-th super-warning flood data point.

6. The karst basin runoff prediction method based on reinforcement learning and deep learning according to claim 1, characterized in that: Step 5 optimizes the hyperparameters of the 2D-CNN-LSTM deep learning network based on the DQN algorithm with improved reward function, including: Step 51: state space definition: define the state space of the DQN algorithm, including the hyperparameters of the current 2D-CNN-LSTM network, including the learning rate, convolution kernel size, number of LSTM units, and pooling layer size; Step 52: Action Space Definition: Define the action space of the DQN algorithm, including the adjustment of the 2D-CNN-LSTM network hyperparameters: Increase or decrease the learning rate; Increase or decrease the number of convolution kernels; Increase or decrease the number of LSTM units; Adjust the size of the pooling layer; Step 53: Q value update: Improve the DQN algorithm through the reward function R, use the Q learning algorithm to update the Q value, and guide the action selection of DQN. The Q value update formula is as follows: Q(S,A i )←Q(S,A i )+α(R+γmaxQ(S',A i ')-Q(S,A i )) Where S is the number of 2D-CNN-LSTM network layers, A i is the hyperparameter of the 2D-CNN-LSTM network, R is the reward function value, α is the learning rate, and γ is the discount factor; Step 54: Obtaining the optimal hyperparameters of the 2D-CNN-LSTM network: Use the ε-greedy strategy to balance exploration and utilization, set the initial ε to 1.0, gradually decay to 0.1, select random actions with a probability of ε, and select the action with the largest current Q value with a probability of 1-ε. Through continuous trial and error and ε-greedy strategy adjustment, the optimal hyperparameters of the 2D-CNN-LSTM network with the maximum Q value are obtained.

7. The karst basin runoff prediction method based on reinforcement learning and deep learning according to claim 1, characterized in that: Step 6 specifically includes: Step 61: Model training: Configure the optimized hyperparameters to the 2D-CNN-LSTM network, use the training data set to train the 2D-CNN-LSTM network, set the upper limit of iterations to 200, and set the early stopping mechanism; Early stopping mechanism: When the loss function does not reach a new minimum value within several consecutive iterations, it is considered that the model may have entered the overfitting stage, triggering early stopping, thereby stopping model training to prevent overfitting; Step 62: Model saving: Save the trained 2D-CNN-LSTM network.

8. The karst basin runoff prediction method based on reinforcement learning and deep learning according to claim 1, characterized in that: Step 7 uses a validation data set to verify the prediction performance of the trained 2D-CNN-LSTM network. The verification method is to evaluate the prediction performance of the trained 2D-CNN-LSTM network through mean square error and determination coefficient.

9. The karst basin runoff prediction method based on reinforcement learning and deep learning according to claim 1, characterized in that: Step 8 uses the verified 2D-CNN-LSTM network to predict runoff, including: The spatial features of the 2D rainfall accumulation map, 2D temperature average map, 2D land use map, 2D soil type map and 2D geological map are extracted through the convolutional layer of 2D-CNN; The spatial features are reduced in dimension through the pooling layer of 2D-CNN, and then the one-dimensional feature vector is converted into a numerical value through the fully connected layer until all data are extracted to obtain a time series containing spatial features; The time series output by 2D-CNN is passed through the input gate, forget gate, output gate and unit state update mechanism of the long short-term memory network LSTM to predict the runoff value.

Images