A karst watershed runoff prediction method based on reinforcement learning and deep learning

Abstract

A karst watershed runoff prediction method based on reinforcement learning and deep learning utilizes hourly two-dimensional rainfall accumulation maps, two-dimensional temperature average maps, and fixed surface and underground features of the watershed. A two-dimensional convolutional neural network performs convolutional dimensionality reduction and feature extraction. The extracted hourly time series are then input into a long-short-term memory (LSTM) network to construct a 2D-CNN-LSTM deep learning network. The hyperparameters of the 2D-CNN-LSTM network are optimized using a DQN algorithm with an improved reward function. The optimized hyperparameters are then assigned to the 2D-CNN-LSTM network. The 2D-CNN-LSTM network is trained using a training dataset, and its prediction performance is verified using a validation dataset. The verified 2D-CNN-LSTM network is then used for runoff prediction. This method can improve the model's accuracy in predicting runoff in karst watersheds.

Description

Technical Field

[0001] The present invention relates to the field of hydrological prediction technology, and in particular to a karst basin runoff prediction method based on reinforcement learning and deep learning. Background Art

[0002] Karst regions, due to their unique geological structure and hydrological characteristics, present numerous challenges in rainfall-runoff forecasting. Karst topography features complex underground drainage systems and porous rock structures, resulting in exceptionally complex and nonlinear rainfall-runoff processes. Traditional runoff forecasting models, such as physical and statistical models, often struggle to accurately simulate hydrological processes in karst regions. This is particularly true when extreme weather events occur frequently, significantly reducing forecast accuracy.

[0003] With the rapid development of deep learning and reinforcement learning technologies, data-driven methods have shown great potential in the field of hydrological forecasting. Deep learning, particularly convolutional neural networks (CNNs) and long short-term memory networks (LSTMs), has significant advantages in processing high-dimensional spatial data and time series data. CNNs can effectively extract important features from two-dimensional spatial data, while LSTMs excel at capturing long-term dependencies in time series data. However, the performance of deep learning models is highly dependent on the selection of hyperparameters, and how to automatically and effectively optimize these hyperparameters has become a key issue.

[0004] Reinforcement learning, specifically the Deep Q-Network (DQN), provides a method for policy optimization in dynamic environments. By defining an appropriate reward function, DQN can reduce prediction errors while enhancing the model's responsiveness to extreme weather events. Incorporating DQN into hyperparameter optimization of deep learning models can automatically adjust model parameters, improving the model's prediction accuracy and robustness in complex environments.

[0005] This paper proposes a karst watershed runoff prediction method based on reinforcement learning and deep learning. It uses two-dimensional rainfall and temperature maps, as well as surface and subsurface characteristics of the watershed, to extract features using a CNN and perform time series prediction using an LSTM. By optimizing hyperparameters using a DQN (deep-query prediction network), and comprehensively considering prediction errors and responses to extreme weather events, an efficient and accurate runoff prediction model is constructed. This model not only accurately captures rainfall and runoff characteristics in karst areas but also effectively copes with complex meteorological and geological conditions, providing reliable hydrological forecasts and offering technical support for disaster prevention, mitigation, and water resource management. Summary of the Invention

[0006] In view of the unique geological and hydrological characteristics of karst areas, this paper proposes a karst basin runoff prediction method based on reinforcement learning and deep learning. It aims to solve the problem that existing models are difficult to accurately simulate the complex runoff process in karst areas. By combining reinforcement learning and deep learning techniques and using a large amount of historical data to train the model, it can achieve accurate simulation and prediction of rainfall runoff processes in karst areas.

[0007] The present invention achieves the above-mentioned purpose through the following technical solutions:

[0008] A karst watershed runoff prediction method based on reinforcement learning and deep learning includes the following steps:

[0009] Step 1: Obtain hourly 2D rainfall accumulation maps, 2D temperature average maps, and fixed surface and underground features of the watershed, including 2D land use maps, 2D soil type maps, and 2D geological maps.

[0010] Step 2: Construct model training and validation datasets based on the data obtained in step 1. The training set is used for model parameter training, and the validation set is used to evaluate model performance.

[0011] Step 3: Using the data obtained in step 1 as input, perform convolution dimensionality reduction and feature extraction through a two-dimensional convolutional neural network (2D-CNN). The extracted hourly time series is input into a long short-term memory (LSTM) network to construct a 2D-CNN-LSTM deep learning network for runoff prediction.

[0012] Step 4: By introducing flood prediction accuracy, the reward function R in the DQN algorithm is redesigned to improve the DQN algorithm and obtain a DQN algorithm based on the improved reward function.

[0013] Step 5: Optimize the hyperparameters of the 2D-CNN-LSTM network constructed in step 3 based on the DQN algorithm with the improved reward function;

[0014] Step 6: Configure the optimized hyperparameters to the 2D-CNN-LSTM network, train the 2D-CNN-LSTM network using the training dataset, and save the trained 2D-CNN-LSTM network.

[0015] Step 7: Use the validation dataset to verify the prediction performance of the trained 2D-CNN-LSTM network. Step 8: Use the validated 2D-CNN-LSTM network to perform runoff prediction.

[0016] Furthermore, step 1 specifically includes:

[0017] Step 11: Rainfall data collection: Collect rainfall data from various stations in the basin and generate hourly two-dimensional rainfall accumulation maps through interpolation;

[0018] Step 12: Temperature data collection: Obtain temperature data in the basin from the weather station, calculate the average temperature per hour, and generate a two-dimensional temperature average map through interpolation;

[0019] Step 13: Land use data collection: Obtain land use data within the watershed and generate a two-dimensional land use map;

[0020] Step 14: Soil type data collection: Collect soil type data in the watershed and generate a two-dimensional soil type map;

[0021] Step 15: Geological data collection: Obtain geological data within the basin and generate a two-dimensional geological map.

[0022] Furthermore, step 2 specifically includes:

[0023] Step 21: Data integration: The 2D rainfall accumulation map, 2D temperature average map, 2D land use map, 2D soil type map, and 2D geological map obtained in step 1 are unified in resolution to form a multidimensional dataset.

[0024] Step 22: Data cleaning: Clean the integrated data to remove missing values ​​and outliers to ensure data integrity and consistency;

[0025] Step 23: Data standardization: Standardize the data and convert data of different dimensions into a unified dimension. The land use map, soil type map, and geological map are one-hot encoded.

[0026] Step 24: Data segmentation: Divide the integrated dataset into a training dataset and a validation dataset;

[0027] Step 25: Data enhancement: Use data enhancement techniques to improve the robustness of the model. The data enhancement techniques include noise addition and data smoothing.

[0028] Furthermore, step 3 specifically includes:

[0029] Step 31: Design a 2D-CNN-LSTM architecture that combines a convolutional neural network (CNN) with a long short-term memory (LSTM) network to extract both spatial features and temporal dynamics.

[0030] Step 32: Convolutional layer design: Design multiple convolutional layers to extract spatial features of the 2D rainfall accumulation map, 2D temperature average map, 2D land use map, 2D soil type map, and 2D geological map.

[0031] Step 33: Pooling layer design: Add a pooling layer after the convolutional layer to reduce dimensionality and computational complexity;

[0032] Step 34: LSTM layer design: Add an LSTM layer after the convolutional layer and pooling layer to capture temporal features and dynamic changes;

[0033] Step 35: Fully connected layer design: Add a fully connected layer after the LSTM layer to integrate the extracted features and output the rainfall runoff prediction results;

[0034] Step 36: Network parameter initialization: Initialize the network parameters, including weights and biases;

[0035] Step 37: Loss function design: Select a suitable loss function to measure the difference between the predicted result and the true value. The loss function includes mean square error (MSE).

[0036] Step 38: Optimization algorithm selection: Select a suitable optimization algorithm for optimizing network parameters, wherein the optimization algorithm includes the Adam optimizer.

[0037] Furthermore, in step 4, the reward function R in the DQN algorithm is redesigned by introducing the flood prediction accuracy, specifically including:

[0038] The model prediction accuracy R t and flood prediction accuracy R e Superposition gives the reward function R:

[0039] R = a × R t +b×R e

[0040] Where a and b are the weight coefficients of model prediction accuracy and flood prediction accuracy respectively; model prediction accuracy R t The calculation formula is:

[0041] R t =1-MSE t

[0042]

[0043] Where, MSE t is the mean squared error of the model prediction, n is the number of data points, and y i is the true value of the i-th data point; is the model’s predicted value for the i-th data point;

[0044] Flood prediction accuracy R e The calculation formula is:

[0045] R e=1-MSE e

[0046]

[0047] Where, MSE e is the mean square error of flood prediction, n e is the number of flood data points exceeding warning level, y ei is the true value of the i-th super-warning flood data point; is the model's predicted value for the i-th super-warning flood data point.

[0048] Furthermore, step 5 optimizes the hyperparameters of the 2D-CNN-LSTM deep learning network based on the DQN algorithm with the improved reward function, specifically including:

[0049] Step 51: State space definition: Define the state space of the DQN algorithm, including the hyperparameters of the current 2D-CNN-LSTM network, such as the learning rate, convolution kernel size, number of LSTM units, and pooling layer size.

[0050] Step 52: Action Space Definition: Define the action space of the DQN algorithm, including adjusting the hyperparameters of the 2D-CNN-LSTM network:

[0051] Increase or decrease the learning rate;

[0052] Increase or decrease the number of convolution kernels;

[0053] Increase or decrease the number of LSTM units;

[0054] Adjust the size of the pooling layer;

[0055] Step 53: Q-value update: Improve the DQN algorithm through the reward function R, use the Q-learning algorithm to update the Q-value, and guide the action selection of DQN. The Q-value update formula is as follows:

[0056] Q(S,A i )←Q(S,A i )+α(R+γmax Q(S',A i ')-Q(S,A i ))

[0057] Where S is the number of 2D-CNN-LSTM network layers, A i are the hyperparameters of the 2D-CNN-LSTM network, R is the reward function value, α is the learning rate, and γ is the discount factor;

[0058] Step 54: Obtaining the optimal hyperparameters of the 2D-CNN-LSTM network: Use the ε-greedy strategy to balance exploration and exploitation. Set the initial ε to 1.0 and gradually decay it to 0.1. Select a random action with a probability of ε and the action with the maximum current Q value with a probability of 1-ε. Through continuous trial and error and ε-greedy strategy adjustment, obtain the optimal hyperparameters of the 2D-CNN-LSTM network with the maximum Q value.

[0059] Furthermore, step 6 specifically includes:

[0060] Step 61: Model training: Configure the optimized hyperparameters into the 2D-CNN-LSTM network, train the 2D-CNN-LSTM network using the training dataset, set the upper limit of iterations to 200, and set an early stopping mechanism.

[0061] Early stopping mechanism: When the loss function does not reach a new minimum value within several consecutive iterations, it is considered that the model may have entered the overfitting stage, triggering early stopping, thereby stopping model training to prevent overfitting;

[0062] Step 62: Model saving: Save the trained 2D-CNN-LSTM network.

[0063] Furthermore, step 7 uses a validation data set to verify the prediction performance of the trained 2D-CNN-LSTM network. The verification method is to evaluate the prediction performance of the trained 2D-CNN-LSTM network through mean square error and determination coefficient.

[0064] Furthermore, step 8 uses the verified 2D-CNN-LSTM network to perform runoff prediction, which specifically includes:

[0065] The spatial features of the 2D rainfall accumulation map, 2D temperature average map, 2D land use map, 2D soil type map, and 2D geological map are extracted through the convolutional layer of the 2D-CNN.

[0066] The spatial features are reduced in dimensionality through the pooling layer of 2D-CNN, and then the one-dimensional feature vector is converted into a numerical value through the fully connected layer until all the data are extracted to obtain a time series containing spatial features;

[0067] The time series output by 2D-CNN is passed through the input gate, forget gate, output gate and unit state update mechanism of the long short-term memory network LSTM to predict the runoff value.

[0068] This paper combines reinforcement learning and deep learning techniques to construct a rainfall-runoff model suitable for karst areas. This method can accurately simulate the complex rainfall-runoff process in karst areas with high prediction accuracy and stability. Compared with traditional methods, this paper combines a two-dimensional convolutional neural network (2D-CNN) with a long short-term memory network (LSTM) to extract the spatial and temporal characteristics of the complex geological and hydrological conditions in the karst basin. It uses a DQN algorithm with an improved reward function to optimize the hyperparameters of the 2D-CNN-LSTM model and train, verify and evaluate the model, thus solving the problem that existing models are difficult to accurately simulate the complex runoff process in karst areas. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 This is a diagram showing the simulation prediction results of the runoff in the Yangji Chong karst basin in Guizhou Province using the 2D-CNN-LSTM network according to an embodiment of the present invention;

[0070] Figure 2 This is the simulation prediction result of the traditional LSTM network for the runoff in the Yangji Chong Karst Basin in Guizhou

[0071] Figure 3 This is a flowchart of a karst watershed runoff prediction method based on reinforcement learning and deep learning in an embodiment of the present invention. DETAILED DESCRIPTION

[0072] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0073] The embodiment of the present invention takes the Yangji Chong karst basin in Guizhou as an example, introduces the DQN technology into the rainfall runoff simulation, and provides a karst basin runoff prediction method based on reinforcement learning and deep learning. The Yangji Chong karst basin in Guizhou is located in the eastern suburbs of Longli County, Guizhou Province, and is located between 106°59′58″~107°4′19″ east longitude and 26°25′4″~26°24′18″ north latitude. The overall terrain of the basin is high in the south and low in the north, with an elevation range of 1080~1670m. The terrain is undulating and belongs to the karst low mountain and hilly landform. The climate type is the northern subtropical humid monsoon climate, with mild climate and abundant rainfall. The average temperature over the years is 14.8℃ and the average rainfall over the years is 1100mm. The rainfall is unevenly distributed throughout the year, mainly concentrated in April-July, accounting for more than 60% of the annual rainfall. The main vegetation types are subtropical limestone evergreen oak forest and evergreen deciduous broad-leaved mixed forest, and the main soil types are lime soil, yellow loam, yellow-brown soil and paddy soil.

[0074] This example uses runoff measured at the basin outlet control station from 2019 to 2022 as the basis for model training and validation.

[0075] See also Figure 3 The embodiment of the present invention provides a karst basin runoff prediction method based on reinforcement learning and deep learning, comprising the following steps:

[0076] Step 1: Obtain hourly two-dimensional rainfall accumulation maps, two-dimensional temperature average maps, and fixed surface and underground features of the basin, including a two-dimensional land use map, a two-dimensional soil type map, and a two-dimensional geological map.

[0077] According to step 1 of the method of the present invention, a two-dimensional rainfall accumulation map, a two-dimensional temperature average map, and fixed surface and underground characteristic maps of the Yangji Chong karst basin in Guizhou Province are obtained every hour. The details are as follows:

[0078] Rainfall data collection (step 11): Hourly rainfall data measured by rain gauges in the Yangji Chong karst basin in Guizhou from 2019 to 2022 were collected, and a two-dimensional hourly rainfall accumulation map of the basin was generated using the Kriging interpolation method.

[0079] Temperature data collection (step 12): Hourly temperature data for the same period are obtained from the weather station, the hourly average temperature is calculated, and a two-dimensional hourly average temperature map of the watershed is generated using the Kriging interpolation method.

[0080] Land use data collection (step 13): Use geographic information system (GIS) technology to interpret watershed remote sensing images, obtain land use data within the watershed, and generate a two-dimensional land use map, which specifically includes the classification of cultivated land, orchards, woodlands, grasslands, and transportation land.

[0081] Soil type data collection (step 14): Through soil surveys and soil databases, soil type data in the basin are obtained to generate a two-dimensional soil type map. Soil types include lime soil, yellow loam, yellow-brown soil, and paddy soil.

[0082] Geological data collection (step 15): Using local geological survey data and geological maps, obtain geological data within the basin and generate a two-dimensional geological map.

[0083] Step 2: Build model training and validation datasets based on the data obtained in step 1

[0084] According to step 2, the data obtained in step 1 is integrated, cleaned, standardized, segmented, and enhanced to construct the model training and validation datasets. Specifically, it includes:

[0085] Data integration (step 21): The 2D rainfall accumulation map, 2D temperature average map, 2D land use map, 2D soil type map, and 2D geological map are unified into a spatial resolution of 100 m × 100 m and aligned in the time dimension to form a multidimensional dataset.

[0086] Data cleaning (step 22): Check and remove missing values ​​and outliers in the data. For example, missing rainfall data is filled by interpolating data from nearby stations; abnormal temperature data is appropriately corrected or removed.

[0087] Data standardization (step 23): Standardize data of different dimensions. Perform Min-Max normalization on rainfall and temperature data, and perform One-Hot Encoding on land use, soil type, and geological maps into binary vectors.

[0088] Data splitting (step 24): The integrated dataset is divided into a training set (80%) and a validation set (20%) in a ratio of 8:2.

[0089] Data augmentation (step 25): Data augmentation techniques are applied to add Gaussian noise to the rainfall data and smooth the temperature data to increase data diversity and improve the robustness of the model.

[0090] Step 3: Build a 2D-CNN-LSTM deep learning network for runoff prediction

[0091] Step 3 specifically includes:

[0092] Network Architecture Design (Step 31): Design a 2D-CNN-LSTM architecture that combines a convolutional neural network (CNN) with a long short-term memory network (LSTM). This architecture first extracts spatial features through multiple layers of convolution and pooling, then captures temporal dynamics through LSTM layers, and finally outputs runoff predictions through fully connected layers.

[0093] Convolutional layer design (step 32): Set up three convolutional layers, each with a 3×3 kernel size and 32 kernels, to extract spatial features of rainfall, temperature, land use, soil type, and geological maps. Each convolution layer is followed by a ReLU activation function.

[0094] Pooling layer design (step 33): A 2×2 max pooling layer is added after each convolutional layer to reduce the size of the feature map and the amount of computation while retaining the main features.

[0095] LSTM layer design (step 34): After the convolution and pooling layers, a two-layer LSTM is added, each layer containing 256 LSTM units to capture timing features and dynamic changes.

[0096] Fully connected layer design (step 35): After the LSTM layer, two fully connected layers are set. The first layer contains 512 neurons and uses the ReLU activation function; the second layer is the output layer, which outputs the predicted runoff.

[0097] Network parameter initialization (step 36): The weights of the convolutional layer and the fully connected layer are initialized using the He initialization method, and the bias is initialized to zero.

[0098] Loss function design (step 37): The mean square error (MSE) is selected as the loss function to measure the difference between the predicted runoff volume and the actual runoff volume.

[0099] Optimization algorithm selection (step 38): Adam optimizer is selected, and the learning rate is initially set to 0.001 to optimize the network parameters.

[0100] Step 4: Introduce flood prediction accuracy to redesign the reward function R in the DQN algorithm, thereby improving the DQN algorithm.

[0101] The reward function design method is: the model prediction accuracy R t and flood prediction accuracy R e Superposition gives the reward function R:

[0102] R = a × R t +b×R e

[0103] Where a and b are the weight coefficients of model prediction accuracy and flood prediction accuracy, respectively. Model prediction accuracy Rt The calculation formula is:

[0104] R t =1-MSE t

[0105]

[0106] Where, MSE t is the mean squared error of the model prediction, n is the number of data points, and y i is the true value of the i-th data point; is the model's predicted value for the i-th data point.

[0107] Flood prediction accuracy R e The calculation formula is:

[0108] R e =1-MSE e

[0109]

[0110] Where, MSE e is the mean square error of flood prediction, n e is the number of flood data points exceeding warning level, y ei is the true value of the i-th super-warning flood data point; is the model's predicted value for the i-th super-warning flood data point.

[0111] Step 5: Based on the DQN algorithm with the improved reward function, optimize the hyperparameters (learning rate, convolution kernel size, number of LSTM units, and pooling layer size) of the 2D-CNN-LSTM network constructed in Step 3.

[0112] Step 5 specifically includes:

[0113] State space definition (step 51): Define the state space of the DQN algorithm, including the hyperparameters of the current 2D-CNN-LSTM network (learning rate, convolution kernel size, number of LSTM units, pooling layer size).

[0114] Action space definition (step 52): Define the action space of the DQN algorithm, including adjusting the hyperparameters of the 2D-CNN-LSTM network:

[0115] Increase or decrease the learning rate (e.g. ±0.0001)

[0116] Increase or decrease the number of convolution kernels (e.g. ±16)

[0117] Increase or decrease the number of LSTM units (e.g. ±64)

[0118] Adjust the pooling layer size (e.g. 2×2, 3×3)

[0119] Q-value update (step 53): The DQN algorithm is improved by the reward function R, and the Q-value is updated using the Q-learning algorithm to guide the DQN's action selection. The Q-value update formula is as follows:

[0120] Q(S,A i )←Q(S,A i )+α(R+γmax Q(S',A i ')-Q(S,A i ))

[0121] Where S is the number of 2D-CNN-LSTM network layers, A i are the hyperparameters of the 2D-CNN-LSTM network, R is the reward function value, α is the learning rate, and γ is the discount factor.

[0122] Obtaining the optimal hyperparameters for the 2D-CNN-LSTM network (step 54): An ε-greedy strategy is used to balance exploration and exploitation. ε is initially set to 1.0 and gradually decayed to 0.1. A random action is selected with a probability of ε, and the action with the highest Q value is selected with a probability of 1-ε. Through trial and error and adjustments made with the ε-greedy strategy, the optimal hyperparameters for the 2D-CNN-LSTM network are obtained for maximum Q value.

[0123] Step 6: Configure the optimized hyperparameters to the 2D-CNN-LSTM network, train the 2D-CNN-LSTM network using the training dataset, and save the trained 2D-CNN-LSTM network.

[0124] Step 6 specifically includes:

[0125] Step 61: Model training: Configure the optimized hyperparameters for the 2D-CNN-LSTM network, train the 2D-CNN-LSTM network using the training dataset, set the upper limit of iterations to 200, and set an early stopping mechanism.

[0126] Early stopping mechanism: When the loss function does not reach a new minimum value within several consecutive iterations (such as 10 consecutive iterations), it is considered that the model may have entered the overfitting stage, triggering early stopping, thereby stopping model training to prevent overfitting.

[0127] Step 62: Model saving: Save the trained 2D-CNN-LSTM network.

[0128] Step 7: Model Validation and Evaluation

[0129] The validation data set is used to validate the trained 2D-CNN-LSTM network, and the mean square error (MSE), coefficient of determination (R 2 )Evaluate the prediction performance of the model after training.

[0130] Comparative analysis:

[0131] Comparison with the traditional LSTM model: The rainfall runoff simulation results of the 2D-CNN-LSTM model are compared with those of the traditional rainfall runoff model (LSTM). Figure 1 and Figure 2 The specific results are as follows:

[0132] Prediction accuracy:

[0133] Mean Square Error (MSE): The MSE of the traditional rainfall-runoff model (LSTM) is 0.24, while the MSE of the optimized 2D-CNN-LSTM model is significantly reduced to 0.046, indicating that the model prediction accuracy is significantly improved.

[0134] Coefficient of determination (R 2 ): Traditional rainfall runoff model (LSTM) R 2 is 0.72, while the R of the optimized 2D-CNN-LSTM model is 2 The value reaches 0.94, indicating that the model can simulate the changes in runoff volume well.

[0135] Flood Forecast:

[0136] Under extreme rainfall events, the optimized 2D-CNN-LSTM model can successfully capture the sudden increase trend of runoff, and the prediction error is controlled within a reasonable range, which has high application value.

[0137] Step 8: Use the verified 2D-CNN-LSTM network to perform runoff prediction, including:

[0138] The spatial features of the 2D rainfall accumulation map, 2D temperature average map, 2D land use map, 2D soil type map, and 2D geological map are extracted through the convolutional layer of the 2D-CNN.

[0139] The spatial features are reduced in dimensionality through the pooling layer of 2D-CNN, and then the one-dimensional feature vector is converted into a numerical value through the fully connected layer until all the data are extracted to obtain a time series containing spatial features;

[0140] The time series output by 2D-CNN is passed through the input gate, forget gate, output gate and unit state update mechanism of the long short-term memory network LSTM to predict the runoff value.

[0141] The specific applications described in the examples demonstrate the effectiveness and superiority of the proposed method for constructing a rainfall-runoff model for karst regions based on reinforcement learning and deep learning. Using the DQN algorithm with an improved reward function to optimize the hyperparameters of a 2D-CNN-LSTM network significantly improves the model's prediction accuracy, successfully addressing the difficulty of traditional models in accurately simulating complex geological and hydrological conditions. This method has broad application prospects and can be extended to rainfall-runoff simulation and prediction in other karst regions and similarly complex geological and geomorphological areas.

[0142] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered by the scope of protection of the claims of the present invention.

Claims

1. A karst basin runoff prediction method based on reinforcement learning and deep learning, characterized in that: The following steps are involved: Step 1: Obtain hourly 2D rainfall accumulation maps, 2D temperature average maps, and fixed surface and underground features of the watershed, including 2D land use maps, 2D soil type maps, and 2D geological maps. Step 2: Construct model training and validation datasets based on the data obtained in step 1. The training set is used for model parameter training, and the validation set is used to evaluate model performance. Step 3: Using the data obtained in step 1 as input, perform convolution dimensionality reduction and feature extraction through a two-dimensional convolutional neural network (2D-CNN). The extracted hourly time series is input into a long short-term memory (LSTM) network to construct a 2D-CNN-LSTM deep learning network for runoff prediction. Step 4: By introducing flood prediction accuracy, the reward function R in the DQN algorithm is redesigned to improve the DQN algorithm and obtain a DQN algorithm based on the improved reward function. Step 5: Optimize the hyperparameters of the 2D-CNN-LSTM network constructed in step 3 based on the DQN algorithm with the improved reward function; Step 6: Configure the optimized hyperparameters to the 2D-CNN-LSTM network, train the 2D-CNN-LSTM network using the training dataset, and save the trained 2D-CNN-LSTM network. Step 7: Use the validation dataset to verify the prediction performance of the trained 2D-CNN-LSTM network. Step 8: Use the verified 2D-CNN-LSTM network to predict runoff; In step 4, the reward function R in the DQN algorithm is redesigned by introducing flood prediction accuracy, including: The model prediction accuracy R t and flood prediction accuracy R e Superposition gives the reward function R: R=a×R t +b×R e ; Where a and b are the weight coefficients of model prediction accuracy and flood prediction accuracy respectively; model prediction accuracy R t The calculation formula is: R t =1-MSE t ; ； Where, is the mean squared error of the model prediction, n is the number of data points, and y i is the true value of the i-th data point; is the model’s predicted value for the i-th data point; Flood prediction accuracy R e The calculation formula is: R e =1-MSE e ; ； Where, MSE e is the mean square error of flood prediction, n e is the number of flood data points exceeding warning level, y ei is the true value of the i-th super-warning flood data point; is the model's predicted value for the i-th super-warning flood data point.

2. The karst basin runoff prediction method based on reinforcement learning and deep learning according to claim 1 is characterized in that: Step 1 specifically includes: Step 11: Rainfall data collection: Collect rainfall data from various stations in the basin and generate hourly two-dimensional rainfall accumulation maps through interpolation; Step 12: Temperature data collection: Obtain temperature data in the basin from the weather station, calculate the average temperature per hour, and generate a two-dimensional temperature average map through interpolation; Step 13: Land use data collection: Obtain land use data within the watershed and generate a two-dimensional land use map; Step 14: Soil type data collection: Collect soil type data in the watershed and generate a two-dimensional soil type map; Step 15: Geological data collection: Obtain geological data within the basin and generate a two-dimensional geological map.

3. The karst basin runoff prediction method based on reinforcement learning and deep learning according to claim 1, characterized in that: Step 2 specifically includes: Step 21: Data integration: The 2D rainfall accumulation map, 2D temperature average map, 2D land use map, 2D soil type map, and 2D geological map obtained in step 1 are unified in resolution to form a multidimensional dataset. Step 22: Data cleaning: Clean the integrated data to remove missing values ​​and outliers to ensure data integrity and consistency; Step 23: Data standardization: Standardize the data and convert data of different dimensions into a unified dimension. The land use map, soil type map, and geological map are one-hot encoded. Step 24: Data segmentation: Divide the integrated dataset into a training dataset and a validation dataset; Step 25: Data enhancement: Use data enhancement techniques to improve the robustness of the model. The data enhancement techniques include noise addition and data smoothing.

4. The karst basin runoff prediction method based on reinforcement learning and deep learning according to claim 1, characterized in that: Step 3 specifically includes: Step 31: Design a 2D-CNN-LSTM architecture that combines a convolutional neural network (CNN) with a long short-term memory (LSTM) network to extract both spatial features and temporal dynamics. Step 32: Convolutional layer design: Design multiple convolutional layers to extract spatial features of the 2D rainfall accumulation map, 2D temperature average map, 2D land use map, 2D soil type map, and 2D geological map. Step 33: Pooling layer design: Add a pooling layer after the convolutional layer to reduce dimensionality and computational complexity; Step 34: LSTM layer design: Add an LSTM layer after the convolutional layer and pooling layer to capture temporal features and dynamic changes; Step 35: Fully connected layer design: Add a fully connected layer after the LSTM layer to integrate the extracted features and output the rainfall runoff prediction results; Step 36: Network parameter initialization: Initialize the network parameters, including weights and biases; Step 37: Loss function design: Select a suitable loss function to measure the difference between the predicted result and the true value. The loss function includes mean square error (MSE). Step 38: Optimization algorithm selection: Select a suitable optimization algorithm for optimizing network parameters, wherein the optimization algorithm includes the Adam optimizer.

5. The karst basin runoff prediction method based on reinforcement learning and deep learning according to claim 1, characterized in that: Step 5 optimizes the hyperparameters of the 2D-CNN-LSTM deep learning network based on the DQN algorithm with the improved reward function, specifically including: Step 51: State space definition: Define the state space of the DQN algorithm, including the hyperparameters of the current 2D-CNN-LSTM network, such as the learning rate, convolution kernel size, number of LSTM units, and pooling layer size. Step 52: Action Space Definition: Define the action space of the DQN algorithm, including adjusting the hyperparameters of the 2D-CNN-LSTM network: Increase or decrease the learning rate; Increase or decrease the number of convolution kernels; Increase or decrease the number of LSTM units; Adjust the size of the pooling layer; Step 53: Q-value update: Improve the DQN algorithm through the reward function R, use the Q-learning algorithm to update the Q-value, and guide the action selection of DQN. The Q-value update formula is as follows: ； Where S is the number of 2D-CNN-LSTM network layers, A i are the hyperparameters of the 2D-CNN-LSTM network, R is the reward function value, α is the learning rate, and γ is the discount factor; Step 54: Obtaining the optimal hyperparameters of the 2D-CNN-LSTM network: Use the ε-greedy strategy to balance exploration and exploitation. Set the initial ε to 1.0 and gradually decay it to 0.

1. Select a random action with a probability of ε and the action with the maximum current Q value with a probability of 1-ε. Through continuous trial and error and ε-greedy strategy adjustment, obtain the optimal hyperparameters of the 2D-CNN-LSTM network with the maximum Q value.

6. The karst basin runoff prediction method based on reinforcement learning and deep learning according to claim 1, characterized in that: Step 6 specifically includes: Step 61: Model training: Configure the optimized hyperparameters into the 2D-CNN-LSTM network, train the 2D-CNN-LSTM network using the training dataset, set the upper limit of iterations to 200, and set an early stopping mechanism. Early stopping mechanism: When the loss function does not reach a new minimum value within several consecutive iterations, it is considered that the model may have entered the overfitting stage, triggering early stopping, thereby stopping model training to prevent overfitting; Step 62: Model saving: Save the trained 2D-CNN-LSTM network.

7. The karst basin runoff prediction method based on reinforcement learning and deep learning according to claim 1, characterized in that: Step 7 uses the validation data set to verify the prediction performance of the trained 2D-CNN-LSTM network. The validation method is to evaluate the prediction performance of the trained 2D-CNN-LSTM network through mean square error and determination coefficient.

8. The karst basin runoff prediction method based on reinforcement learning and deep learning according to claim 1, characterized in that: Step 8 uses the verified 2D-CNN-LSTM network to perform runoff prediction, including: The spatial features of the 2D rainfall accumulation map, 2D temperature average map, 2D land use map, 2D soil type map, and 2D geological map are extracted through the convolutional layer of the 2D-CNN. The spatial features are reduced in dimension through the pooling layer of 2D-CNN, and then the one-dimensional feature vector is converted into a numerical value through the fully connected layer until all the data are extracted to obtain a time series containing spatial features; The time series output by 2D-CNN is passed through the input gate, forget gate, output gate and unit state update mechanism of the long short-term memory network LSTM to predict the runoff value.

Images