River water quality space-time adaptive prediction method based on feature enhancement

Abstract

The invention discloses a river water quality space-time adaptive prediction method based on feature enhancement, and relates to the field of environmental engineering. The invention aims to solve the problems that key features of river water quality are difficult to screen and the water quality prediction is inaccurate. The method comprises the following steps: collecting river water quality characteristic data of a target area for cleaning and complementing, and constructing a data set; the method comprises the following steps: carrying out rough screening on features by adopting a Pearson's correlation coefficient method, then carrying out two training settings on a random forest model, selecting IncMSE and IncNodePurity methods to obtain importance scores of river water quality features, carrying out normalization and reordering on the two importance scores, and finding key features of the river water quality features; and an optimal training mode and a corresponding model thereof are found, and prediction of water quality indexes of the whole river network is realized.

Description

Technical Field

[0001] The invention belongs to the field of environmental engineering, and in particular relates to a technology for predicting river water quality. Background Art

[0002] River water quality is an important indicator reflecting the health of river ecosystems and an important resource for human life and economic development. Monitoring and evaluation of river water quality is the basis and premise of water environment management, and an important means of water resource protection and utilization. In order to effectively monitor and manage river water quality, it is necessary to collect water quality indicators such as dissolved oxygen DO, chemical oxygen demand COD, total nitrogen TN, total phosphorus TP, etc. at different locations and specific times in the river network to reflect the material cycle and ecological function of the river. However, due to the complexity of the river network and cost constraints, traditional water quality sampling methods often find it difficult to cover the entire range of the river network, resulting in insufficient spatial resolution and accuracy of water quality data, and unable to fully reflect the spatiotemporal distribution and change patterns of river water quality.

[0003] River water quality is not only affected by the natural environment, but also by human activities. It is the result of the combined effects of multiple factors. Climate factors such as rainfall and temperature can indirectly affect river water quality by affecting river water levels and flow rates. Human activities such as urbanization and industrial emissions affect river water quality by affecting the migration and transformation of pollutants. Therefore, since the factors affecting river water quality are complex, identifying and screening the key features that affect river water quality indicators and accurately predicting the water quality can help policymakers and environmental managers formulate and implement management measures more specifically to improve river water quality.

[0004] The spatiotemporal characteristics of river water quality are crucial for effective monitoring and management. Water quality data not only show significant heterogeneity in space, but also present complex dynamic changes in time. However, existing water quality monitoring technologies often lack the ability to adapt to such spatiotemporal characteristics, making it difficult to accurately predict and respond to water quality changes in practical applications. Spatiotemporal adaptive technology provides a new approach to solving this problem by dynamically adjusting spatiotemporal characteristics to adapt to the spatiotemporal changes of water quality data.

[0005] The extrapolation and inversion of river water quality refers to the process of predicting and inferring the water quality conditions in unmonitored areas or unobserved time points based on existing water quality monitoring data and related information, with the help of mathematical models or statistical methods. Existing models and technologies for water quality extrapolation and inversion often require a large number of water quality characteristics for learning and training. The combined effects of human activities and natural factors also make water quality show complex changes in time and space. Existing models cannot use limited data to accurately predict water quality. Therefore, it is urgent to propose a method that can use the limited water quality characteristics of river sections to efficiently infer water quality indicators in areas with missing data and unmonitored areas from both the time and space dimensions, so as to improve the accuracy of water quality prediction. Summary of the invention

[0006] The present invention aims to solve the problem that it is difficult to screen the key characteristics of river water quality due to the complexity of the influencing factors of river water quality, and the existing models cannot accurately predict water quality using limited data. A spatiotemporal adaptive prediction method for river water quality based on feature enhancement is now provided.

[0007] A spatiotemporal adaptive prediction method for river water quality based on feature enhancement, comprising:

[0008] The water quality characteristic data of any river are divided using time and space as step units respectively to obtain a temporal water quality characteristic data set and a spatial water quality characteristic data set;

[0009] The temporal water quality characteristic data set and the spatial water quality characteristic data set are used to construct and train a temporal random forest model set in a temporal training mode and a spatial random forest model set in a spatial training mode, respectively. The inputs of the temporal random forest model and the spatial random forest model are both water quality characteristic data, and the outputs are both water quality indicators, and the water quality indicators are response variables of water quality;

[0010] Based on the two random forest model sets, the features in the initial water quality feature set are preliminarily screened to obtain a time key feature subset in the time training mode and a space key feature subset in the space training mode. The two random forest model sets perform the same preliminarily screening method, and the preliminarily screening includes:

[0011] The importance score of each feature in the initial water quality feature set under the current random forest model set is calculated respectively, and the features in the initial water quality feature set are reordered according to the importance score to obtain a feature sorting set; the feature with the lowest importance score in the feature sorting set is removed to obtain a feature sorting subset, and then the feature sorting subset is used as a new feature sorting set and the feature with the lowest importance score is removed to obtain a new feature sorting subset, and so on to obtain multiple feature sorting subsets; each random forest model in the current random forest model set is trained respectively using multiple feature sorting subsets, and the root mean square error of the current random forest model set after training with each feature sorting subset is calculated, and the feature sorting subset corresponding to the minimum root mean square error is used as the key feature subset in the current training mode;

[0012] The features are finely screened according to the number of features in the key feature subsets under the two training modes and the corresponding root mean square errors, and the fine screening includes:

[0013] Let S g_P and S h_K are the key feature subsets under the two training modes, P and K represent the two training modes, g and h represent S g_P and Sh_K The number of features in , RMSE g_P and RMSE h_K S g_P and S h_K The root mean square error of the random forest model set after training is:

[0014] When g≤h and RMSE g_P <RMSE h_K When S g_P As the optimal set of river water quality characteristics;

[0015] When g ≥ h and RMSE g_P >RMSE h_K When S h_K As the optimal set of river water quality characteristics;

[0016] When g≤h and RMSE g_P >RMSE h_K When , calculate the g-th feature sorting subset S under K training mode g_K The corresponding root mean square error RMSE g_K , and then judge the RMSE g_K and RMSE g_P The size of RMSE g_K >RMSE g_P , then S g_P As the optimal river water quality feature set, if RMSE g_K <RMSE g_P , then S h_K As the optimal set of river water quality characteristics;

[0017] When g ≥ h and RMSE g_P <RMSE h_K When P is used to calculate the hth feature sorting subset S in the P training mode h_P The corresponding root mean square error RMSE h_P , and then judge the RMSE h_P and RMSE h_K The size of RMSE h_P <RMSE h_K , then S g_P As the optimal river water quality feature set, if RMSE h_P >RMSE h_K , then S h_K As the optimal set of river water quality characteristics;

[0018] The training mode corresponding to the optimal river water quality feature set is used as the optimal training mode, and the optimal random forest model set is trained using the optimal river water quality feature set under the optimal training mode;

[0019] The optimal random forest model set is used to predict the river water quality indicators in the target area and obtain the prediction result set;

[0020] The average value of all prediction results in the prediction result set is taken as the final prediction result of river water quality in the target area.

[0021] Furthermore, the method for obtaining the above-mentioned initial water quality feature set includes:

[0022] Collect digital elevation model data of the target area to obtain the high and low points of the target area, use the water flow direction algorithm to determine the water flow direction and water flow accumulation of the target area in combination with the high and low points of the target area, extract the river network of the target area according to the water flow direction and water flow accumulation, and use the watershed division algorithm to determine the watershed boundary of the river network according to the water flow direction and the terrain characteristics of the target area to form a watershed area contour map;

[0023] Taking each area in the watershed area outline map as a unit, collect water quality characteristic data of the target area, eliminate invalid and erroneous water quality characteristic data, interpolate or complete the missing water quality characteristic data, smooth or correct the abnormal water quality characteristic data, and obtain a rough water quality characteristic data set;

[0024] The Pearson correlation coefficient matrix R is constructed using the water quality characteristics and water quality response variables in the crude water quality characteristic data set. The elements R ij represents the Pearson correlation coefficient between the i-th water quality characteristic and the j-th response variable, 1≤i≤M, 1≤j≤M′, M is the number of water quality characteristics in the crude water quality characteristic dataset, and M′ is the number of response variables;

[0025] The features corresponding to the elements in R that are less than a preset coefficient threshold are removed, and the remaining water quality features in the rough water quality feature data set constitute the initial water quality feature set.

[0026] Furthermore, in the time training mode, the water quality characteristic data of any river are divided into A time step units to obtain A time water quality characteristic data sets; A-1 time water quality characteristic data sets are arbitrarily selected for training to obtain A time random forest models, and the A time random forest models constitute the time random forest model set in the time training mode.

[0027] Furthermore, the temporal random forest model set is used to preliminarily screen the features in the initial water quality feature set to obtain a subset of temporal key features in the temporal training mode, including:

[0028] Calculate the average precision reduction value and node impurity reduction value of each feature in the initial water quality feature set under the random forest model at each time;

[0029] The importance score of each water quality feature under the temporal random forest model set was calculated using the normalization method;

[0030] Reorder the features in the initial water quality feature set according to their importance scores to obtain a feature ranking set S N_P = {I 1_P ,I 2_P ,I 3_P ,...,I N_P}, where I n_P is the feature with the highest importance score in the time training mode, then the feature ranking subset is represented as S N-1_P ,S N-2_P ,...,S 1_P ;

[0031] Use S N-1_P ,S N-2_P ,...,S 1_P The models in the temporal random forest training model set are trained respectively, and the root mean square error of the temporal random forest model set after training with each feature sorting subset is calculated;

[0032] The feature sorting subset corresponding to the minimum RMS error is used as the temporal key feature subset in the temporal training mode.

[0033] Furthermore, under the spatial training mode, the water quality characteristic data of any river are divided into B monitoring locations as spatial units to obtain B spatial water quality characteristic data sets; B-1 spatial water quality characteristic data sets are arbitrarily selected for training to obtain B spatial random forest models, and the B spatial random forest models constitute the spatial random forest model set under the spatial training mode.

[0034] Furthermore, the spatial random forest model set was used to preliminarily screen the features in the initial water quality feature set to obtain a subset of spatial key features under the spatial training mode, including:

[0035] Calculate the average precision reduction value and node impurity reduction value of each feature in the initial water quality feature set under each spatial random forest model respectively;

[0036] The importance score of each water quality feature under the spatial random forest model set was calculated using the normalization method;

[0037] Reorder the features in the initial water quality feature set according to their importance scores to obtain a feature ranking set S N_K = {I 1_K ,I 2_K ,I 3_K ,...,I N_K}, where In_ Kis the feature with the highest importance score in the spatial training mode, then the feature ranking subset is represented as S N-1_K ,S N-2_K ,...,S 1_K ;

[0038] Use S N-1_K ,S N-2_K ,...,S 1_K The models in the spatial random forest training model set are trained respectively, and the root mean square error of the spatial random forest model set after training with each feature sorting subset is calculated;

[0039] The feature sorting subset corresponding to the minimum root mean square error is used as the spatial key feature subset in the spatial training mode.

[0040] Furthermore, the normalization method is used to calculate the importance score of each water quality feature under the current random forest model set. The expression is as follows:

[0041]

[0042] Where: 1<n<N, N is the number of features in the initial water quality feature set, 1<q<Q, Q is the number of models included in the current random forest model set, FFIS n is the importance score of the nth feature under the current random forest model set, IM q is the average reduction in accuracy of the qth random forest model, IM Q_min and IM Q_max are the minimum and maximum average reduction values ​​of the precision in the Q random forest training models, IN q is the node impurity reduction value of the qth random forest model, IN Q_min and IN Q_max are the minimum and maximum values ​​of node impurity reduction in Q random forest training models, respectively.

[0043] Furthermore, the root mean square error of the current random forest model set after training with each feature sorting subset is calculated, and the expression is as follows:

[0044]

[0045] Among them, RMSE f is the root mean square error of the current spatial random forest model set trained by the f-th feature sorting subset, 1≤f≤N-1, y is the actual response variable value, is the value of the response variable predicted by the qth random forest model.

[0046] Furthermore, when the best training mode is the time training mode, the final prediction result of the river water quality is:

[0047]

[0048] When the best training mode is the spatial training mode, the final prediction result of river water quality is:

[0049]

[0050] T 1_P ,T 2_P ,...,T A_P represents the prediction result obtained by the best training mode, T P is the final prediction result of river water quality obtained by using the time training model; T 1_P ,T 2_P ,...,T A_P represents the prediction result obtained by the best training mode, T K The final prediction results of river water quality obtained using the spatial training model.

[0051] Furthermore, after obtaining the key feature subset, the key feature subset is tested, and fine screening is performed after passing the test.

[0052] The method for spatiotemporal adaptive prediction of river water quality based on feature enhancement described in the present invention has the following five beneficial effects:

[0053] 1. Generalized model training based on time and space dimensions: Two training modes are proposed, namely time training and space training. Through these two training modes, the spatial and temporal complexity of the river network can be better captured, the generalization ability of the model can be improved, the model can be better adapted to new data, and the limited data can be used to maximize the accurate prediction of missing data or unobserved locations.

[0054] 2. Spatiotemporal adaptive feature iteration screening: Establish a spatiotemporal feature adaptive iteration mechanism to dynamically adjust the feature set according to the collected spatiotemporal data. Eliminate the least important features one by one, calculate the RMSE of the remaining feature sets, and dynamically adjust the feature set through feedback to obtain the best feature subset, which can improve the prediction accuracy of the model while reducing the complexity of the model.

[0055] 3. Optimization of models with fewer features and higher precision: By comprehensively considering the number of features and prediction accuracy of the two training modes, the most appropriate model is selected according to the principle of fewer features and higher precision to perform extrapolation and inversion of water quality indicators.

[0056] 4. Comprehensive evaluation of the importance of multiple features: The final importance score of the feature is obtained by comprehensively considering the node impurity reduction value and the average precision reduction value of each feature, avoiding the limitations of using only a single indicator and more accurately reflecting the importance of each water quality indicator to the predicted water quality indicator.

[0057] 5. Limited features enable accurate prediction of watersheds with missing data: By adaptively screening key features and combining them with optimized prediction models, water quality indicators for the entire river network can be effectively predicted. Even if water quality data is missing for some time periods or locations, the water quality of these areas can still be accurately estimated with limited data. This provides strong data support for the comprehensive management and governance of the watershed. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 It is a flow chart of a spatiotemporal adaptive prediction method of river water quality based on feature enhancement;

[0059] Figure 2 This is the root mean square error curve of the test set under different feature combinations in the time training mode;

[0060] Figure 3 This is the root mean square error curve of the test set under different feature combinations in the spatial training mode;

[0061] Figure 4 Root mean square error curves of different feature combinations after retraining for the selected feature subset;

[0062] Figure 5 The root mean square error curve of different feature combinations after re-training the space for the selected feature subset;

[0063] Figure 6 This is a comparison chart between the random forest predicted value and the actual value of a certain site in the time training mode. DETAILED DESCRIPTION

[0064] The following will be combined with the accompanying drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work belong to the scope of protection of the present invention. It should be noted that the embodiments of the present invention and the features in the embodiments can be combined with each other without conflict.

[0065] Specific implementation method 1: refer to Figure 1Specifically explain this embodiment, a spatiotemporal adaptive prediction method for river water quality based on feature enhancement described in this embodiment includes: collecting data on river water quality characteristics in a study area, cleaning and completing the collected data, constructing a data set, using the Pearson correlation coefficient method to roughly screen the features, and then using random forests for two training settings, selecting the mean reduction in precision (increase in mean squared error, IncMSE) and the increase in node impurity (increase in node purity, IncNodePurity) to obtain the importance score of the river water quality characteristics, and using the normalization method to rank the importance of key features affecting river water quality indicators based on the importance scores obtained by the two methods to find the key features; re-constructing the model based on the screened out key features to check whether it is overfitting and the contribution of the selected features to the prediction ability, and at the same time, based on the principle of using fewer features to achieve higher prediction accuracy, selecting the most suitable prediction model for the situation, and using limited feature data to deduce to the entire river network to achieve the prediction of water quality indicators for the entire river network.

[0066] The details are as follows:

[0067] S1: Use DEM to divide the target watershed

[0068] Collect the digital elevation model (DEM) data of the target area, and identify the high and low points of the target area terrain by analyzing the DEM data. Based on the high and low points of the target area terrain, apply the water flow direction algorithm to determine the water flow direction of the target area and calculate the accumulated water flow. Use the water flow direction and accumulated water flow in the target area to identify the water flow path and extract the river network. On this basis, according to the water flow direction and the terrain characteristics of the target area, use the watershed division algorithm to determine the watershed boundary, analyze the properties of the watershed, and form a contour map of the catchment area. Finally, verify and correct the extracted river network and watershed by comparing with existing maps or field survey data to ensure their accuracy.

[0069] S2: Data cleaning of the watershed and construction of the dataset

[0070] Comprehensively consider various factors affecting river water quality and construct a river water quality prediction index system. Specifically, based on various statistical yearbooks, environmental bulletins and public data platforms, water quality characteristic data of the target basin are collected in units of the catchment area divided by S1. Quality control and outlier detection are performed on the collected water quality characteristic data, specifically: invalid and erroneous data are eliminated, missing data are interpolated or completed, and abnormal data are smoothed or corrected to make the data more realistic and continuous. A water quality characteristic data set suitable for subsequent analysis is formed, making the data set representative and comparable.

[0071] S3: Preliminary screening of water quality characteristics based on Pearson correlation coefficient method

[0072] The Pearson correlation coefficient is used to detect the degree of linear correlation between two continuous variables. The value range is [-1,1]. Positive values ​​indicate positive correlation, negative values ​​indicate negative correlation, and the larger the absolute value, the higher the degree of linear correlation. Using M water quality characteristics and M′ response variables to construct an M×M′ correlation coefficient matrix R, then R ij represents the Pearson correlation coefficient between the i-th water quality characteristic and the j-th response variable, 1≤i≤M, 1≤j≤M′. ij |>τ (τ is the threshold), the i-th feature is considered to be significantly correlated with the j-th response variable. Excluding water quality features that are not significantly correlated with the response variable, the number of remaining water quality features is N, N<M, which constitutes the initial water quality feature set.

[0073] S4: Feature importance evaluation based on time training and iterative feature screening

[0074] The model constructed by random forest is trained in time (P training), and the specific steps are as follows:

[0075] Using the water quality characteristic data of a river with A time steps (here it can be months, years, etc., if the time step is data of days or hours, randomly select 2 / 3 of the water quality characteristic data for training, and the remaining 1 / 3 of the water quality characteristic data for testing), A time water quality characteristic data sets are obtained. Any one of the time water quality characteristic data sets is used as a test set to test the prediction ability of the random forest, and the remaining A-1 time water quality characteristic data sets are used to construct and train the time random forest model. A total of A time random forest models are trained to form a time random forest model set. The input of the time random forest model is water quality characteristics, and the output is water quality indicators, that is, response variables.

[0076] The average reduction in precision and the reduction in node impurity of the N features in the initial water quality feature set under A time random forest models are calculated respectively.

[0077] According to the following formula, the importance score (Final Feature Importance Score, FFIS for short) of each water quality feature under the temporal random forest model set is obtained by normalization method:

[0078]

[0079] Where: 1<n<N, N is the number of features in the initial water quality feature set, 1<a<A, FFIS n_P is the importance score of the nth feature under the temporal random forest model set, IM a is the average reduction in accuracy of the random forest model at the ath time, IM A_min and IM A_max are the minimum and maximum average reduction values ​​of the precision in the A-time random forest training model, IN a is the node impurity reduction value of the random forest model at the ath time, IN A_min and IN A_max are the minimum and maximum values ​​of node impurity reduction in A time random forest training models, respectively.

[0080] Sort the N water quality features in the initial water quality feature set according to their importance scores to obtain the feature sorting set S N_P = {I 1_P ,I 2_P ,I 3_P ,...,I N_P}, where I n_P is the feature with the highest importance score in the temporal training mode.

[0081] Remove the feature sorting set S N_P = {I 1_P ,I 2_P ,I 3_P ,...,I N_P The feature with the lowest importance score in N_P , get the feature sorting subset S N-1_P = {I 1_P ,I 2_P ,I 3_P ,...,I N-1_P}, and then sort the feature subset S N-1_P = {I 1_P ,I 2_P ,I 3_P ,...,I N-1_P} as a new feature sorting set and remove the feature I with the lowest importance score N-1_P , get a new feature sorting subset S N-2_P = {I 1_P ,I2_P ,I 3_P ,...,I N-2_P}, and so on, to obtain multiple feature sorting subsets S N-1_P ,S N-2_P ,...,S 1_P .

[0082] Use S N-1_P ,S N-2_P ,...,S 1_P The models in the temporal random forest training model set are trained respectively, and the root mean square error (RMSE) of the temporal random forest model set after training with each feature sorting subset is calculated as the evaluation standard of the model performance:

[0083]

[0084] Among them, RMSE g_P is the subset S sorted by the gth feature g_P The root mean square error of the random forest model set after training, 1≤g≤N-1, y is the actual response variable value, is the value of the response variable predicted by the random forest model at time a.

[0085] Get the root mean square error RMSE of the temporal random forest model set after training with N-1 feature sorting subsets in temporal training mode N-1_P ,RMSE N-2_P ,...,RMSE 1_P , and select the feature sorting subset corresponding to the minimum RMS error as the selected time-critical feature subset S g_P .

[0086] S5: Feature importance evaluation based on spatial training and iterative feature screening

[0087] The model constructed by random forest is spatially trained (K training). The specific steps are as follows:

[0088] The water quality characteristic data is divided into B spatial units using the locations of B monitoring stations on a river to obtain B spatial water quality characteristic data sets. Any one of the spatial water quality characteristic data sets is used as a test set to test the prediction ability of the random forest, and the remaining B-1 spatial water quality characteristic data sets are used to construct and train the spatial random forest model. A total of B spatial random forest models are trained to form a spatial random forest model set. The input of the spatial random forest model is the water quality characteristic, and the output is the water quality index, i.e., the response variable.

[0089] The average reduction in precision and the reduction in node impurity of the N features in the initial water quality feature set under B spatial random forest models were calculated respectively.

[0090] According to the following formula, the importance score of each water quality feature in the spatial random forest model set is obtained by normalization:

[0091]

[0092] Where: 1<n<N, N is the number of features in the initial water quality feature set, 1<b<B, FFIS n_K is the importance score of the nth feature under the spatial random forest model set, IM b is the average reduction in accuracy of the b-th spatial random forest model, IM B_min and IM B_max are the minimum and maximum average reduction values ​​of the precision in B spatial random forest training models, IN b is the node impurity reduction value of the b-th spatial random forest model, IN B_min and IN B_max are the minimum and maximum values ​​of node impurity reduction in B spatial random forest training models, respectively.

[0093] Sort the N water quality features in the initial water quality feature set according to their importance scores to obtain the feature sorting set S N_K = {I 1_K ,I 2_K ,I 3_K ,...,I N_K}, where In_ K is the feature with the nth-ranked importance score in the spatial training mode.

[0094] Remove the feature sorting set S N_K = {I 1_K ,I 2_K ,I 3_K ,...,I N_K The feature with the lowest importance score in N _ K , get the feature sorting subset S N-1_K = {I 1_K ,I 2_K ,I 3_K ,...,I N-1_K}, and then sort the feature subset S N-1_K = {I 1_K ,I 2_K ,I 3_K ,...,I N-1_K} as a new feature sorting set and remove the feature I with the lowest importance score N-1_ K , get a new feature sorting subset S N-2_K = {I 1_K ,I 2_K ,I 3_K ,...,I N-2_K}, and so on, to obtain multiple feature sorting subsets S N-1_K ,S N-2_K ,...,S 1_K .

[0095] Use S N-1_K ,S N-2_K ,...,S 1_K The models in the spatial random forest training model set are trained respectively, and the root mean square error of the spatial random forest model set after training with each feature sorting subset is calculated:

[0096]

[0097] Among them, RMSE h_K is the subset S sorted by the hth feature h_K The root mean square error of the spatial random forest model set after training, 1≤h≤N-1, y is the actual response variable value, is the value of the response variable predicted by the b-th spatial random forest model.

[0098] Get the root mean square error RMSE of the spatial random forest model set after training with N-1 feature sorting subsets in spatial training mode N-1_K ,RMSE N-2_K ,...,RMSE 1_K , and select the feature sorting subset corresponding to the minimum root mean square error as the selected spatial key feature subset S h_K .

[0099] S6: Dynamic optimization of feature selection and robustness verification of prediction models

[0100] For the time-critical feature subset S obtained in S4 g_P Eliminate the least important features in turn to obtain a new feature subset S g-1_P ,S g-2_P ,...,S 1_P , calculate the corresponding RMSE in turn g-1_P ,RMSE g-2_P ,...,RMSE 1_P , test the key feature subset S g_P Whether there is overfitting and the contribution of the selected features to the model's predictive ability, ensure that these features do play a key role in spatial and temporal extrapolation, and obtain the final prediction error of the selected feature subset.

[0101] Similarly, for the spatial key feature subset S obtained in S5 h_K The above method is also used to test the feature subset and obtain the final prediction error.

[0102] S7: Prediction model selection based on feature reduction and accuracy optimization

[0103] For the temporal key feature subset S in the temporal training mode g_P And its corresponding root mean square error RMSE g_P , and the spatial key feature subset S in the spatial training mode h_K And its corresponding root mean square error RMSE h_K , the training model is selected based on the principle of using fewer water quality features to achieve higher prediction accuracy. Specifically:

[0104] The key feature subset S for judgment time training g_P The number of features g and the feature subset S of spatial training h_K The size of the number of features h, and RMSE g_P and RMSE h_K The size of the judgment process is as follows:

[0105] When g≤h and RMSE g_P <RMSE h_K When S g_P As the optimal river water quality feature set, the time training mode is the best training mode.

[0106] When g ≥ h and RMSE g_P >RMSE h_K When S h_K As the optimal river water quality feature set, the spatial training mode is the best training mode.

[0107] When g<h and RMSE g_P >RMSE h_K When , calculate the key feature subset S of the g-th feature in the spatial training mode g_K The corresponding root mean square error RMSE g_K , and then judge the RMSE g_K and RMSE g_P The size of RMSE g_K >RMSE g_P , then S g_P As the optimal river water quality feature set, the time training mode is the best training mode. g_K <RMSE g_P , then S h_K As the optimal river water quality feature set, the spatial training mode is the best training mode.

[0108] When g>h and RMSE g_P <RMSE h_K When , calculate the key feature subset S of the hth feature in the time training mode h_P The corresponding root mean square error RMSE h_P , and then judge the RMSE h_P and RMSE h_K The size of RMSE h_P <RMSE h_K , then S g_P As the optimal river water quality feature set, the time training mode is the best training mode. h_P >RMSE h_K , then S h_K As the optimal river water quality feature set, the spatial training mode is the best training mode.

[0109] S8: Water Quality Prediction

[0110] In river network water quality monitoring, due to the lack of data in certain periods or locations, in order to fill the information gaps of these missing points, it is necessary to predict river water quality, including:

[0111] The training mode corresponding to the optimal river water quality feature set is used as the optimal training mode. Under the optimal training mode, the optimal river water quality feature set is used to train the optimal random forest model set. The optimal random forest model set is used to predict the river water quality indicators in the target area to obtain the prediction result set. The average value of all prediction results in the prediction result set is taken as the final prediction result of the river water quality in the target area. Specifically:

[0112] When the optimal training mode is the time training mode, A time random forest models are used to predict the water quality indicators of the missing points respectively, and A prediction results T are obtained. 1_P ,T 2_P ,...,T A_P Then take the average of these A prediction results as the water quality index T of the missing point P :

[0113]

[0114] When the best training mode is the spatial training mode, B spatial random forest models are used to predict the water quality indicators of the missing points respectively, and B prediction results T are obtained. 1_K ,T 2_K ,...,T B_K Then take the average of these B prediction results as the water quality index T of the missing point K :

[0115]

[0116] Through this method, water quality data of the entire river network can be effectively inferred and predicted, thereby obtaining a comprehensive water quality data set.

[0117] This implementation method uses Random Forest (RF) to perform two training settings based on the two dimensions of time and space, time training and space training, which can better capture the complexity of the river network in space and time, and improve the prediction robustness of the model for missing data or unobserved locations. The importance ranking of water quality features is obtained by using multiple feature importance evaluation methods, and the least important features are eliminated in turn to obtain a new feature set and the prediction error of the new set is evaluated. The best feature set under the two training settings is obtained through iterative simplification of the feature set. According to the principle of few features and high precision, the best extrapolation model is selected to perform spatiotemporal extrapolation of river water quality, providing a theoretical basis for pollution control and comprehensive management of rivers.

[0118] Specific implementation method 2: refer to Figures 2 to 6 This implementation method is specifically described. This implementation method takes the COD concentration of a river in City B, Province A, in the Yellow River Basin as an example. The specific implementation process is as follows:

[0119] (1) Using DEM to study the division of watershed

[0120] Collect the digital elevation model (DEM) data of the target area, and identify the high and low points of the target area terrain by analyzing the DEM data. Based on the high and low points of the target area terrain, apply the water flow direction algorithm to determine the water flow direction of the target area and calculate the accumulated water flow. Use the water flow direction and accumulated water flow in the target area to identify the water flow path and extract the river network. On this basis, according to the water flow direction and the terrain characteristics of the target area, use the watershed division algorithm to determine the watershed boundary, analyze the properties of the watershed, and form a contour map of the catchment area. Finally, verify and correct the extracted river network and watershed by comparing with existing maps or field survey data to ensure their accuracy.

[0121] (2) Data cleaning and data set construction

[0122] Based on statistical yearbooks, environmental bulletins and public data platforms, the multiple factors affecting the chemical oxygen demand (COD) of rivers were analyzed. These factors include not only natural conditions, such as meteorological and climatic characteristics, but also the impact of human activities, such as land use type and pollution caused by human activities. A comprehensive water quality characteristic system was constructed, covering cultivated land (TDA), forest land (TDB), grassland (TDC), water area (TDD), urban and rural industrial and mining residential land (TDE), urban permanent population (CP), rural permanent population (RP), gross domestic product (GDP), maximum temperature (HT), minimum temperature (LT), rainfall (R), wind speed (WS), wind direction (WD), monitoring station number (S), month (M), date (D), water temperature (T), turbidity (NTU) and pH (pH).

[0123] Since the river is frozen during the period of ice cover, there is a lack of water quality data from November to April of the following year. Therefore, the data of the four monitoring stations (No. 1, 2, 3, and 4) of the river are selected from May to October 2023 for 6 months on an hourly scale. These data are averaged on a daily scale to obtain the required environmental data on a daily scale. The missing values ​​are identified by heat map, and the Pandas interpolation filling method is used to fill the missing values; then, the Winsorizing technology is used to effectively remove the outliers in the data to ensure the accuracy and reliability of the data set. After the above processing, a data set suitable for subsequent modeling is obtained.

[0124] (3) Preliminary screening of water quality characteristics based on the Pearson coefficient

[0125] The Pearson coefficient method was used to calculate the correlation coefficient between the response variable COD and the initially selected characteristic system, including cultivated land TDA, forest land TDB, grassland TDC, water area TDD, urban and rural industrial and mining residential land TDE, urban permanent population CP, rural permanent population RP, gross domestic product GDP, maximum temperature HT, minimum temperature LT, rainfall R, wind speed WS, wind direction WD, site number S, month M, date D, water temperature T, turbidity NTU and pH. By calculating the Pearson correlation coefficients of these characteristics with COD, water quality characteristics with a Pearson correlation coefficient greater than 0.3 were selected as the initial water quality characteristics for subsequent model training.

[0126] Table 1 Pearson correlation coefficients between COD and screening characteristics

[0127]

[0128] (4) Feature importance evaluation based on time training and iterative feature screening

[0129] Random forest was used to train the model over time. Five months of data from all sites were selected for model training, and the remaining month of data was used for model testing. In order to ensure that all months of data could be used for training and testing, a total of 6 random forest models were trained. IncMSE and IncNodePurity were used to evaluate the importance of each feature variable in the 6 models. The importance scores of the 11 feature variables were obtained by normalization. a :

[0130] Table 2. The importance score order of feature variables in time training setting

[0131]

[0132] The water quality data of the excluded months were used as the test set to verify the model. The RMSE mean of the six models was used as an indicator to evaluate the quality of the selected feature set. The least important features were gradually excluded to form a new feature subset, and the root mean square error (RMSE) of the new subset was calculated. Through iterative simplification, the subset that minimized the RMSE was obtained as the selected feature set. Figure 2 This is a RMS error analysis diagram for different feature combinations, where each circle contains the RMS error of the feature subset combined with all the features on the left, and the red circle represents the feature set that minimizes the RMS error of the test set. Figure 2 It can be seen that under the time training setting, the feature set that minimizes the RMSE of the model's test set is pH, T, RP, TDC, HT, LT, and TDE.

[0133] (5) Feature importance evaluation based on spatial training and iterative feature screening

[0134] The model was spatially trained using random forests. Six months of data from three sites were selected for model training, and the data from the remaining site was used for model testing. In order to ensure that data from all sites could be used for training and testing, a total of four random forest models were trained. IncMSE and IncNodePurity were used to calculate the importance score of each feature variable for each of the four random forest models. The FFIS of the 11 feature variables was obtained using the normalized method. b :

[0135] Table 3 Importance score order of feature variables in spatial training setting

[0136]

[0137] The water quality data of the excluded stations were selected as the test set to test the model. The RMSE mean of the four models was used as an indicator to evaluate the quality of the selected feature set. The least important features were gradually excluded to form a new feature subset, and the RMSE of the new subset was calculated. Through iterative simplification, the subset that can minimize the RMSE is the required feature set, such as Figure 3 As shown, according to Figure 3 It can be seen that under the spatial training setting, the feature set that minimizes the RMSE of the model's test set is LT, pH, NTU, TDA, PR, TDC, TDE, and M.

[0138] (6) Dynamic optimization of feature selection and robustness verification of prediction models

[0139] The feature subsets selected in the time training setting are retrained in the six random forest models to obtain the RMSE mean of the excluded months of the six models. The least important features are gradually excluded to obtain a new feature set. The RMSE of the new set is calculated to test whether the feature subset selected in step 4 is the best set and obtain the final prediction error. Figure 4 The RMSE obtained by retraining the feature subset selected for time training. Figure 4 It can be seen that the seven features pH, T, RP, TDC, HT, LT, and TDE selected in step (4) are the optimal feature set R that minimizes RMSE under the time training setting. j , the RMSE at this time j =5.94.

[0140] The feature subsets selected in the spatial training setting were retrained in the four random forest models to obtain the RMSE mean of the excluded sites in the four models, and the least important features were gradually excluded to obtain a new feature set. The RMSE of the new set was calculated to test whether the feature subset selected in step 5 was the optimal set, and the final prediction error was obtained. Figure 5 The RMSE obtained by retraining the feature subset selected for spatial training. Figure 5 It can be seen that the eight features LT, pH, NTU, TDA, PR, TDC, TDE and M selected in step (5) are the optimal feature set R that minimizes RMSE under the spatial training setting. m , the RMSE at this time m =8.65.

[0141] (7) Prediction model selection based on feature simplification and accuracy optimization

[0142] From step (5), we can see that the key feature subset R in the time training mode j The number of features is 7, and the final prediction error RMSE j=5.94, key feature subset R in spatial training mode m The number of features is 8, and the final prediction error RMSE m =8.65, due to R j <R m And RMSE j <RMSE m , so 6 random forest models in the time training mode are selected as the final prediction models.

[0143] (8) Prediction and filling of river network water quality data based on optimal model

[0144] Six random forest models in the time training mode were used to predict the COD of station 5 of the river (the water quality data of this station was not used for model training or model testing) for 31 days from September 15, 2023 to October 15, 2023. The predicted results were different from the measured values ​​as shown in the figure. Figure 6 As shown by Figure 6 It can be seen that this implementation has a good prediction effect on the simulation of water quality indicators in unmonitored areas, and limited data can be further used to perform spatiotemporal extrapolation prediction of water quality indicators in the entire river network.

[0145] Although the present invention is described herein with reference to specific embodiments, it should be understood that these embodiments are merely examples of the principles and applications of the present invention. It should therefore be understood that many modifications may be made to the exemplary embodiments and that other arrangements may be devised without departing from the spirit and scope of the present invention as defined by the appended claims. It should be understood that the various dependent claims and features described herein may be combined in a manner different from that described in the original claims. It should also be understood that the features described in conjunction with a single embodiment may be used in other described embodiments.

Claims

1. A spatiotemporal adaptive prediction method for river water quality based on feature enhancement, characterized in that: include: The water quality characteristic data of any river are divided using time and space as step units respectively to obtain a temporal water quality characteristic data set and a spatial water quality characteristic data set; The temporal water quality characteristic data set and the spatial water quality characteristic data set are used to construct and train a temporal random forest model set in a temporal training mode and a spatial random forest model set in a spatial training mode, respectively. The inputs of the temporal random forest model and the spatial random forest model are both water quality characteristic data, and the outputs are both water quality indicators, and the water quality indicators are response variables of water quality; Based on the two random forest model sets, the features in the initial water quality feature set are preliminarily screened to obtain a time key feature subset in the time training mode and a space key feature subset in the space training mode. The two random forest model sets perform the same preliminarily screening method, and the preliminarily screening includes: Calculate the importance score of each feature in the initial water quality feature set under the current random forest model set respectively, reorder the features in the initial water quality feature set according to the importance score to obtain a feature sorting set; remove the feature with the lowest importance score in the feature sorting set to obtain a feature sorting subset, then use the feature sorting subset as a new feature sorting set and remove the feature with the lowest importance score therein to obtain a new feature sorting subset, and so on to obtain multiple feature sorting subsets; use multiple feature sorting subsets to train each random forest model in the current random forest model set respectively, calculate the root mean square error of the current random forest model set after training with each feature sorting subset, and use the feature sorting subset corresponding to the minimum root mean square error as the key feature subset under the current training mode; The features are finely screened according to the number of features in the key feature subsets under the two training modes and the corresponding root mean square errors, and the fine screening includes: Let S g_P and S h_K are the key feature subsets under the two training modes, P and K represent the two training modes, g and h represent S g_P and S h_K The number of features in , RMSE g_P and RMSE h_K S g_P and S h_K The root mean square error of the random forest model set after training is: When g≤h and RMSE g_P <RMSE h_K When S g_P As the optimal set of river water quality characteristics; When g ≥ h and RMSE g_P >RMSE h_K When S h_K As the optimal set of river water quality characteristics; When g≤h and RMSE g_P >RMSE h_K When , calculate the g-th feature sorting subset S under K training mode g_K The corresponding root mean square error RMSE g_K , and then judge the RMSE g_K and RMSE g_P The size of RMSE g_K >RMSE g_P , then S g_P As the optimal river water quality feature set, if RMSE g_K <RMSE g_P , then S h_K As the optimal set of river water quality characteristics; When g ≥ h and RMSE g_P <RMSE h_K When P is used to calculate the hth feature sorting subset S in the P training mode h_P The corresponding root mean square error RMSE h_P , and then judge the RMSE h_P and RMSE h_K The size of RMSE h_P <RMSE h_K , then S g_P As the optimal river water quality feature set, if RMSE h_P >RMSE h_K , then S h_K As the optimal set of river water quality characteristics; The training mode corresponding to the optimal river water quality feature set is used as the optimal training mode, and the optimal random forest model set is trained using the optimal river water quality feature set under the optimal training mode; The optimal random forest model set is used to predict the river water quality indicators in the target area and obtain the prediction result set; The average value of all prediction results in the prediction result set is taken as the final prediction result of river water quality in the target area.

2. The method for spatiotemporal adaptive prediction of river water quality based on feature enhancement according to claim 1 is characterized in that: The method for obtaining the initial water quality feature set includes: Collect digital elevation model data of the target area to obtain the high and low points of the target area, use the water flow direction algorithm to determine the water flow direction and water flow accumulation of the target area in combination with the high and low points of the target area, extract the river network of the target area according to the water flow direction and water flow accumulation, and use the watershed division algorithm to determine the watershed boundary of the river network according to the water flow direction and the terrain characteristics of the target area to form a watershed area contour map; Taking each area in the watershed area outline map as a unit, collect water quality characteristic data of the target area, eliminate invalid and erroneous water quality characteristic data, interpolate or complete the missing water quality characteristic data, smooth or correct the abnormal water quality characteristic data, and obtain a rough water quality characteristic data set; The Pearson correlation coefficient matrix R is constructed using the water quality characteristics and water quality response variables in the crude water quality characteristic data set. The elements R ij represents the Pearson correlation coefficient between the i-th water quality characteristic and the j-th response variable, 1≤i≤M, 1≤j≤M′, M is the number of water quality characteristics in the crude water quality characteristic dataset, and M′ is the number of response variables; The features corresponding to the elements in R that are less than a preset coefficient threshold are removed, and the remaining water quality features in the rough water quality feature data set constitute the initial water quality feature set.

3. A spatiotemporal adaptive prediction method for river water quality based on feature enhancement according to claim 1 or 2, characterized in that: In the time training mode, the water quality characteristic data of any river are divided into A time step units to obtain A time water quality characteristic data sets; A-1 time water quality characteristic data sets are arbitrarily selected for training to obtain A time random forest models, and the A time random forest models constitute the time random forest model set in the time training mode.

4. The method for spatiotemporal adaptive prediction of river water quality based on feature enhancement according to claim 3 is characterized in that: The temporal random forest model set is used to preliminarily screen the features in the initial water quality feature set to obtain the temporal key feature subset under the temporal training mode, including: Calculate the average precision reduction value and node impurity reduction value of each feature in the initial water quality feature set under the random forest model at each time; The importance score of each water quality feature under the temporal random forest model set was calculated using the normalization method; Reorder the features in the initial water quality feature set according to their importance scores to obtain a feature ranking set S N_P = {I 1_P ,I 2_P ,I 3_P ,...,I N_P }, where I n_P is the feature with the highest importance score in the time training mode, then the feature ranking subset is represented as S N-1_P ,S N-2_P ,...,S 1_P ; Use S N-1_P ,S N-2_P ,...,S 1_P The models in the temporal random forest training model set are trained respectively, and the root mean square error of the temporal random forest model set after training with each feature sorting subset is calculated; The feature sorting subset corresponding to the minimum RMS error is used as the temporal key feature subset in the temporal training mode.

5. A spatiotemporal adaptive prediction method for river water quality based on feature enhancement according to claim 1 or 2, characterized in that: In the spatial training mode, the water quality characteristic data of any river are divided into B monitoring locations as spatial units to obtain B spatial water quality characteristic data sets; B-1 spatial water quality characteristic data sets are arbitrarily selected for training to obtain B spatial random forest models, and the B spatial random forest models constitute the spatial random forest model set in the spatial training mode.

6. A spatiotemporal adaptive prediction method for river water quality based on feature enhancement according to claim 5, characterized in that: The spatial random forest model set is used to preliminarily screen the features in the initial water quality feature set to obtain a subset of spatial key features under the spatial training mode, including: Calculate the average precision reduction value and node impurity reduction value of each feature in the initial water quality feature set under each spatial random forest model respectively; The importance score of each water quality feature under the spatial random forest model set was calculated using the normalization method; Reorder the features in the initial water quality feature set according to their importance scores to obtain a feature ranking set S N_K = {I 1_K ,I 2_K ,I 3_K ,...,I N_K }, where In_ K is the feature with the highest importance score in the spatial training mode, then the feature ranking subset is represented as S N-1_K ,S N-2_K ,...,S 1_K ; Use S N-1_K ,S N-2_K ,...,S 1_K The models in the spatial random forest training model set are trained respectively, and the root mean square error of the spatial random forest model set after training with each feature sorting subset is calculated; The feature sorting subset corresponding to the minimum root mean square error is used as the spatial key feature subset in the spatial training mode.

7. A spatiotemporal adaptive prediction method for river water quality based on feature enhancement according to claim 1, 4 or 6, characterized in that: The normalization method is used to calculate the importance score of each water quality feature under the current random forest model set. The expression is as follows: Where: 1<n<N, N is the number of features in the initial water quality feature set, 1<q<Q, Q is the number of models included in the current random forest model set, FFIS n is the importance score of the nth feature under the current random forest model set, IM q is the average reduction in accuracy of the qth random forest model, IM Q_min and IM Q_max are the minimum and maximum average reduction values ​​of the precision in the Q random forest training models, IN q is the node impurity reduction value of the qth random forest model, IN Q_min and IN Q_max are the minimum and maximum values ​​of node impurity reduction in Q random forest training models, respectively.

8. The method for spatiotemporal adaptive prediction of river water quality based on feature enhancement according to claim 7 is characterized in that: Calculate the root mean square error of the current random forest model set after training with each feature sorting subset. The expression is as follows: Among them, RMSE f is the root mean square error of the current spatial random forest model set trained by the f-th feature sorting subset, 1≤f≤N-1, y is the actual response variable value, is the value of the response variable predicted by the qth random forest model.

9. The method for spatiotemporal adaptive prediction of river water quality based on feature enhancement according to claim 1, characterized in that: When the best training mode is the time training mode, the final prediction result of river water quality is: When the best training mode is the spatial training mode, the final prediction result of river water quality is: T 1_P ,T 2_P ,...,T A_P represents the prediction result obtained by the best training mode, T P is the final prediction result of river water quality obtained by using the time training model; T 1_P ,T 2_P ,...,T A_P represents the prediction result obtained by the best training mode, T K The final prediction results of river water quality obtained using the spatial training model.

10. The method for spatiotemporal adaptive prediction of river water quality based on feature enhancement according to claim 1, characterized in that: After the key feature subset is obtained, the key feature subset is tested, and fine screening is performed after passing the test.

Images