A feature enhancement-based spatiotemporal adaptive prediction method for river water quality

Abstract

The application discloses a river water quality space-time self-adaptive prediction method based on feature enhancement, and relates to the field of environmental engineering.The application is to solve the problems that key features of river water quality are difficult to screen and water quality prediction is inaccurate.The application collects river water quality feature data of a target area to perform cleaning and complementing, and constructs a data set;Pearson correlation coefficient method is adopted to perform rough screening on the features, then two training settings are performed on a random forest model, IncMSE and IncNodePurity two methods are selected to obtain importance scores of the river water quality features, the two importance scores are normalized and reordered, and key features are found;and then, an optimal training mode and a corresponding model are found, and prediction of water quality indexes of the whole river network is realized.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of environmental engineering, and particularly relates to river water quality prediction technology. BACKGROUND

[0002] River water quality is an important indicator of the health of the river ecosystem, and is also an important resource for human life and economic development. Monitoring and evaluation of river water quality is the basis and premise of water environment management, and is also an important means of water resources protection and utilization. In order to effectively monitor and manage river water quality, water quality indicators such as dissolved oxygen DO, chemical oxygen demand COD, total nitrogen TN, total phosphorus TP, etc. need to be collected at different locations and specific times in the river network to reflect the material cycle and ecological function of the river. However, due to the complexity of the river network and cost constraints, traditional water quality sampling methods often fail to cover the entire range of the river network, resulting in insufficient spatial resolution and accuracy of water quality data, and failing to fully reflect the spatial and temporal distribution and variation of river water quality.

[0003] River water quality is not only affected by natural environment, but also by human activities, and is the result of the combined action of various factors. Rainfall, temperature and other climate can indirectly affect river water quality by affecting river water level, flow rate, etc. Urbanization and industrial emissions can affect river water quality by affecting the migration and transformation of pollutants. Therefore, due to the complex factors affecting river water quality, identifying and selecting key features that affect river water quality indicators can help policymakers and environmental managers develop and implement more targeted management measures to improve river water quality.

[0004] The spatial and temporal characteristics of river water quality are crucial for effective monitoring and management. Water quality data not only shows significant heterogeneity in space, but also presents complex dynamic changes in time. However, existing water quality monitoring techniques often lack the ability to adapt to such spatio-temporal characteristics, making it difficult to accurately predict and respond to water quality changes in practical applications. Spatio-temporal adaptive technology, by dynamically adjusting spatio-temporal features to adapt to the spatio-temporal changes of water quality data, provides a new approach to solving this problem.

[0005] River water quality extrapolation inversion refers to the process of predicting and inferring the water quality of unmonitored areas or unobserved time points based on existing water quality monitoring data and related information, with the help of mathematical models or statistical methods. Existing water quality extrapolation inversion models and techniques often require a large number of water quality features for learning and training. Human activities and natural factors also make water quality present complex variation rules in time and space. Existing models cannot accurately predict water quality with limited data. Therefore, it is urgent to propose a method that can efficiently infer the missing data and water quality indicators of unmonitored areas from the two dimensions of time and space using limited water quality features of river sections, and improve the accuracy of water quality prediction. SUMMARY

[0006] The present application is to solve the problem that it is difficult to screen the key characteristics of river water quality due to the complex influencing factors of river water quality, and the existing model cannot accurately predict water quality with limited data. A river water quality spatio-temporal adaptive prediction method based on feature enhancement is provided.

[0007] A river water quality spatio-temporal adaptive prediction method based on feature enhancement, comprising:

[0008] Divide the water quality feature data of any river by taking time and space as step units respectively to obtain a time water quality feature data set and a space water quality feature data set;

[0009] Use the time water quality feature data set and the space water quality feature data set to respectively construct and train a time random forest model set under a time training mode and a space random forest model set under a space training mode, the inputs of the time random forest model and the space random forest model are both water quality feature data, and the outputs are both water quality indexes, the water quality indexes being response variables of water quality;

[0010] Based on the two random forest model sets, perform initial screening on the features in the initial water quality feature set to obtain a time key feature subset under the time training mode and a space key feature subset under the space training mode, the initial screening methods of the two random forest model sets being the same, and the initial screening comprising:

[0011] Calculate the importance score of each feature in the initial water quality feature set under the current random forest model set respectively, reorder the features in the initial water quality feature set according to the importance scores to obtain a feature ordering set, remove the feature with the lowest importance score in the feature ordering set to obtain a feature ordering subset, and then take the feature ordering subset as a new feature ordering set and remove the feature with the lowest importance score therefrom to obtain a new feature ordering subset, and so on to obtain multiple feature ordering subsets; use the multiple feature ordering subsets to train each random forest model in the current random forest model set respectively, calculate the root mean square error of the current random forest model set after training by each feature ordering subset, and take the feature ordering subset corresponding to the minimum root mean square error as the key feature subset under the current training mode;

[0012] According to the number of features in the key feature subsets under the two training modes and the corresponding root mean square errors, perform fine screening on the features, the fine screening comprising:

[0013] Let S g_P and S h_K be the key feature subsets under the two training modes, P and K represent the two training modes respectively, and g and h be S g_P and Sh_K the number of features in S g_P and RMSE h_K are the root mean square errors of the set of random forest models trained on S g_P and S h_K respectively, then we have:

[0014] When g < h and RMSE g_P < RMSE h_K , S g_P is the optimal set of river water quality features;

[0015] When g > h and RMSE g_P > RMSE h_K , S h_K is the optimal set of river water quality features;

[0016] When g < h and RMSE g_P > RMSE h_K , calculate the root mean square error RMSE g_K of the gth feature ranking subset S g_K under the K training mode, then judge the size of RMSE g_K and RMSE g_P , if RMSE g_K > RMSE g_P , then S g_P is the optimal set of river water quality features, if RMSE g_K < RMSE g_P , then S h_K is the optimal set of river water quality features;

[0017] When g > h and RMSE g_P < RMSE h_K , calculate the root mean square error RMSE h_P of the hth feature ranking subset S h_P under the P training mode, then judge the size of RMSE h_P and RMSE h_K , if RMSE h_P < RMSE h_K , then S g_P is the optimal set of river water quality features, if RMSE h_P > RMSE h_K , then S h_K is the optimal set of river water quality features;

[0018] The optimal set of river water quality features corresponds to the best training mode, and the optimal set of random forest models is trained on the best training mode using the optimal set of river water quality features;

[0019] The optimal random forest model set is used to predict the river water quality indexes of the target region, and a prediction result set is obtained.

[0020] The average value of all prediction results in the prediction result set is taken as the final prediction result of the river water quality of the target region.

[0021] Further, the method for obtaining the initial water quality feature set comprises:

[0022] Collecting digital elevation model data of the target region to obtain high and low points of the target region, and using a water flow direction algorithm to determine the water flow direction and water flow accumulation of the target region in combination with the high and low points of the target region, extracting the river network of the target region according to the water flow direction and water flow accumulation, determining the watershed boundary of the river network using a watershed division algorithm according to the water flow direction and the topographic features of the target region, and forming a watershed area contour map;

[0023] Collecting water quality feature data of the target region in units of each region in the watershed area contour map, eliminating invalid and erroneous water quality feature data, interpolating or completing missing water quality feature data, and smoothing or correcting abnormal water quality feature data to obtain a coarse water quality feature data set;

[0024] Using the water quality features in the coarse water quality feature data set and the response variables of water quality to construct a Pearson correlation coefficient matrix R, the elements R ij in R represent the Pearson correlation coefficient between the ith water quality feature and the jth response variable, 1≤i≤M, 1≤j≤M', M is the number of water quality features in the coarse water quality feature data set, and M' is the number of response variables;

[0025] Removing the features corresponding to the elements in R that are less than a preset coefficient threshold, and the remaining water quality features in the coarse water quality feature data set constitute the initial water quality feature set.

[0026] Further, in the time training mode, the water quality feature data of any river is divided into A time step units to obtain A time water quality feature data sets; A-1 time water quality feature data sets are randomly selected to train A time random forest models, and the A time random forest models constitute a time random forest model set in the time training mode.

[0027] Further, the time random forest model set is used to preliminarily screen the features in the initial water quality feature set to obtain a time key feature subset in the time training mode, which comprises:

[0028] The average precision reduction value and the node impurity reduction value of each feature in the initial water quality feature set under each time random forest model are calculated respectively;

[0029] The importance score of each water quality feature under the temporal random forest model set was calculated using a normalization method.

[0030] The features in the initial water quality feature set are reordered according to their importance scores to obtain the feature sorting set S. N_P ={I 1_P ,I 2_P ,I 3_P ,...,I N_P}, where I n_P Let S be the feature ranked nth in importance score under the time-based training model. Then the feature ranking subset is represented as S. N-1_P ,S N-2_P ,...,S 1_P ;

[0031] Use S N-1_P ,S N-2_P ,...,S 1_P The models in the Temporal Random Forest training model set are trained separately, and the root mean square error of the Temporal Random Forest model set after training with each feature sorting subset is calculated.

[0032] The feature sorting subset corresponding to the minimum root mean square error is used as the time key feature subset in the time training mode.

[0033] Furthermore, in the spatial training mode, the water quality characteristic data of any river is divided into B spatial water quality characteristic datasets using B monitoring locations as spatial units; B-1 spatial water quality characteristic datasets are arbitrarily selected to train B spatial random forest models, and the B spatial random forest models constitute the spatial random forest model set in the spatial training mode.

[0034] Furthermore, a spatial random forest model set is used to initially screen the features in the initial water quality feature set, obtaining a subset of key spatial features under the spatial training mode, including:

[0035] Calculate the average reduction in precision and the reduction in node impurity for each feature in the initial water quality feature set under each spatial random forest model;

[0036] The importance score of each water quality feature under the spatial random forest model set was calculated using a normalization method.

[0037] The features in the initial water quality feature set are reordered according to their importance scores to obtain the feature sorting set S. N_K ={I 1_K ,I 2_K ,I 3_K ,...,I N_K}, where I n_KFor the feature ranked n in the importance score in the spatial training mode, the feature ranking subset is represented as S N-1_K N-2_K 1_K

[0038] The spatial random forest training model set is trained using S N-1_K N-2_K 1_K respectively, and the root mean square error of the spatial random forest model set trained by each feature ranking subset is calculated.

[0039] The feature ranking subset corresponding to the minimum root mean square error is taken as the spatial key feature subset in the spatial training mode.

[0040] Further, the importance score of each water quality feature in the current random forest model set is calculated using the normalization method, and the expression is as follows:

[0041]

[0042] In the formula: 1 n n q is the importance score of the nth feature in the current random forest model set, IM Q_min q Q_max is the accuracy average reduction value of the qth random forest model, IM q and IM Q_min are the minimum and maximum values of the accuracy average reduction value in the Q random forest training models, IN q is the node impurity reduction value of the qth random forest model, IN Q_min and IN Q_max are the minimum and maximum values of the node impurity reduction value in the Q random forest training models.

[0043] Further, the root mean square error of the current random forest model set trained by each feature ranking subset is calculated, and the expression is as follows:

[0044]

[0045] where RMSE f is the root mean square error of the current spatial random forest model set trained by the fth feature ranking subset, 1 is the actual response variable value,

[0046] Further, when the best training mode is the time training mode, the final prediction result of the river water quality is: ​

[0047]

[0048] When the optimal training mode is the spatial training mode, the final prediction result of the river water quality is:

[0049]

[0050] T 1_P ,T 2_P ,...,T A_P The prediction result obtained by training the optimal training mode is represented by T P The final prediction result of the river water quality obtained by using the time training mode is represented by T 1_P ,T 2_P ,...,T A_P The prediction result obtained by training the optimal training mode is represented by T K The final prediction result of the river water quality obtained by using the spatial training mode is represented by T

[0051] Further, after obtaining the key feature subset, the key feature subset is tested, and after passing the test, fine screening is performed.

[0052] The river water quality spatio-temporal adaptive prediction method based on feature enhancement has the following beneficial effects:

[0053] 1. Generalization model training based on space-time dimensions: two training modes, namely time training and space training, are proposed, which can better capture the complexity of the river network in space and time, improve the generalization ability of the model, and better adapt to new data, and use limited data to accurately predict missing data or unobserved positions.

[0054] 2. Spatio-temporal adaptive feature iteration screening: a spatio-temporal feature adaptive iteration mechanism is established, and the feature set is dynamically adjusted according to the collected spatio-temporal data. A least important feature is excluded in turn, and the RMSE of the remaining feature set is calculated. The feature set is dynamically adjusted through feedback to obtain the best feature subset, which can improve the prediction accuracy of the model while reducing the complexity of the model.

[0055] 3. Model optimization with few features and high precision: by comprehensively considering the feature quantity and prediction accuracy of the two training modes, the most suitable model is selected according to the principle of few features and high precision to extrapolate and invert the water quality indicators.

[0056] 4. Multiple feature importance comprehensive evaluation: The final importance score of the feature is obtained by comprehensively considering the node impurity reduction value and the average precision reduction value of each feature, avoiding the limitations brought by using a single indicator, and more accurately reflecting the importance of each water quality indicator to the prediction of water quality indicators.

[0057] 5. Limited feature implementation data missing basin accurate prediction: Through adaptive screening of key features and combining an optimized prediction model, the water quality indicators of the entire river network can be effectively predicted. Even in the case of missing water quality data in some time periods or locations, the water quality conditions of these regions can still be accurately calculated with limited data. This provides strong data support for the comprehensive management and governance of the basin. BRIEF DESCRIPTION OF DRAWINGS

[0058] Figure 1 A flowchart of a river water quality spatio-temporal adaptive prediction method based on feature enhancement;

[0059] Figure 2 A root mean square error curve graph of the test set under different feature combinations in the time training mode;

[0060] Figure 3 A root mean square error curve graph of the test set under different feature combinations in the spatial training mode;

[0061] Figure 4 A root mean square error curve graph of different feature combinations after re-training in the time training mode;

[0062] Figure 5 A root mean square error curve graph of different feature combinations after re-training in the spatial training mode;

[0063] Figure 6 A comparison graph of random forest prediction values and actual values of a certain station in the time training mode. DETAILED DESCRIPTION

[0064] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor are within the scope of protection of the present application. It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0065] Specific embodiment one: reference Figure 1To illustrate the present embodiment, the present embodiment describes a river water quality spatio-temporal self-adaptive prediction method based on feature enhancement, which comprises the following steps: collecting data of river water quality features in a study area, cleaning and completing the collected data, constructing a data set, performing coarse screening of features by using the Pearson correlation coefficient method, then using random forest for two training settings, selecting two methods of increase in mean squared error (IncMSE) and increase in node purity (IncNodePurity) to obtain the importance score of river water quality features, using the normalization method to sort the importance of the key features affecting the river water quality index, and finding the key features; according to the selected key features, re-conducting model construction, testing whether overfitting occurs and the contribution of the selected features to the prediction ability, and at the same time, according to the principle of achieving higher prediction accuracy with fewer features, selecting the most suitable prediction model, using limited feature data to deduce the entire river network, and realizing the prediction of the water quality index of the entire river network.

[0066] The specific process is as follows:

[0067] S1: Divide the target basin by using DEM

[0068] Collect the digital elevation model (DEM) data of the target area, identify the high points and low points of the terrain of the target area by analyzing the DEM data. Based on the high points and low points of the terrain of the target area, determine the water flow direction of the target area by using the water flow direction algorithm, and calculate the water flow accumulation. Identify the water flow path and extract the river network by using the water flow direction and water flow accumulation of the target area. On this basis, according to the water flow direction and the topographic features of the target area, determine the watershed boundary by using the watershed division algorithm, and analyze the attributes of the watershed to form the contour map of the catchment area. Finally, compare with the existing map or field survey data to verify and correct the extracted river network and watershed, and ensure its accuracy.

[0069] S2: Data cleaning of the watershed and construction of the data set

[0070] A river water quality prediction index system is constructed by comprehensively considering various influencing factors of river water quality. Specifically, based on various statistical yearbooks, environmental bulletins and public data platforms, water quality characteristic data of the target basin is collected in units of S1 divided catchment areas. Quality control and outlier detection are performed on the collected water quality characteristic data, specifically: invalid and erroneous data are removed, missing data are interpolated or completed, and abnormal data are smoothed or corrected to make the data more true and continuous. A water quality characteristic data set suitable for subsequent analysis is formed, making the data set representative and comparable.

[0071] S3: Water quality characteristic preliminary screening based on Pearson correlation coefficient method

[0072] Pearson correlation coefficient is used to detect the degree of linear correlation between two continuous variables, with a value range of [-1, 1], positive value indicating positive correlation, negative value indicating negative correlation, and the greater the absolute value indicating the higher the degree of linear correlation. An MxM' correlation coefficient matrix R is constructed using M water quality characteristics and M' response variables, then R ij represents the Pearson correlation coefficient between the ith water quality characteristic and the jth response variable, 1≤i≤M, 1≤j≤M'. When |R ij |>τ (τ is a threshold value), it is considered that the ith feature is significantly correlated with the jth response variable. Exclude water quality characteristics with insignificant correlation with response variables, and the remaining number of water quality characteristics is N, N

[0073] S4: Feature importance evaluation and iterative selection based on time training

[0074] The model constructed by random forest is time trained (P training), and the specific steps are as follows:

[0075] Using water quality characteristic data of a river A time steps (here it can be month, year, etc., if the time step is day or hour data, randomly select 2 / 3 of the water quality characteristic data for training, and the remaining 1 / 3 of the water quality characteristic data for testing), A time water quality characteristic data sets are obtained. Any one of the time water quality characteristic data sets is used as a test set to test the prediction ability of the random forest, and the remaining A-1 time water quality characteristic data sets are used for construction and training of the time random forest model, a total of A time random forest models are trained to form a time random forest model set. The input of the time random forest model is water quality characteristics, and the output is water quality indicators, i.e. response variables.

[0076] The average precision reduction value and node impurity reduction value of the N features in the initial water quality characteristic set under the A time random forest models are calculated respectively.

[0077] The Final Feature Importance Score (FFIS) for each water quality feature in the temporal random forest model set is obtained using the following formula and normalization method:

[0078]

[0079] In the formula: 1 < n < N, N is the number of features in the initial water quality feature set, 1 < a < A, FFIS n_P IM is the importance score of the nth feature in the temporal random forest model set. a IM represents the average reduction in accuracy of the random forest model at time a. A_min and IM A_max Let IN be the minimum and maximum values ​​of the average decrease in precision in A time-random forest training models. a IN represents the reduction in node impurity in the random forest model at time a. A_min and IN A_max These are the minimum and maximum values ​​of the reduction in node impurity in A time-random forest training models, respectively.

[0080] The N water quality features in the initial water quality feature set are ranked according to their importance scores to obtain the feature ranking set S. N_P ={I 1_P ,I 2_P ,I 3_P ,...,I N_P}, where I n_P The nth feature is ranked by importance score in the time-based training mode.

[0081] Remove the feature sorting set S N_P ={I 1_P ,I 2_P ,I 3_P ,...,I N_P The feature with the lowest importance score in} N_P Obtain the feature-ranked subset S N-1_P ={I 1_P ,I 2_P ,I 3_P ,...,I N-1_P Then sort the feature subset S. N-1_P ={I 1_P ,I 2_P ,I 3_P ,...,I N-1_P} as a new feature ranking set and remove the feature I with the lowest importance score. N-1_P Obtain a new feature-ranked subset S N-2_P ={I 1_P ,I2_P I 3_P ..., I N-2_P}, and so on, to obtain a plurality of feature ranking subsets S N-1_P S N-2_P ..., S 1_P .

[0082] The S N-1_P S N-2_P ..., S 1_P is trained respectively, and the root mean square error (RMSE) of the time random forest model set trained by each feature ranking subset is calculated as an evaluation standard of model performance:

[0083]

[0084] wherein, RMSE g_P is the root mean square error of the time random forest model set trained by the gth feature ranking subset S g_P , 1≤g≤N-1, y is the actual response variable value, is the response variable value predicted by the ath time random forest model.

[0085] The root mean square errors RMSE N-1_P RMSE N-2_P ..., RMSE 1_P of the time random forest model set trained by N-1 feature ranking subsets under the time training mode are obtained, and the feature ranking subset corresponding to the minimum root mean square error is selected as the screened time key feature subset S g_P .

[0086] S5: Feature importance evaluation and iterative screening based on spatial training

[0087] The model constructed by the random forest is spatially trained (K training), and the specific steps are as follows:

[0088] A river B monitoring station position is used as a spatial unit to divide the water quality feature data, and B spatial water quality feature data sets are obtained. Any one of the spatial water quality feature data sets is used as a test set to test the prediction ability of the random forest, and the remaining B-1 spatial water quality feature data sets are used for construction and training of the spatial random forest model. A total of B spatial random forest models are trained to form a spatial random forest model set. The input of the spatial random forest model is the water quality feature, and the output is the water quality index, i.e., the response variable.

[0089] Calculate the average reduction in precision and the reduction in node impurity for N features in the initial water quality feature set under B spatial random forest models.

[0090] The importance score of each water quality feature in the spatial random forest model set is obtained using the following formula and normalization method:

[0091]

[0092] In the formula: 1 < n < N, N is the number of features in the initial water quality feature set, 1 < b < B, FFIS n_K IM is the importance score of the nth feature in the spatial random forest model set. b IM represents the average reduction in accuracy for the b-th spatial random forest model. B_min and IM B_max These represent the minimum and maximum values ​​of the average reduction in accuracy among B spatial random forest training models, respectively. b IN represents the reduction in node impurity for the b-th spatial random forest model. B_min and IN B_max These are the minimum and maximum values ​​of the reduction in node impurity in B spatial random forest training models, respectively.

[0093] The N water quality features in the initial water quality feature set are ranked according to their importance scores to obtain the feature ranking set S. N_K ={I 1_K ,I 2_K ,I 3_K ,...,I N_K}, where In_ K The nth feature is ranked by importance score in the spatial training mode.

[0094] Remove the feature sorting set S N_K ={I 1_K ,I 2_K ,I 3_K ,...,I N_K The feature with the lowest importance score in} N _ K Obtain the feature-ranked subset S N-1_K ={I 1_K ,I 2_K ,I 3_K ,...,I N-1_K Then sort the feature subset S. N-1_K ={I 1_K ,I 2_K ,I 3_K ,...,I N-1_K} as a new feature ranking set and remove the feature I with the lowest importance score. N-1_K, obtain a new feature ranking subset S N-2_K = {I 1_K ,I 2_K ,I 3_K ,...,I N-2_K}, and so on, to obtain a plurality of feature ranking subsets S N-1_K ,S N-2_K ,...,S 1_K .

[0095] S N-1_K ,S N-2_K ,...,S 1_K are used to train the models in the spatial random forest model set respectively, and the root mean square error of the spatial random forest model set trained by each feature ranking subset is calculated:

[0096]

[0097] where RMSE h_K is the root mean square error of the spatial random forest model set trained by the hth feature ranking subset S h_K , 1≤h≤N-1, y is the actual response variable value, is the response variable value predicted by the bth spatial random forest model.

[0098] The root mean square errors RMSE N-1_K ,RMSE N-2_K ,...,RMSE 1_K of the spatial random forest model set trained by N-1 feature ranking subsets under the spatial training mode are obtained, and the feature ranking subset corresponding to the minimum root mean square error is selected as the screened spatial key feature subset S h_K .

[0099] S6: Dynamic optimization of feature selection and verification of robustness of prediction model

[0100] The time key feature subset S g_P obtained in S4 is sequentially excluded from the least important features to obtain a new feature subset S g-1_P ,S g-2_P ,...,S 1_P , and the corresponding RMSE g-1_P ,RMSE g-2_P ,...,RMSE 1_P are sequentially calculated to test whether the key feature subset S g_P exists overfitting and the contribution of the selected features to the prediction ability of the model, to ensure that these features indeed play a key role in spatial and temporal extrapolation, and to obtain the final prediction error of the selected feature subset.

[0101] Similarly, for the spatial key feature subset S obtained in S5 h_K The feature subset is also tested by the above method and the final prediction error is obtained.

[0102] S7: Prediction model optimization based on feature reduction and accuracy optimization

[0103] The time key feature subset S in the time training mode g_P And its corresponding root mean square error RMSE g_P And the spatial key feature subset S in the spatial training mode h_K And its corresponding root mean square error RMSE h_K The principle of using fewer water quality features to achieve higher prediction accuracy is adopted to select the training model. Specifically:

[0104] Determine the key feature subset S in the time training mode g_P The size of the number of features g and the number of features h in the feature subset S h_K The size of RMSE g_P And RMSE h_K The judgment process is as follows:

[0105] When g≤h and RMSE g_P < RMSE h_K , S g_P is the optimal river water quality feature set, and the time training mode is the best training mode.

[0106] When g≥h and RMSE g_P > RMSE h_K , S h_K is the optimal river water quality feature set, and the spatial training mode is the best training mode.

[0107] When g g_P > RMSE h_K , calculate the root mean square error RMSE g_K Corresponding to the key feature subset S g_K In the spatial training mode of the gth feature, then judge the size of RMSE g_K And RMSE g_P If RMSE g_K > RMSE g_P , S g_P is the optimal river water quality feature set, and the time training mode is the best training mode. If RMSE g_K < RMSE g_P , S h_K is the optimal river water quality feature set, and the spatial training mode is the best training mode.

[0108] When g > h and RMSE g_P < RMSE h_K , the key feature subset S h_P of the hth feature in the time training mode is calculated h_P , and the size of RMSE h_P and RMSE h_K is judged, if RMSE h_P < RMSE h_K , S g_P is taken as the optimal river water quality feature set, and the time training mode is the best training mode. If RMSE h_P > RMSE h_K , S h_K is taken as the optimal river water quality feature set, and the spatial training mode is the best training mode.

[0109] S8: Water quality prediction

[0110] In river network water quality monitoring, due to the existence of data missing in some period or place, in order to fill the information gap of these missing points, river water quality prediction needs to be predicted, including:

[0111] The optimal river water quality feature set corresponds to the best training mode, and the optimal random forest model set is trained by using the optimal river water quality feature set in the best training mode. The optimal random forest model set is used to predict the river water quality index of the target area to obtain a prediction result set, and the average value of all prediction results in the prediction result set is taken as the final prediction result of the river water quality of the target area. Specifically:

[0112] When the best training mode is the time training mode, A time random forest models are used to predict the water quality index of the missing point respectively to obtain A prediction results T 1_P , T 2_P ,..., T A_P . Then the average value of the A prediction results is taken as the water quality index T P of the missing point:

[0113]

[0114] When the best training mode is the spatial training mode, B spatial random forest models are used to predict the water quality index of the missing point respectively to obtain B prediction results T 1_K , T 2_K ,..., T B_K . Then the average value of the B prediction results is taken as the water quality index T K of the missing point:

[0115]

[0116] Through this method, the water quality data of the entire river network can be effectively inferred and predicted, thereby obtaining a comprehensive water quality data set.

[0117] The present embodiment is based on two dimensions of time and space to use random forest (RF) for two training settings, time training and space training, which can better capture the complexity of river network in space and time and improve the prediction robustness of the model for missing data or unobserved positions. The importance of water quality characteristics is obtained by using multiple feature importance evaluation methods, and the least important features are excluded in turn to obtain a new feature set and evaluate the prediction error of the new set. Through feature set iteration and refinement, the best feature set under the two training settings is obtained, and according to the principle of less features and high precision, the best extrapolation model is selected to perform spatio-temporal extrapolation of river water quality, providing a theoretical basis for pollution control and comprehensive management of rivers.

[0118] Specific implementation method two: reference Figures 2 to 6 To specifically explain the present embodiment, the present embodiment takes the COD concentration of a river in A province and B city of the Yellow River Basin as an example for explanation, and the specific implementation process is as follows:

[0119] (1) Divide the study basin using DEM

[0120] Collect the digital elevation model (DEM) data of the target area, identify the high points and low points of the terrain of the target area by analyzing the DEM data. Based on the high points and low points of the terrain of the target area, determine the water flow direction of the target area using a water flow direction algorithm, and calculate the water flow accumulation. Identify the water flow path and extract the river network using the water flow direction and water flow accumulation of the target area. On this basis, according to the water flow direction and the topographic features of the target area, use a watershed division algorithm to determine the watershed boundary, and analyze the properties of the watershed to form a contour map of the catchment area. Finally, compare with existing maps or field survey data to verify and correct the extracted river network and watershed, and ensure its accuracy.

[0121] (2) Data cleaning and construction of data set

[0122] Based on statistical yearbook, environmental bulletin and open data platform, the multiple factors affecting the chemical oxygen demand (COD) of the river were analyzed. These factors not only include natural conditions such as meteorological and climatic characteristics, but also cover the impact of human activities, such as land use types and pollution generated by human activities, building a comprehensive water quality feature system covering cultivated land (TDA), forest land (TDB), grassland (TDC), water area (TDD), urban and rural industrial and residential land (TDE), urban permanent population (CP), rural permanent population (RP), gross domestic product (GDP), highest temperature (HT), lowest temperature (LT), rainfall (R), wind speed (WS), wind direction (WD), monitoring site number (S), month (M), date (D), water temperature (T), turbidity (NTU) and pH.

[0123] Due to the existence of ice sealing period in the river, there is a lack of water quality data from November to April of the next year, so the data of the river from May to October in 2023 for 6 months at the hourly scale are selected, and the daily scale average of these data is obtained. The missing values are identified by heat map, and the missing values are filled by Pandas interpolation method; then, the Winsorizing technology is used to effectively eliminate the outliers in the data, ensuring the accuracy and reliability of the data set. After the above processing, the data set suitable for subsequent modeling is obtained.

[0124] (3) Water quality feature preliminary screening based on Pearson coefficient

[0125] The Pearson coefficient method is used to calculate the correlation coefficient between the response variable COD and the preliminary selected feature system, including cultivated land TDA, forest land TDB, grassland TDC, water area TDD, urban and rural industrial and residential land TDE, urban permanent population CP, rural permanent population RP, gross domestic product GDP, highest temperature HT, lowest temperature LT, rainfall R, wind speed WS, wind direction WD, site number S, month M, date D, water temperature T, turbidity NTU and pH. By calculating the Pearson correlation coefficient of these features and COD, the water quality features with Pearson correlation coefficient greater than 0.3 are selected as the initial water quality features for subsequent model training.

[0126] Table 1 Pearson correlation coefficient of COD and screened features

[0127]

[0128] (4) Feature importance evaluation and feature iteration screening based on time training

[0129] The random forest is used for time training of the model, five months of data of all stations are selected for training of the model, and the remaining one month of data is used for detection of the model. In order to ensure that the data of all months can be used for training and detection, 6 random forest models are trained, and IncMSE and IncNodePurity two characteristic importance evaluation methods are used to calculate the importance scores of the characteristic variables of the 6 models respectively. The importance scores of the 11 characteristic variables are obtained by using the normalization method a :

[0130] Table 2 Importance score order table of characteristic variables in time training setting

[0131]

[0132] The water quality data of the excluded months is used as the test set for model verification. The mean of the RMSE of the 6 models is used as the evaluation index of the selected feature set. The least important features are excluded to form a new feature subset, and the root mean square error RMSE of the new subset is calculated. Through iterative reduction, the subset that can minimize the RMSE is obtained as the selected feature set, Figure 2 is the root mean square error analysis diagram under different feature combinations. Each circle contains the root mean square error of its feature subset with all feature combinations on the left side. The red circle represents the feature set that minimizes the root mean square error of the test set. According to Figure 2 it can be seen that under the time training setting, the feature set that minimizes the RMSE of the test set of the model is pH, T, RP, TDC, HT, LT and TDE.

[0133] (5) Feature importance evaluation and iterative screening based on spatial training

[0134] The random forest is used for spatial training of the model, 6 months of data of three stations are selected for training of the model, and the data of the remaining one station is used for detection of the model. In order to ensure that the data of all stations can be used for training and detection, 4 random forest models are trained, and IncMSE and IncNodePurity are used to calculate the importance scores of the characteristic variables respectively. The importance scores of the 11 characteristic variables are obtained by using the normalization method b :

[0135] Table 3 Importance score order table of characteristic variables in spatial training setting

[0136]

[0137] The water quality data of the excluded sites is selected as the test set for model verification. The average RMSE of the four models is used as an evaluation index to evaluate the importance of the selected feature set. The least important features are gradually excluded to form a new feature subset, and the RMSE of the new subset is calculated. Through iterative reduction, the subset that can minimize the RMSE is obtained, which is the required feature set, as shown in Figure 3 Figure 3 It can be seen that under the spatial training setting, the feature set that minimizes the RMSE of the test set of the model is LT, pH, NTU, TDA, PR, TDC, TDE, and M.

[0138] (6) Dynamic optimization of feature selection and robustness verification of prediction model

[0139] The feature subset selected in the time training setting is retrained in the six random forest models to obtain the average RMSE of the excluded months of the six models. The least important features are gradually excluded to obtain a new feature set, and the RMSE of the new set is calculated. The feature subset selected in step 4 is tested to determine whether it is the optimal set, and the final prediction error is obtained. Figure 4 The RMSE obtained by retraining the feature subset selected in the time training. Through Figure 4 It can be seen that the seven features pH, T, RP, TDC, HT, LT, and TDE selected in step (4) are the optimal feature set R j The RMSE at this time is j = 5.94.

[0140] The feature subset selected in the spatial training setting is retrained in the four random forest models to obtain the average RMSE of the excluded sites of the four models. The least important features are gradually excluded to obtain a new feature set, and the RMSE of the new set is calculated. The feature subset selected in step 5 is tested to determine whether it is the optimal set, and the final prediction error is obtained. Figure 5 The RMSE obtained by retraining the feature subset selected in the spatial training. Through Figure 5 It can be seen that the eight features LT, pH, NTU, TDA, PR, TDC, TDE, and M selected in step (5) are the optimal feature set R m The RMSE at this time is m = 8.65.

[0141] (7) Optimal prediction model based on feature reduction and accuracy optimization

[0142] As shown in step (5), the number of features in the key feature subset R j is 7, and the final prediction error RMSE j ​=5.94, key feature subset R in spatial training mode m The number of features is 8, and its final prediction error RMSE m = 8.65, because R j <R m And RMSE j <RMSE m Therefore, six random forest models in the time-trained mode were selected as the final prediction models.

[0143] (8) River network water quality data prediction and filling based on the optimal model

[0144] Six random forest models trained under time-based conditions were used to predict and fill in the COD (Chemical Oxygen Demand) at station 5 of this river (the water quality data at this station was neither used for model training nor for model testing) for 31 days from September 15, 2023 to October 15, 2023. The predicted results and the measured values ​​are shown below. Figure 6 As shown, by Figure 6 It can be seen that this implementation method has a good predictive effect on the simulation of water quality indicators in unmonitored areas, and can further utilize limited data to extrapolate and predict the spatiotemporal water quality indicators of the entire river network.

[0145] While the invention has been described herein with reference to specific embodiments, it should be understood that these embodiments are merely examples of the principles and applications of the invention. Therefore, it should be understood that many modifications can be made to the exemplary embodiments, and other arrangements can be designed without departing from the spirit and scope of the invention as defined by the appended claims. It should be understood that different dependent claims and features described herein can be combined in ways different from those described in the original claims. It is also understood that features described in conjunction with individual embodiments can be used in other described embodiments.

Claims

1. A spatiotemporal adaptive prediction method for river water quality based on feature enhancement, characterized in that, include: The water quality characteristic data of any river are divided into temporal and spatial water quality characteristic datasets, respectively, using time and space as the step units. Using the aforementioned time-based water quality feature dataset and spatial water quality feature dataset, a time-based random forest model set and a spatial random forest model set under the time-based training mode and the spatial training mode, respectively, are constructed and trained. The input of both the time-based random forest model and the spatial random forest model is water quality feature data, and the output is water quality index, which is the response variable of water quality. Based on two random forest model sets, the features in the initial water quality feature set are initially screened to obtain a temporal key feature subset under the time training mode and a spatial key feature subset under the spatial training mode. The initial screening method is the same for both random forest model sets, and the initial screening includes: Calculate the importance score of each feature in the initial water quality feature set under the current random forest model set. Re-rank the features in the initial water quality feature set according to their importance scores to obtain a feature ranking set. Remove the feature with the lowest importance score from the feature ranking set to obtain a feature ranking subset. Then, use this feature ranking subset as a new feature ranking set and remove the feature with the lowest importance score from it to obtain a new feature ranking subset. Repeat this process to obtain multiple feature ranking subsets. Use multiple feature ranking subsets to train each random forest model in the current random forest model set. Calculate the root mean square error of the current random forest model set after training with each feature ranking subset. The feature ranking subset corresponding to the minimum root mean square error is taken as the key feature subset in the current training mode. The features are finely screened based on the number of features in the key feature subsets under the two training modes and the corresponding root mean square error. The fine screening includes: Let S g_P and S h_K These are the key feature subsets under two training modes, where P and K represent the two training modes respectively, and g and h represent the key feature subsets under two training modes respectively. g_P and S h_K Number of features in, RMSE g_P and RMSE h_K They are respectively via S g_P and S h_K The root mean square error of the trained random forest model set is as follows: When g≤h and RMSE g_P <RMSE h_K At that time, S g_P As the optimal set of river water quality characteristics; When g≥h and RMSE g_P >RMSE h_K At that time, S h_K As the optimal set of river water quality characteristics; When g≤h and RMSE g_P >RMSE h_K When calculating the g-th ranked feature subset S under training mode K, g_K The corresponding root mean square error (RMSE) g_K Then determine RMSE. g_K With RMSE g_P The size, if RMSE g_K >RMSE g_P Then S g_P As the optimal set of river water quality characteristics, if RMSE g_K <RMSE g_P Then S h_K As the optimal set of river water quality characteristics; When g≥h and RMSE g_P <RMSE h_K When calculating the h-th feature ranking subset S under training mode P, h_P The corresponding root mean square error (RMSE) h_P Then determine RMSE. h_P With RMSE h_K The size, if RMSE h_P <RMSE h_K Then S g_P As the optimal set of river water quality characteristics, if RMSE h_P >RMSE h_K Then S h_K As the optimal set of river water quality characteristics; The training mode corresponding to the optimal river water quality feature set is taken as the best training mode, and the optimal random forest model set is trained using the optimal river water quality feature set under the best training mode. The optimal random forest model set is used to predict the river water quality indicators in the target area, and the prediction result set is obtained. The average value of all prediction results in the prediction result set is taken as the final prediction result of the river water quality in the target area.

2. The spatiotemporal adaptive prediction method for river water quality based on feature enhancement according to claim 1, characterized in that, The method for obtaining the initial water quality feature set includes: Digital elevation model data of the target area is collected to obtain the high and low points of the target area. The water flow direction and water flow accumulation of the target area are determined by combining the high and low points of the target area with the water flow direction algorithm. The river network of the target area is extracted based on the water flow direction and water flow accumulation. The watershed boundary of the river network is determined by the watershed division algorithm based on the water flow direction and the topographic features of the target area, forming a water catchment area outline map. Using each region in the water catchment area outline map as a unit, collect water quality characteristic data of the target area, remove invalid and erroneous water quality characteristic data, interpolate or complete missing water quality characteristic data, and smooth or correct abnormal water quality characteristic data to obtain a coarse water quality characteristic dataset. A Pearson correlation coefficient matrix R is constructed using water quality features and water quality response variables from a coarse water quality feature dataset. The elements of R are... ij Let M represent the Pearson correlation coefficient between the i-th water quality feature and the j-th response variable, 1≤i≤M, 1≤j≤M′, where M is the number of water quality features in the coarse water quality feature dataset, and M′ is the number of response variables. Remove the features corresponding to elements in R that are less than a preset coefficient threshold, and the remaining water quality features in the coarse water quality feature dataset constitute the initial water quality feature set.

3. A spatiotemporal adaptive prediction method for river water quality based on feature enhancement according to claim 1 or 2, characterized in that, In the time-based training mode, the water quality characteristic data of any river is divided into A time-based water quality characteristic datasets with A time step units; A-1 time-based water quality characteristic datasets are arbitrarily selected to train A time-based random forest models, and the A time-based random forest models constitute the time-based random forest model set in the time-based training mode.

4. The spatiotemporal adaptive prediction method for river water quality based on feature enhancement according to claim 3, characterized in that, A temporal random forest model set is used to initially screen features from the initial water quality feature set, obtaining a subset of key temporal features under the time training mode, including: Calculate the average reduction in precision and the reduction in node impurity for each feature in the initial water quality feature set under the random forest model at each time point; The importance score of each water quality feature under the temporal random forest model set was calculated using a normalization method. The features in the initial water quality feature set are reordered according to their importance scores to obtain the feature sorting set S. N_P ={I 1_P ,I 2_P ,I 3_P ,...,I N_P }, where I n_P Let S be the feature ranked nth in importance score under the time-based training model. N-1_P ,S N-2_P ,...,S 1_P ; Use S N-1_P ,S N-2_P ,...,S 1_P The models in the Temporal Random Forest training model set are trained separately, and the root mean square error of the Temporal Random Forest model set after training with each feature sorting subset is calculated. The feature sorting subset corresponding to the minimum root mean square error is used as the time key feature subset in the time training mode.

5. A spatiotemporal adaptive prediction method for river water quality based on feature enhancement according to claim 1 or 2, characterized in that, In the spatial training mode, the water quality characteristic data of any river is divided into B spatial water quality characteristic datasets using B monitoring locations as spatial units. B-1 spatial water quality characteristic datasets are then arbitrarily selected to train B spatial random forest models. These B spatial random forest models constitute the spatial random forest model set in the spatial training mode.

6. The spatiotemporal adaptive prediction method for river water quality based on feature enhancement according to claim 5, characterized in that, A spatial random forest model set is used to initially screen features in the initial water quality feature set, obtaining a subset of key spatial features under the spatial training mode, including: Calculate the average reduction in precision and the reduction in node impurity for each feature in the initial water quality feature set under each spatial random forest model; The importance score of each water quality feature under the spatial random forest model set was calculated using a normalization method. The features in the initial water quality feature set are reordered according to their importance scores to obtain the feature sorting set S. N_K ={I 1_K ,I 2_K ,I 3_K ,...,I N_K }, where I n_K Let S be the feature ranked nth in importance score under the spatial training mode. Then the feature ranking subset is represented as S. N-1_K ,S N-2_K ,...,S 1_K ; Use S N-1_K ,S N-2_K ,...,S 1_K The models in the spatial random forest training model set are trained separately, and the root mean square error of the spatial random forest model set after training with each feature sorting subset is calculated. The feature sorting subset corresponding to the minimum root mean square error is used as the spatial key feature subset in the spatial training mode.

7. A spatiotemporal adaptive prediction method for river water quality based on feature enhancement according to claim 1, 4, or 6, characterized in that, The importance score of each water quality feature in the current random forest model set is calculated using a normalization method, as shown in the following expression: In the formula: 1 < n < N, where N is the number of features in the initial water quality feature set; 1 < q < Q, where Q is the number of models contained in the current random forest model set; FFIS n IM is the importance score of the nth feature in the current random forest model set. q IM represents the average reduction in accuracy for the q-th random forest model. Q_min and IM Q_max These are the minimum and maximum values ​​of the average decrease in accuracy among the Q random forest training models, respectively, IN q IN represents the reduction in node impurity for the q-th random forest model. Q_min and IN Q_max These are the minimum and maximum values ​​of the reduction in node impurity in the Q random forest training models, respectively.

8. The spatiotemporal adaptive prediction method for river water quality based on feature enhancement according to claim 7, characterized in that, The root mean square error of the current random forest model set after training with each feature-ranked subset is calculated as follows: Among them, RMSE f Let y be the root mean square error of the current spatial random forest model set trained on the f-th feature sorted subset, 1 ≤ f ≤ N-1, and y be the actual response variable value. It is the value of the response variable predicted by the q-th random forest model.

9. The spatiotemporal adaptive prediction method for river water quality based on feature enhancement according to claim 1, characterized in that, When the optimal training mode is time-based training, the final prediction result for river water quality is: When the optimal training mode is spatial training mode, the final prediction result for river water quality is: T 1_P ,T 2_P ,...,T A_P T represents the prediction result obtained from training with the optimal training mode. P The final prediction result of river water quality obtained using the time-based training model; T 1_P ,T 2_P ,...,T A_P T represents the prediction result obtained from training with the optimal training mode. K This is the final prediction result of river water quality obtained using the spatial training model.

10. The spatiotemporal adaptive prediction method for river water quality based on feature enhancement according to claim 1, characterized in that, After obtaining the key feature subset, the key feature subset is tested, and if it passes the test, it is then further screened.

Images