Analysis method of factors affecting groundwater hardness based on random forest and SHAP

Abstract

The present invention discloses a method for analyzing factors affecting groundwater hardness based on random forest and SHAP, collects relevant data on total groundwater hardness in a study area and a study period, and constructs a data set of total groundwater hardness; constructs a data set of predictive variables at the same time; uses the predictive variables as model input, trains and optimizes a random forest model, and thereby constructs a mapping relationship between the predictive variables and the total groundwater hardness through the model; based on the optimized random forest model, predicts the total groundwater hardness of all points in the study area, and obtains the temporal variation trend of the total groundwater hardness and the spatial distribution and variation characteristics; performs SHAP analysis on the trained random forest model to determine the importance ratio of each predictive variable. The present invention can realize the prediction of the total groundwater hardness in the entire study area and the study period, and can obtain the influence characteristics of each influencing factor on the total groundwater hardness.

Description

Technical Field

[0001] The invention belongs to the field of groundwater protection and pollution prevention and control, and specifically relates to a groundwater hardness influencing factor analysis method based on random forest and SHAP. Background Art

[0002] The total hardness of groundwater refers to the sum of the concentrations of calcium and magnesium ions in the water, which mainly comes from geological structures such as limestone and dolomite. Existing studies have confirmed that high-hardness groundwater is associated with a series of health problems such as childhood eczema and chronic kidney disease. In addition, when high-hardness groundwater is used for irrigation, it will accelerate the salinization process of the soil; when used in the industrial field, it is easy to form scale in pipes and equipment, which not only hinders the smooth flow of water, but also leads to an increase in energy consumption; high-hardness groundwater has become a global problem that cannot be ignored. Existing observation data reveal a phenomenon: areas with high-hardness groundwater are often distributed in arid or coastal areas; in arid areas, due to excessive groundwater exploitation, the hydrodynamic conditions and hydrogeochemical characteristics of the aquifer have changed, prompting more calcium and magnesium ions to dissolve into the groundwater system; the high-hardness groundwater in coastal areas is mainly attributed to the intrusion of seawater with high calcium and magnesium content. This process not only directly increases the total hardness level of groundwater, but also may accelerate the dissolution process of calcium and magnesium ions in the aquifer, thereby further exacerbating the increasing trend of total groundwater hardness.

[0003] The total hardness of groundwater is determined by the total amount of water entering and leaving the groundwater system and the total amount of calcium and magnesium ions. The above process will be jointly determined by geological soil parameters, climatic conditions, human activities, and vegetation cover, so it is difficult to determine the quantitative impact of each driving factor on the above process. For example, precipitation will carry calcium and magnesium ions into the groundwater system while replenishing groundwater. In addition to the total amount of precipitation and precipitation patterns, the total amount of water entering the groundwater system depends on local climatic conditions, geology and soil properties. The calcium and magnesium ions entering the groundwater system will depend on factors such as local lithology and soil cation exchange capacity. For the above complex processes, the impact of various factors on the total hardness of groundwater is difficult to characterize. At present, the limited total groundwater hardness observation data are concentrated in areas where the total groundwater hardness seriously exceeds the standard, which means that many areas will be exposed to the risk of high groundwater hardness. There is no mature technology and method to effectively identify areas with high groundwater hardness. In addition, the limited total groundwater hardness observation data poses a huge challenge to the analysis of its driving factors. Although some studies have explored the main driving factors of high-hardness groundwater areas, their research still remains in local areas and uses different research methods. Therefore, there is still a lack of systematic research on the driving factors of the spatiotemporal dynamic changes of total groundwater hardness in large-scale research areas. Developing a spatiotemporal prediction method for total groundwater hardness and analysis of its driving mechanism has become an urgent problem to be solved. Summary of the invention

[0004] Purpose of the invention: In order to obtain the spatiotemporal evolution law and driving mechanism of total groundwater hardness, the present invention provides a method for analyzing groundwater hardness influencing factors based on random forest and SHAP, which provides scientific technology and methods for early warning of high groundwater hardness risks and analysis of the driving mechanism of total groundwater hardness in large-scale study areas.

[0005] Invention content: The method for analyzing factors affecting groundwater hardness based on random forest and SHAP of the present invention comprises the following steps:

[0006] (1) Obtain information on total groundwater hardness and land use type of the sampling area in the study area and study period, and construct a dataset of total groundwater hardness at the target resolution;

[0007] (2) Obtain multiple indicator data sets related to total groundwater hardness, select the indicator most relevant to total groundwater hardness as the predictor variable, and construct a predictor variable data set at the target resolution;

[0008] (3) The mapping relationship between the prediction variables and the total groundwater hardness was constructed through the random forest model. The prediction variables were used as the input of the random forest model, and the total groundwater hardness was used as the model output. The random forest model was trained and optimized.

[0009] (4) Based on the optimized random forest model, the total groundwater hardness of all points in the study area was predicted; the temporal trend of the total groundwater hardness and the spatial distribution and variation characteristics were obtained through the prediction results;

[0010] (5) SHAP analysis was performed on the trained random forest model to determine the importance of each predictor variable and reveal the specific impact characteristics of the predictor variable on the total groundwater hardness.

[0011] Furthermore, the implementation process of step (1) is as follows:

[0012] Obtain the total groundwater hardness data of the area to be studied. While recording the total groundwater hardness data, record the corresponding sampling time, sampling location and local land use type; convert all collected data into a mass concentration equivalent to calcium carbonate, and perform data resolution conversion on this basis to obtain a total groundwater hardness data set covering some points in the study area at the target resolution.

[0013] Furthermore, the implementation process of step (2) is as follows:

[0014] Obtain relevant indicators that can cover the relevant climate, geological soil, human activities, and vegetation conditions of the entire study area; select the indicators most relevant to the total groundwater hardness as prediction variables, and convert the data resolution to obtain a total groundwater hardness prediction variable data set at the target resolution covering the entire study area and study period.

[0015] Furthermore, the implementation process of step (3) is as follows:

[0016] Divide the data in the predictor variable dataset and the total groundwater hardness dataset into a training set With the test set ; In the test set, there are samples, each with predictor variables and an output value of total groundwater hardness; the training data is represented as ,in, , Indicates The predictor vector for samples, Indicates Total groundwater hardness output value of samples;

[0017] Building a decision tree: randomly draw from the training set with replacement samples to form a training set To build each decision tree; for each node split, from Randomly select from indivual, , forming a set of candidate predictor variables ; Then, select the optimal predictor variable for node division according to the splitting criterion; repeat the process times, get Decision Tree ;

[0018] Prediction process: For a new sample , which is input into each decision tree for prediction, and we get Predicted values ; The average of all decision tree predictions is used as the final prediction result of the random forest model ;

[0019] Selection of the number of decision trees: The random forest model is trained under specific values, and the number of decision trees of the random forest model is selected based on the fitting effect between the predicted results and the actual results under the test set. .

[0020] Furthermore, the implementation process of step (4) is as follows:

[0021] Based on the random forest model trained with a better number of decision trees, starting from the prediction variables covering the entire study area and the entire study period, the total groundwater hardness prediction covering the entire study area and the entire study period was obtained; based on the prediction results, the spatial distribution map of the total groundwater hardness was drawn to understand the spatial distribution characteristics of the total groundwater hardness in the study area, and by comparing the spatial distribution maps at different times, the dynamic change law of the spatial distribution of the total groundwater hardness in the study area was understood.

[0022] Furthermore, the implementation process of step (5) is as follows:

[0023] SHAP characterizes the impact of each predictor variable on the total groundwater hardness. The conditional SHAP algorithm calculates the marginal contribution of each predictor variable through the values ​​of other predictors, thereby more accurately capturing the interdependence between predictors to determine the impact pattern of each predictor variable on the total groundwater hardness. The formula for the conditional SHAP value is:

[0024] ;

[0025] in, is the predictor variable Contribution to the prediction of total groundwater hardness, is the set of all predictor variables, Does not include predictor variables A subset of Only use The model predictions for the predictor variables in , All other predictor variables The value of is the conditional expectation.

[0026] Furthermore, the splitting criterion is to minimize the mean square error between the predicted value and the true value of the decision tree:

[0027] ;

[0028] in, is the true value, is the predicted value, is the number of samples.

[0029] Beneficial effects: Compared with the prior art, the beneficial effects of the present invention are as follows: By constructing a random forest model of total groundwater hardness and other related indicators, the present invention can predict the total groundwater hardness in the entire study area and study period, providing a basis for clarifying the distribution and change characteristics of the total groundwater hardness; further, by implementing SHAP analysis on the established random forest model, the influence characteristics of each influencing factor on the total groundwater hardness can be obtained, providing a scientific basis for groundwater management and control. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 is a flow chart of the present invention;

[0031] Figure 2 Optimize the selection graph for the number of decision trees for machine learning models; Figure 2 (a) Comparison of the effects of setting the random forest model training set and test set for different decision trees; Figure 2 (b) is a graph showing the effect of random forest modeling based on a better number of decision trees;

[0032] Figure 3 This is the prediction diagram of the total groundwater hardness in different years by the random forest model;

[0033] Figure 4 This is a ranking chart of the importance of each predictor variable in the random forest model;

[0034] Figure 5 The influence characteristics of each predictor variable on the total hardness of groundwater when predicting by the random forest model. DETAILED DESCRIPTION

[0035] The technical solution of the invention is described in detail below with reference to the accompanying drawings.

[0036] The present invention provides a method for analyzing factors affecting groundwater hardness based on random forest and SHAP. Figure 1 As shown, the following steps are included:

[0037] Step 1: Obtain information on total groundwater hardness and land use type of the sampling area in the study area and study period, convert the data into mass concentration of calcium carbonate, and construct a dataset of total groundwater hardness at the target resolution.

[0038] Obtain the total groundwater hardness data of the area to be studied, and record the sampling time, sampling location and local land use type corresponding to the data while recording the total groundwater hardness data. Convert all collected data into a mass concentration equivalent to calcium carbonate, and perform data resolution conversion on this basis to obtain a total groundwater hardness data set covering some points in the study area at the target resolution. .

[0039] Step 2: Obtain multiple indicator data sets related to total groundwater hardness, which cover the entire study area and study period. Consider four factors, namely climate, geological soil, human activities, and vegetation coverage. Select the most relevant indicator to total groundwater hardness from these indicator data sets as the predictor variable, and construct a comprehensive predictor variable data set at the target resolution. .

[0040] Obtain relevant indicators that can cover the relevant climate, geological soil, human activities, and vegetation conditions of the entire study area; select the indicators most relevant to the total groundwater hardness as prediction variables, and convert the data resolution to obtain a total groundwater hardness prediction variable data set at the target resolution covering the entire study area and study period.

[0041] Step 3: Construct the mapping relationship between the prediction variables and the total groundwater hardness through the random forest model (RF), use the prediction variables as the input of the random forest model, and the total groundwater hardness as the model output, and train and optimize the random forest model.

[0042] Divide the data from the predictor variable dataset and the total groundwater hardness dataset into a training set for the machine learning model With the test set ; If there is samples, each with predictor variables and an output value of total groundwater hardness; the training data is represented as ,in, , Indicates The predictor vector for samples, Indicates Output value of total groundwater hardness for each sample.

[0043] Building a decision tree: randomly draw from the training set with replacement samples to form a training set To build each decision tree; for each node split, from Randomly select from indivual( ), forming a set of candidate predictor variables .

[0044] Then, according to the splitting criterion (mean square error , is the true value, is the predicted value) select the optimal predictor variable for node division; repeat the process times, get Decision Tree .

[0045] Prediction process: For a new sample , which is input into each decision tree for prediction, and we get Predicted values ; The average of all decision tree predictions is used as the final prediction result of the random forest model .

[0046] Selection of the number of decision trees: The random forest model is trained under specific values, and the number of decision trees of the random forest model is selected based on the fitting effect between the predicted results and the actual results under the test set. .

[0047] Step 4: Based on the optimized random forest model, the total groundwater hardness of all points in the study area was predicted; through these prediction results, the temporal change trend of the total groundwater hardness and the spatial distribution and change characteristics can be obtained.

[0048] Based on the random forest model trained with a better number of decision trees, the total groundwater hardness prediction covering the entire study area and the entire study period is obtained from the prediction variables covering the entire study area and the entire study period. Based on the prediction results, the spatial distribution map of the total groundwater hardness is drawn to understand the spatial distribution characteristics of the total groundwater hardness in the study area. By comparing the spatial distribution maps at different times, the dynamic change law of the spatial distribution of the total groundwater hardness in the study area is understood.

[0049] Step 5: The trained random forest model was subjected to Shapley Additive Explanations (SHAP) analysis. Through SHAP analysis, the importance of each predictor variable was determined and their specific impact characteristics on the total hardness of groundwater were revealed.

[0050] The conditional Shapley additive explanatory model characterizes the impact of each predictor variable on the total groundwater hardness. The conditional SHAP algorithm calculates the marginal contribution of each predictor variable through the values ​​of other predictors, thereby more accurately capturing the interdependence between predictors to determine the impact pattern of each predictor variable on the total groundwater hardness. The formula for the conditional SHAP value is:

[0051] ;

[0052] in, is the predictor variable Contribution to the prediction of total groundwater hardness, is the set of all predictor variables, Does not include predictor variables A subset of Only use The model predictions for the predictor variables in , All other predictor variables The value of is the conditional expectation.

[0053] In this implementation, the total groundwater hardness data in China from 1980 to 2020 are widely collected. These data are usually sparse in time and space distribution. During the collection process, the time, location and local land use type of data sampling are recorded in detail. In order to unify the data standards, all data are converted into the mass concentration of calcium carbonate (in mg / L). These data are downscaled. For the measured data of total groundwater hardness that can only be located in the county, 1% of the 1km2 samples in the county are randomly selected. 2 The grid point assignment is the measured data value of the total groundwater hardness. This forms a sparsely distributed 1km grid in China. 2 A groundwater total hardness dataset with high resolution.

[0054] Collect multiple indicator data sets related to total groundwater hardness in China from 1980 to 2100. These data sets mainly include geological soil factors such as soil parent material, soil and sediment thickness, cation exchange capacity; climate factors such as average annual temperature and average annual precipitation; human activity factors such as groundwater supply, industrial wastewater discharge, and fertilizer use; and vegetation coverage factors such as normalized difference vegetation index. For geological soil factors and vegetation coverage factors, set them as static variables, that is, they remain unchanged from 1980 to 2100; for climate factors, the data from 1980 to 2020 can be obtained from measured data, and the data from 2021 to 2100 can be obtained through model predictions from existing studies; for human activity factors, the data from 1980 to 2020 can be obtained from measured data, and the data from 2021 to 2100 are set to be the same as in 2020. Select 28 indicator data with strong correlation with total groundwater hardness and locate them to 1km 2 A 1km grid is constructed on the 2 It is worth noting that for dynamic variables such as climate factors and human activity factors, in order to consider their historical impact, the cumulative average value from 1980 to the target year is used as the final prediction variable.

[0055] The Chinese groundwater total hardness dataset includes 36,217 1 km 2 The China groundwater total hardness prediction variable dataset contains 28 prediction variables, each of which covers all 1km in China. 2 The data in the prediction variable dataset and the groundwater total hardness dataset are randomly divided into the training set and test set of the machine learning model at a ratio of 90% and 10%. Therefore, the test set includes 32,595 samples, each of which has 28 prediction variables and an output value of the total groundwater hardness. The training data can be represented as ,in , Indicates The predictor vector for samples, Indicates Output value of total groundwater hardness for each sample.

[0056] Random sampling, randomly extracting from the training set with replacement samples to form a training set To build each decision tree, we randomly select from the 28 predictors for each node split. indivual( ), forming a set of candidate predictor variables Then, the optimal predictor variable is selected for node partitioning according to the splitting criterion. The process is repeated times, get Decision Tree .

[0057] For a new sample , which is input into each decision tree for prediction, and we get Predicted values The average of all decision tree predictions is used as the final prediction result of the random forest model.

[0058] In the training set, in different decision trees The random forest model is trained under specific values, and the fitting effect between the predicted results and the actual results under the test set is used, such as Figure 2 As shown in (a), the test set Finally, 128 random forest model decision trees were selected for the indicator selection. The test set R 2 is 0.75, such as Figure 2 As shown in (b), it illustrates the reliability of the model.

[0059] Based on the random forest model trained with 128 decision trees, the total groundwater hardness prediction in China from 1980 to 2100 was obtained based on the total groundwater hardness prediction variables in China from 1980 to 2100. Figure 3 As shown in Figure 1, the total hardness of groundwater in China shows a trend of continuous growth amid fluctuations. The distribution of total groundwater hardness in China in different years also shows significant regional differences. Specifically, in China, the total groundwater hardness in the northern region is significantly higher than that in the southern region. In addition, there is a certain degree of dynamic change in China's groundwater between years, but its spatial distribution pattern is relatively fixed.

[0060] Based on the trained random forest model, SHAP analysis was used to understand the driving factors of total groundwater hardness at various locations in China. Figure 4As shown in Figure 2, the factors that have the most significant impact on the total groundwater hardness in China belong to the category of climate factors. In general, the geological soil factor has the most prominent impact on the total groundwater hardness, while the vegetation cover factor has the least impact, which is attributed to the fact that it is represented by only one factor, the normalized vegetation index. However, the single impact of the vegetation cover factor cannot be ignored. Figure 5 The influence characteristics of different predictor variables on the total groundwater hardness are shown. Although the influence characteristics of different predictor variables on the total groundwater hardness are different, some predictor variables have similar influence characteristics.

[0061] Through the above analysis, the present invention has the ability to predict the total groundwater hardness with high resolution in the study area. In addition, the SHAP analysis of this method provides strong support for the analysis of the driving mechanism of the total groundwater hardness in the large-scale study area, and provides scientific methods and techniques for early warning of high groundwater hardness risks and formulation of control plans.

[0062] The above is only an implementation case of the present invention, and its description is relatively specific and detailed, but it cannot be understood as limiting the scope of the invention patent. It should be pointed out that for ordinary technicians in this field, several modifications and improvements can be made without departing from the concept of this application, which all belong to the protection scope of this application. Therefore, the protection scope of the patent of this application shall be based on the attached claims.

Claims

1. A method for analyzing factors affecting groundwater hardness based on random forest and SHAP, characterized in that: The following steps are involved: (1) Obtain information on total groundwater hardness and land use type of the sampling area in the study area and study period, and construct a dataset of total groundwater hardness at the target resolution; (2) Obtain multiple indicator data sets related to total groundwater hardness, select the indicator most relevant to total groundwater hardness as the predictor variable, and construct a predictor variable data set at the target resolution; (3) The mapping relationship between the prediction variables and the total groundwater hardness was constructed through the random forest model. The prediction variables were used as the input of the random forest model, and the total groundwater hardness was used as the model output. The random forest model was trained and optimized. (4) Based on the optimized random forest model, the total groundwater hardness of all points in the study area was predicted; the temporal trend of the total groundwater hardness and the spatial distribution and variation characteristics were obtained through the prediction results; (5) Perform SHAP analysis on the trained random forest model to determine the importance of each predictor variable and reveal the specific impact characteristics of the predictor variable on the total groundwater hardness; The implementation process of step (1) is as follows: Obtain the total groundwater hardness data of the area to be studied, and record the sampling time, sampling location and local land use type corresponding to the data while recording the total groundwater hardness data; All collected data were converted into the mass concentration equivalent to calcium carbonate, and the resolution of the data was converted on this basis to obtain a total groundwater hardness data set covering some points in the study area at the target resolution; The implementation process of step (4) is as follows: Based on the random forest model trained with a better number of decision trees, starting from the prediction variables covering the entire study area and the entire study period, the total groundwater hardness prediction covering the entire study area and the entire study period was obtained; based on the prediction results, the spatial distribution map of the total groundwater hardness was drawn to understand the spatial distribution characteristics of the total groundwater hardness in the study area, and by comparing the spatial distribution maps at different times, the dynamic change law of the spatial distribution of the total groundwater hardness in the study area was understood.

2. The method for analyzing the influencing factors of groundwater hardness based on random forest and SHAP according to claim 1 is characterized in that: The implementation process of step (2) is as follows: Obtain relevant indicators that can cover the relevant climate, geological soil, human activities, and vegetation conditions of the entire study area; select the indicators most relevant to the total groundwater hardness as prediction variables, and convert the data resolution to obtain a total groundwater hardness prediction variable data set at the target resolution covering the entire study area and study period.

3. The method for analyzing the influencing factors of groundwater hardness based on random forest and SHAP according to claim 1 is characterized in that: The implementation process of step (3) is as follows: The data in the predictor variable dataset and the total groundwater hardness dataset are divided into the training set {X train ,y train } and the test set {X test ,y test }; There are n train samples, each sample has m predictor variables and an output value of total groundwater hardness; the training data is represented as Where i = 1, 2, ..., n train , represents the predictor variable vector of the i-th sample, represents the total groundwater hardness output value of the i-th sample; Construct a decision tree: Randomly and with replacement draw k samples from the training set to form a training set to construct each decision tree; for the splitting of each node, randomly select p from m predictor variables, where p < m, to form a candidate set of predictor variables F = {f1, f2, …, f p}; then, select the optimal predictor variable according to the splitting criterion to perform node partitioning; repeat this process N times to obtain N decision trees {T1, T2, …, T N}; Prediction process: For a new sample Input it into each decision tree for prediction, and get N predicted values ​​{y1,y2,…y N }; The average of all decision tree predictions is used as the final prediction result of the random forest model Selection of the number of decision trees: Perform random forest model training under different specific values ​​of decision tree N, and select the optimal number of random forest model decision trees N based on the fitting effect between the predicted results and the actual results under the test set. t .

4. The method for analyzing the influencing factors of groundwater hardness based on random forest and SHAP according to claim 1 is characterized in that: The implementation process of step (5) is as follows: SHAP characterizes the influence of each predictor variable on the total groundwater hardness. The conditional SHAP algorithm calculates the marginal contribution of each predictor variable through the values ​​of other predictors, thereby more accurately capturing the interdependence between predictors to determine the influence pattern of each predictor variable on the total groundwater hardness. The formula for the conditional SHAP value is: Among them, φ i is the contribution of predictor variable i to the prediction of total groundwater hardness, N is the set of all predictor variables, S is the subset excluding predictor variable i, f(S) is the model prediction using only the predictor variables in S, E[·|x -i ] means that all other predictors (x -i ) is the conditional expectation.

5. The method for analyzing the influencing factors of groundwater hardness based on random forest and SHAP according to claim 3 is characterized in that: The splitting criterion is to minimize the mean square error between the predicted value and the true value of the decision tree: Among them, y j is the true value, is the predicted value, and k is the number of samples.

Images