A method and apparatus for predicting soil mineral-bound organic carbon based on random forest and environmental variables.

Abstract

This invention discloses a method for predicting soil mineral-bound organic carbon based on random forest and environmental variables. The method includes: obtaining the true soil mineral-bound organic carbon content and related environmental variable sets from multiple soil samples; filtering the environmental variable set using a recursive feature elimination algorithm to obtain important environmental variables; using the true soil mineral-bound organic carbon content and the corresponding filtered environmental variable set as a sample set, dividing the sample set into a modeling set and an independent validation set; training an initial random forest prediction model based on the modeling set to obtain the random forest prediction model; evaluating the prediction accuracy of the random forest prediction model using the coefficient of determination and root mean square error based on the independent validation set; and obtaining the final random forest prediction model when the prediction accuracy threshold is reached. This invention also discloses a soil mineral-bound organic carbon prediction device based on random forest and environmental variables.

Description

Technical Field

[0001] This invention relates to methods for predicting soil organic carbon components, and more particularly to a method and apparatus for predicting soil mineral-bound organic carbon based on random forests and environmental variables. Background Technology

[0002] The increase in atmospheric carbon dioxide concentration has had a significant impact on the Earth, making it imperative to develop effective strategies for capturing carbon dioxide from the atmosphere. Soil organic carbon pools account for two-thirds of the carbon pools in terrestrial ecosystems, storing more organic carbon than the combined atmospheric and vegetation carbon pools. Therefore, soil organic carbon sequestration is considered one of the natural climate solutions for capturing atmospheric carbon dioxide. Simultaneously, soil organic carbon sequestration also helps improve various soil functions and ecosystem services. Promoting and adopting land management practices aimed at increasing soil organic carbon requires a deep understanding of its formation, persistence, and functions.

[0003] Because soil organic carbon (SOC) is composed of organic compounds with varying microbial decomposition behaviors, we cannot predict its response to changes in the external environment based solely on its total amount. Therefore, we need to use soil SOC grouping methods to gain a deeper understanding of its complexity. Currently, physical grouping techniques for classifying soil SOC into particulate and mineral-bound forms are gaining increasing attention in the soil science community. Particulate and mineral-bound SOCs differ fundamentally in their formation, persistence, and function. Particulate SOCs are primarily composed of undecomposed, lightweight fragments from plants, with an average turnover period of less than 10 years, and their proportion of total soil SOC can range from a few percentage points to 50%. Mineral-bound SOCs constitute the majority of soil SOC, mainly composed of highly decomposed organic matter from microorganisms and plants, with an average turnover period ranging from 10 to 100 years.

[0004] Although particulate and mineral-bound organic carbon in soil are important for understanding the evolution of soil organic carbon, they are rarely recorded in soil databases, especially in large-scale studies. When soil databases lack data on soil organic carbon components, prediction is often made using soil transformation functions. Lee and Viscarra Rossel (2020) successfully predicted soil carbon component data by fitting a linear soil transformation function based on soil clay and organic carbon. The shortcomings of the above studies are: (1) soil clay and organic carbon content are not the only relevant environmental variables for soil carbon components; (2) the evolution of soil organic carbon is complex, and soil organic carbon components and environmental variables often exhibit nonlinear interactions, and the current linear regression model is seriously inadequate. Therefore, it is urgent to develop a soil mineral-bound organic carbon prediction model that combines richer environmental variables with a nonlinear soil transformation function.

[0005] Random forest is a tree-based machine learning method that can handle non-linear relationships well and avoid model overfitting. Summary of the Invention

[0006] This invention provides a method for predicting soil mineral-bound organic carbon based on random forest and environmental variables. This method can quickly and accurately predict the content of soil mineral-bound organic carbon.

[0007] A method for predicting soil mineral-bound organic carbon based on random forest and environmental variables includes:

[0008] (1) Obtain multiple soil samples, use the real soil mineral-bound organic carbon content of multiple soil samples as a soil dataset, and obtain a set of environmental variables based on the spatial location of soil samples through a soil database and a remote sensing data product library.

[0009] (2) Based on the set of environmental variables, multiple sets of environmental variables and the corresponding accuracy of random forest models are obtained through recursive feature elimination algorithm. The set of environmental variables corresponding to the highest accuracy of random forest models is taken as important environmental variables.

[0010] (3) The soil dataset and the corresponding set of screened environmental variables are used as the sample set. The sample set is divided into the modeling set and the independent validation set. The initial random forest prediction model is obtained. Based on the modeling set, the parameters of the initial random forest prediction model are optimized by ten-fold cross-validation to obtain the random forest prediction model. Based on the independent validation set, the prediction accuracy of the random forest prediction model is evaluated by the coefficient of determination and the root mean square error. When the prediction accuracy threshold is reached, the final random forest prediction model is obtained.

[0011] (4) When applying, the important environmental variables of the soil sample to be tested are input into the final random forest prediction model to obtain the predicted soil mineral-bound organic carbon.

[0012] The soil dataset comprises the actual soil mineral-bound organic carbon content of multiple soil samples, including:

[0013] Multiple soil samples covering farmland and forestland were collected based on land cover and classification maps. The true soil mineral-bound organic carbon of each sample was obtained through physicochemical analysis. The true soil mineral-bound organic carbon of multiple samples was used to construct a soil dataset.

[0014] Environmental variables are obtained based on the spatial location of soil samples through a soil database and a remote sensing data product library. The set of environmental variables includes soil variables, climate variables, biological variables, and topographic variables.

[0015] The soil variables include soil organic carbon, total nitrogen, total phosphorus, total potassium, cation exchange capacity, sand, silt, clay, silt + clay, pH, and soil rainfall erosion.

[0016] The climate variables include annual precipitation, annual mean temperature, potential evapotranspiration, and atmospheric nitrogen deposition.

[0017] The biological variables include the normalized vegetation index, the enhanced vegetation index, and net primary productivity;

[0018] The terrain variable is elevation.

[0019] The method for obtaining multiple sets of environmental variables and the corresponding random forest model accuracy based on the environmental variable set through a recursive feature elimination algorithm includes:

[0020] S1: Fit a first random forest model based on the set of environmental variables. Fit a second random forest model to the remaining environmental variables after removing a single environmental variable from the set of environmental variables. Obtain the accuracy of the first random forest model and the second random forest model respectively through cross-validation. Use the difference between the accuracy of the second random forest model and the accuracy of the first random forest model as the importance of the corresponding single environmental variable removed. The larger the difference, the higher the importance. Remove single environmental variables in turn until the importance of each environmental variable in the set of environmental variables is obtained.

[0021] S2: Remove the single environmental variable with the lowest importance to obtain the combination of environmental variables. Fit the combination of environmental variables to the third random forest model. After removing the single environmental variable from the combination of environmental variables, fit the remaining environmental variables to the fourth random forest model. Again, use the cross-validation method to obtain the accuracy of the third and fourth random forest models respectively. Calculate the difference between the accuracy of the fourth random forest model and the accuracy of the third random forest model to obtain the corresponding importance of the removed single environmental variable. Continue to remove single environmental variables until each environmental variable in the combination of environmental variables has obtained an importance.

[0022] S3: Repeat step S2 until the environmental variables are combined into a single environmental variable and then stop repeating, thereby obtaining the accuracy of multiple random forest models fitted by multiple sets of environmental variables.

[0023] The accuracy evaluation metric for the random forest model is the root mean square error (RMSE), which is:

[0024]

[0025] Where n is the number of samples, y i Let represent the measured mineral-bound organic carbon content of the i-th sample. Let be the predicted mineral-bound organic carbon content of the i-th sample generated by the random forest model.

[0026] Multiple datasets of the same size as the original training sample set are generated using the bootstrap method to construct multiple decision trees for the random forest prediction model. In each decision tree, environmental variables are randomly divided into multiple environmental variable subsets, and the node branches of each decision tree are constructed by randomly dividing the environmental variable subsets.

[0027] The prediction accuracy of the random forest prediction model is evaluated using the coefficient of determination and root mean square error based on an independent validation set. The coefficient of determination R... 2 for:

[0028]

[0029] The mean square root error (RMSE) is:

[0030]

[0031] Where n is the number of samples, y i Let represent the measured mineral-bound organic carbon content of the i-th sample. Let be the predicted mineral-bound organic carbon content of the i-th sample generated by the random forest model.

[0032] A soil mineral-bound organic carbon prediction device based on random forest and environmental variables includes a computer memory, a computer processor, and a computer program stored in the computer memory and executable on the computer processor. The computer memory contains a final random forest prediction model constructed using the soil mineral-bound organic carbon prediction method based on random forest and environmental variables.

[0033] When the computer processor executes the computer program, it performs the following steps:

[0034] The key environmental variables of the soil sample to be tested are input into the final random forest prediction model to obtain the predicted soil mineral-bound organic carbon.

[0035] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0036] This invention obtains important environmental variables that are strongly correlated with the actual soil mineral bound state through a recursive feature elimination algorithm. The actual soil mineral bound state organic carbon content and the corresponding important environmental variables are used as a sample set to train and validate the initial random forest prediction model to obtain the final random forest model. The soil mineral bound state organic carbon content is obtained by inputting important environmental variables into the final random forest model. Compared with the existing technology that obtains the soil mineral bound state organic carbon content through physicochemical methods, this method is more accurate and efficient.

[0037] This invention establishes a mapping relationship between important environmental variables and soil mineral-bound organic carbon content through the final random forest model. Therefore, the soil mineral-bound organic carbon content can be obtained based on the values ​​of important environmental variables through the final random forest model constructed in this invention, thereby avoiding the problem of not being able to obtain the soil mineral-bound organic carbon content of a soil sample due to the loss of the soil sample. Attached Figure Description

[0038] Figure 1 A flowchart of a method for predicting soil mineral-bound organic carbon based on random forest and environmental variables, provided in an embodiment of the present invention;

[0039] Figure 2 Scatter plot of measured and predicted values ​​of soil mineral-bound organic carbon in an independent validation set provided in this embodiment of the invention. Detailed Implementation

[0040] To make the objectives, technical solutions, and technical effects of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings. In this embodiment, environmental variables of typical soil samples from Zhejiang Province are selected for modeling, ultimately yielding a soil mineral-bound organic carbon prediction model.

[0041] This invention provides a method for predicting soil mineral-bound organic carbon based on random forests and environmental variables, comprising:

[0042] (1) Obtaining soil datasets and environmental variable sets: Based on land cover and use classification maps, 350 soil samples covering farmland and forestland were collected from various parts of Zhejiang Province. The soil mineral-bound organic carbon was obtained through laboratory physicochemical analysis, forming a soil dataset. Based on the spatial location of the soil samples, i.e., latitude and longitude coordinates, the soil mineral-bound organic carbon was collected through soil databases and remote sensing data product libraries to form a set of relevant and easily accessible environmental variables.

[0043] The environmental variable set provided by this invention includes soil variables, climate variables, biological variables, and topographic variables. Soil variables include soil organic carbon, total nitrogen, total phosphorus, total potassium, cation exchange capacity, sand, silt, clay, silt + clay, pH, and soil rainfall erosion; climate variables include annual precipitation, annual mean temperature, potential evapotranspiration, and atmospheric nitrogen deposition; biological variables include normalized difference vegetation index, enhanced vegetation index, and net primary productivity; and topographic variable is elevation.

[0044] The soil variables provided in this invention, including soil organic carbon, total nitrogen, total phosphorus, total potassium, cation exchange capacity, sand, silt, clay, silt + clay, and pH, were measured in the laboratory according to national standards. Soil rainfall erosion was estimated by modifying the general soil loss equation, and the data came from global soil rainfall erosion data at a resolution of 250 meters (https: / / esdac.jrc.ec.europa.eu / content / global-soil-erosion). The climate variables, including annual precipitation and annual mean temperature, came from WorldClim. Version 2 global 1km climate data product (http: / / www.worldclim.com / version2), potential evapotranspiration is from Global-PET global potential evapotranspiration data product (https: / / cgiarcsi.community / data / global-aridity-and-pet-database), and atmospheric nitrogen deposition is from the global nitrogen deposition database (https: / / conservancy.umn.edu / handle / 11299 / 197613); the normalized difference vegetation index (NDVI) and enhanced vegetation index (EVI) in the biological variables are calculated from Landsat 8 data according to the following formula.

[0045]

[0046] EVI=2.5×((NIR-R) / (NIR+6×R-7.5×B+1)

[0047] Wherein, NIR stands for near-infrared band (Band 5), R stands for infrared band (Band 4), and B stands for blue band (Band 2).

[0048] The elevation data for topographic variables provided in this invention are derived from the 90-meter resolution global SRTM digital elevation database (https: / / srtm.csi.cgiar.org). All environmental variable projection transformations and sample point data extractions were implemented using the sp and raster software packages in the free and open-source software R.

[0049] (2) Screening the set of environmental variables: The important environmental variables that are strongly correlated with the prediction of soil mineral-bound organic carbon content are screened out by the recursive feature elimination algorithm.

[0050] The specific steps of the recursive feature elimination algorithm provided by this invention are as follows:

[0051] S1: Fit a first random forest model based on the set of environmental variables. Fit a second random forest model to the remaining environmental variables after removing a single environmental variable from the set of environmental variables. Obtain the accuracy of the first random forest model and the second random forest model respectively through cross-validation. Use the difference between the accuracy of the second random forest model and the accuracy of the first random forest model as the importance of the corresponding single environmental variable removed. The larger the difference, the higher the importance. Remove single environmental variables in turn until the importance of each environmental variable in the set of environmental variables is obtained.

[0052] S2: Remove the single environmental variable with the lowest importance to obtain the combination of environmental variables. Fit the combination of environmental variables to the third random forest model. After removing the single environmental variable from the combination of environmental variables, fit the remaining environmental variables to the fourth random forest model. Again, use the cross-validation method to obtain the accuracy of the third and fourth random forest models respectively. Calculate the difference between the accuracy of the fourth random forest model and the accuracy of the third random forest model to obtain the corresponding importance of the removed single environmental variable. Continue to remove single environmental variables until each environmental variable in the combination of environmental variables has obtained an importance.

[0053] S3: Repeat step S2 until the environmental variables are combined into a single environmental variable and then stop repeating, thereby obtaining the accuracy of multiple random forest models fitted by multiple sets of environmental variables.

[0054] The recursive feature elimination method provided by this invention identifies 12 of the most relevant environmental variables, including soil organic matter, total nitrogen, cation exchange capacity, silt + clay, sand, clay, net primary productivity, normalized vegetation index, potential evapotranspiration, mean annual temperature, enhanced vegetation index, and atmospheric nitrogen deposition.

[0055] The accuracy evaluation index of the random forest model provided by this invention is the root mean square error (RMSE), which is:

[0056]

[0057] Where n is the number of samples, y i Let represent the measured mineral-bound organic carbon content of the i-th sample. Let be the predicted mineral-bound organic carbon content of the i-th sample generated by the random forest model.

[0058] (3) Constructing the modeling set and independent validation set: The soil dataset and the corresponding set of selected environmental variables are used as the sample set. The sample set is divided into a modeling set and an independent validation set. By random sampling, 70% of the data in the soil dataset is assigned to the modeling set, and the remaining 30% is assigned to the independent validation set. In this embodiment, the modeling set contains 245 soil samples, and the independent validation set contains 105 soil samples. These samples all contain the content of soil mineral-bound organic carbon and environmental variables selected by a recursive elimination algorithm.

[0059] (4) Training the initial random forest prediction model: Using the soil mineral-bound organic carbon content and selected environmental variables from the modeling set as training data, the initial random forest prediction model is trained. The parameters of the initial random forest prediction model are optimized through ten-fold cross-validation to obtain the random forest prediction model. In this embodiment, the number of decision tree nodes in the random forest is optimized to 5 for predicting variables.

[0060] The random forest prediction model provided in this invention is a data mining algorithm that generates n datasets (n is usually set to 500) of the same size as the training dataset using the bootstrap method (sampling with replacement), and these datasets are used to construct n decision trees. Environmental variables are randomly divided into multiple subsets, and within each decision tree, branches are generated by randomly dividing these subsets. The final prediction result of the model is the average of the prediction results from all decision trees.

[0061] (5) Validate the random forest prediction model: The established random forest prediction model is used to predict the independent validation set. The measured mineral-bound organic carbon of the real soil samples in the independent validation set is compared with the predicted mineral-bound organic carbon to evaluate the prediction accuracy of the random forest transformation function. When the prediction accuracy meets the standard, the established random forest transformation function can be used to predict the mineral-bound organic carbon in the soil.

[0062] This embodiment uses the coefficient of determination (R²) 2 The prediction accuracy on the independent validation set is evaluated using the root mean square error (RMSE) and the root mean square error (RMSE). The evaluation results are as follows: Figure 2 As shown. By Figure 2 It can be seen that the R of the independent validation set 2The value was 0.93, and the RMSE was 3.44 g / kg. -1 It has a very good predictive effect.

[0063] The determination coefficient R provided by this invention 2 for:

[0064]

[0065] The mean square root error (RMSE) provided by this invention is:

[0066]

[0067] Where n is the number of samples, y i Let represent the measured mineral-bound organic carbon content of the i-th sample. Let be the predicted mineral-bound organic carbon content of the i-th sample generated by the random forest model.

[0068] When applying the method, the key environmental variables of the soil sample to be tested are input into the final random forest prediction model to obtain the predicted soil mineral-bound organic carbon.

[0069] The present invention also provides a soil mineral-bound organic carbon prediction device based on random forest and environmental variables, including a computer memory, a computer processor, and a computer program stored in the computer memory and executable on the computer processor. The computer memory contains a final random forest prediction model constructed using a soil mineral-bound organic carbon prediction method based on random forest and environmental variables.

[0070] When the computer processor executes the computer program, it performs the following steps:

[0071] The key environmental variables of the soil sample to be tested are input into the final random forest prediction model to obtain the predicted soil mineral-bound organic carbon.

[0072] The beneficial effects of this invention are as follows: The random forest transformation function proposed in this invention, combined with environmental variables, can quickly and accurately predict the content of soil mineral-bound organic carbon. This not only reduces the testing costs of traditional soil laboratory physicochemical analysis but also greatly improves the efficiency of using historical soil databases, providing a new approach for acquiring key soil data for low-cost, high-precision, large-scale soil resource surveys and ecological environment modeling. The prediction method provided by this invention can trace the content of soil mineral-bound organic carbon based on historically important environmental variables.

Claims

1. A method for predicting soil mineral-bound organic carbon based on random forest and environmental variables, characterized in that, include: (1) Obtain multiple soil samples, use the real soil mineral-bound organic carbon content of multiple soil samples as a soil dataset, and obtain a set of environmental variables based on the spatial location of soil samples through a soil database and a remote sensing data product library. (2) Based on the set of environmental variables, multiple sets of environmental variables and the corresponding accuracy of random forest models are obtained through recursive feature elimination algorithm. The set of environmental variables corresponding to the highest accuracy of random forest models is taken as important environmental variables. The method for obtaining multiple sets of environmental variables and the corresponding random forest model accuracy based on the set of environmental variables through a recursive feature elimination algorithm includes: S1: Fit a first random forest model based on the set of environmental variables. Fit a second random forest model to the remaining environmental variables after removing a single environmental variable from the set of environmental variables. Obtain the accuracy of the first random forest model and the second random forest model respectively through cross-validation. Use the difference between the accuracy of the second random forest model and the accuracy of the first random forest model as the importance of the corresponding single environmental variable removed. The larger the difference, the higher the importance. Remove single environmental variables in turn until the importance of each environmental variable in the set of environmental variables is obtained. S2: Remove the single environmental variable with the lowest importance to obtain the combination of environmental variables. Fit the combination of environmental variables to the third random forest model. After removing the single environmental variable from the combination of environmental variables, fit the remaining environmental variables to the fourth random forest model. Again, use the cross-validation method to obtain the accuracy of the third and fourth random forest models respectively. Calculate the difference between the accuracy of the fourth random forest model and the accuracy of the third random forest model to obtain the corresponding importance of the removed single environmental variable. Continue to remove single environmental variables until each environmental variable in the combination of environmental variables has obtained an importance. S3: Repeat step S2 until the environmental variables are combined into a single environmental variable and then stop repeating, thereby obtaining the accuracy of multiple random forest models fitted to multiple sets of environmental variables; (3) The soil dataset and the corresponding set of screened environmental variables are used as the sample set. The sample set is divided into the modeling set and the independent validation set. The initial random forest prediction model is obtained. Based on the modeling set, the parameters of the initial random forest prediction model are optimized by ten-fold cross-validation to obtain the random forest prediction model. Based on the independent validation set, the prediction accuracy of the random forest prediction model is evaluated by the coefficient of determination and the root mean square error. When the prediction accuracy threshold is reached, the final random forest prediction model is obtained. (4) When applying, the important environmental variables of the soil sample to be tested are input into the final random forest prediction model to obtain the predicted soil mineral-bound organic carbon.

2. The method for predicting soil mineral-bound organic carbon based on random forest and environmental variables according to claim 1, characterized in that, The soil dataset comprises the actual soil mineral-bound organic carbon content of multiple soil samples, including: Multiple soil samples covering farmland and forestland were collected based on land cover and classification maps. The true soil mineral-bound organic carbon of each sample was obtained through physicochemical analysis. The true soil mineral-bound organic carbon of multiple samples was used to construct a soil dataset.

3. The method for predicting soil mineral-bound organic carbon based on random forest and environmental variables according to claim 1, characterized in that, Environmental variables are obtained based on the spatial location of soil samples through a soil database and a remote sensing data product library. The set of environmental variables includes soil variables, climate variables, biological variables, and topographic variables.

4. The method for predicting soil mineral-bound organic carbon based on random forest and environmental variables according to claim 3, characterized in that, The soil variables include soil organic carbon, total nitrogen, total phosphorus, total potassium, cation exchange capacity, sand, silt, clay, silt + clay, pH, and soil rainfall erosion. The climate variables include annual precipitation, annual mean temperature, potential evapotranspiration, and atmospheric nitrogen deposition. The biological variables include the normalized vegetation index, the enhanced vegetation index, and net primary productivity; The terrain variable is elevation.

5. The method for predicting soil mineral-bound organic carbon based on random forest and environmental variables according to claim 1, characterized in that, The accuracy evaluation metric for the random forest model is the root mean square error (RMSE), which is: Where n is the number of samples, y i Let represent the measured mineral-bound organic carbon content of the i-th sample. Let be the predicted mineral-bound organic carbon content of the i-th sample generated by the random forest model.

6. The method for predicting soil mineral-bound organic carbon based on random forest and environmental variables according to claim 1, characterized in that, Multiple datasets of the same size as the training sample set are generated using the bootstrap method to construct multiple decision trees for the random forest prediction model. In each decision tree, environmental variables are randomly divided into multiple environmental variable subsets, and the node branches of each decision tree are constructed by randomly dividing the environmental variable subsets.

7. The method for predicting soil mineral-bound organic carbon based on random forest and environmental variables according to claim 1, characterized in that, The prediction accuracy of the random forest prediction model is evaluated using the coefficient of determination and root mean square error based on an independent validation set. The coefficient of determination R... 2 for: The mean square root error (RMSE) is: Where n is the number of samples, y i Let represent the measured mineral-bound organic carbon content of the i-th sample. Let be the predicted mineral-bound organic carbon content of the i-th sample generated by the random forest model.

8. A soil mineral-bound organic carbon prediction device based on random forest and environmental variables, comprising a computer memory, a computer processor, and a computer program stored in the computer memory and executable on the computer processor, characterized in that, The computer memory contains the final random forest prediction model constructed using the soil mineral-bound organic carbon prediction method based on random forest and environmental variables as described in any one of claims 1 to 7. When the computer processor executes the computer program, it performs the following steps: The key environmental variables of the soil sample to be tested are input into the final random forest prediction model to obtain the predicted soil mineral-bound organic carbon.

Images