Soil organic matter content inversion method based on machine learning and macrotopographic factors
By combining machine learning and macro-topographic factors and optimizing model parameters, the problem of low accuracy in soil organic matter inversion was solved, and a more accurate and widely applicable soil organic matter inversion method was achieved.
Patent Information
- Application Number
- CN202511027970.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-24
- Publication Date
- 2025-11-11
AI Technical Summary
Existing methods for inverting soil organic matter content are difficult to accurately capture nonlinear relationships, are affected by various factors, have poor generalization ability, and ignore the spatial relationships of topographic factors, resulting in low inversion accuracy.
By combining machine learning and macro-topographic factors, remote sensing data and topographic data were collected, and models such as LightGBM, XGBoost, RF and RBFNN were used to invert soil organic matter, select the optimal model, and introduce macro-topographic factors such as topographic roughness and surface incision depth to optimize model parameters.
The accuracy and generalization ability of soil organic matter inversion were improved, with the model accuracy increasing by 3.28%. The XGBoost model performed best, with R2 reaching 0.69 and RMSE of 5.1923. The inversion results showed a high degree of agreement with the measured values.
Smart Images
Figure CN120927620A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of soil quality assessment, and in particular to a method for retrieving soil organic matter content based on machine learning and macro-topographic factors. Background Technology
[0002] While existing methods for retrieving soil organic matter content have made some progress, the following key issues still exist:
[0003] The relationship between soil organic matter content and its spectral reflectance is a highly complex nonlinear relationship. Traditional statistical models (such as multiple linear regression) struggle to accurately capture and simulate this complex nonlinear relationship, and they are highly dependent on training data. Models built in one region are difficult to directly apply to other regions, exhibiting poor generalization ability.
[0004] Soil is a complex mixture whose spectral characteristics are affected not only by organic matter, but also by soil texture (the ratio of clay, silt, and sand), mineral composition (such as iron oxides and carbonates), soil structure, and surface roughness. As a result, the spectral signal of organic matter is easily "submerged" by the signals of other background components, making it difficult to separate and extract accurately.
[0005] Traditional pixel-level inversion methods typically treat each pixel as an independent unit, neglecting its spatial relationship with the surrounding environment. Topographic factors significantly control the spatial distribution pattern of organic matter by influencing water and heat redistribution, erosion, and deposition processes. Ignoring these macroscopic control factors results in the loss of crucial information for predicting the spatial distribution of SOM.
[0006] Therefore, there is an urgent need in this field for a technical solution that can solve the problems existing in current methods for inverting soil organic matter content. Summary of the Invention
[0007] The purpose of this invention is to provide a soil organic matter content inversion scheme based on machine learning and macroscopic topographic factors that can solve the problems of existing soil organic matter content inversion methods.
[0008] To achieve the above objectives, the present invention provides the following solution:
[0009] A method for inverting soil organic matter content based on machine learning and macro-topographic factors includes:
[0010] Collect soil organic matter data.
[0011] Acquire remote sensing data and construct spectral indices for reflectance in each band of the multispectral data;
[0012] Collect topographic data, including: elevation, micro-topographic factors, and macro-topographic factors;
[0013] Multiple modeling methods were used to invert soil organic matter, and the inversion results of different models were compared and analyzed to select the optimal model.
[0014] Optionally, the collection of soil organic matter data specifically includes:
[0015] Sampling point locations were designed based on the spatial distribution and area proportion of each soil type. Soil samples from the top 0-20cm layer were collected using the five-point mixing method, and the organic matter content was determined using the potassium dichromate method.
[0016] Optionally, the remote sensing data uses MODIS MCD43A3 V6 as the data source, with a data period of 16 days, covering 10 spectral bands from visible light to shortwave infrared.
[0017] Optionally, three vegetation indices can be selected as modeling factors: normalized vegetation index, green normalized difference vegetation index, and ratio vegetation index.
[0018] Optionally, the micro-topographic factors include: slope, aspect, profile curvature, and planar curvature;
[0019] Macro-topographic factors include: topographic relief, topographic roughness, and surface incision depth.
[0020] Optionally, load the matplatlib, numpy, pandas, and sklearn libraries in the Python environment for data visualization, multidimensional array computation, data processing, and machine learning model building, respectively; divide the dataset into 80% for model training data and the remaining data for test set data.
[0021] Optionally, the modeling methods include: LightGBM, XGBoost, RF, and RBFNN;
[0022] The specific steps for retrieving soil organic matter using the LightGBM algorithm include:
[0023] Data preprocessing is performed using gradient one-sided sampling algorithm and mutually exclusive feature binding algorithm. The gradient one-sided sampling algorithm adjusts the sample weights according to the sample gradient to improve training efficiency and model accuracy while reducing computation.
[0024] The parameters in the constructed LightGBM-based model include:
[0025] max_deep = [5, 10, 15]; learning_rate is set to 0.01; num_leaves = 50. The values of the above three parameters are the final model parameter settings obtained through experiments with a large number of sample data. Other parameters are set to default values.
[0026] Optionally, the optimal configuration parameters determined by inverting soil organic matter using the XGBoost algorithm include:
[0027] max_deep = [2,5,10]; learning_rate is set to 0.1; n_estimators = [50,100,150], and other parameters are set to their default values.
[0028] Optionally, the optimal configuration parameters determined by inverting soil organic matter using the RF algorithm include:
[0029] max_deep = [5, 10, 15]; n_estimators = [50, 100, 150], and other parameters are set to their default values.
[0030] Optionally, the RBFNNRF algorithm can be used to invert soil organic matter, specifically including:
[0031] Clustering algorithms are used to determine the centers of hidden layer nodes, and linear regression is used to optimize the weights of the output layer.
[0032] After extensive sample experiments, it was determined that the number of hidden layer center points was 300 and the initial standard deviation was 1. The model training efficiency was optimal under these parameter settings. For large-scale training datasets, training efficiency was improved by interval sampling, with the data interval set to 500 to reduce the amount of raster data sampling.
[0033] Compared with the prior art, the present invention has the following beneficial effects:
[0034] Traditional research often focuses on spectral information. This invention innovatively integrates remote sensing spectroscopy, vegetation factors, and two major categories of multi-scale topographic factors: microscopic and macroscopic. By introducing macroscopic factors such as topographic roughness and surface incision depth, it can more comprehensively characterize the cumulative effects of complex topography on the redistribution of water, soil, and nutrients, thereby more accurately capturing the key driving forces influencing the spatial differentiation of soil organic matter.
[0035] This invention overcomes the limitations of a single model by systematically comparing current mainstream machine learning algorithms (including gradient boosting tree models such as LightGBM and XGBoost, random forests, and neural networks). This cross-type model comparison and evaluation not only verifies the applicability of advanced algorithms in soil property inversion but also empirically selects the optimal model for the study area, providing a reliable and replicable technical path for achieving high-precision and high-efficiency regional soil mapping. Attached Figure Description
[0036] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0037] Figure 1 Scatter plot of measured and inverted values of soil organic matter provided in embodiments of the present invention.
[0038] Figure 2 The inversion diagram is provided for an embodiment of the present invention.
[0039] Figure 3 This is a schematic diagram comparing the distribution of inversion values of different models provided in embodiments of the present invention. Detailed Implementation
[0040] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0041] The purpose of this invention is to provide a soil organic matter content inversion scheme based on machine learning and macroscopic topographic factors that can solve the problems of existing soil organic matter content inversion methods.
[0042] Example 1:
[0043] This embodiment provides a method for inverting soil organic matter content based on machine learning and macro-topographic factors. Taking a certain region as the study area, based on MODIS multispectral remote sensing data, combined with vegetation factors, elevation, micro-topographic factors (such as slope, aspect, profile curvature, and plane curvature), and macro-topographic factors (such as topographic roughness, topographic relief, and surface incision depth), various modeling methods such as LightGBM, XGBoost, RF, and RBFNN are used to invert soil organic matter. The inversion results of different models are compared and analyzed, and the optimal model is selected. It is found that after adding macro-topographic factors, the model inversion accuracy is improved by an average of about 3.28%, and the XGBoost model performs the best. 2 The model achieved an accuracy of 0.69, a model precision of 85.51%, and an RMSE of 5.1923. Combining spectral information with multi-scale topographic features is an effective way to improve the accuracy of soil organic matter inversion in this region.
[0044] The study area has a temperate monsoon climate with four distinct seasons, large diurnal temperature range, and complex and diverse topography.
[0045] The method for collecting soil organic matter data is to design sampling point locations based on the spatial distribution and area ratio of soil types in each region. Soil samples from the top 0 to 20 cm layer are collected using the five-point mixing method, and the organic matter content is determined using the potassium dichromate method.
[0046] For remote sensing data, this embodiment uses MODIS MCD43A3 V6 as the data source, with a data period of 16 days, covering 10 spectral bands from visible light to shortwave infrared. MODIS image data from the bare soil period in the study area are selected. By constructing spectral indices from the reflectance of each band in the multispectral data, the interference of factors such as topography and atmosphere on reflectance can be effectively reduced, thereby improving the accuracy of the prediction model.
[0047] This embodiment selects three commonly used vegetation indices as modeling factors (Table 1):
[0048] Table 1 Comparison of SOM inversion performance of each model
[0049]
[0050] The Normalized Difference Vegetation Index (NDVI), the Green Normalized Difference Vegetation Index (GNDVI), and the Ratio Vegetation Index (RVI) are three key indicators for soil organic matter retrieval. NDVI indirectly characterizes the spatial distribution of soil organic matter by reflecting vegetation growth status and is an important indicator variable in soil organic matter retrieval. GNDVI is highly sensitive to vegetation health, particularly nitrogen content, which is closely related to soil organic matter content. GNDVI demonstrates strong applicability in both agricultural and natural vegetation environments. RVI reflects vegetation cover by the ratio of near-infrared to red light reflectance. Since areas with higher soil organic matter content are usually accompanied by higher vegetation cover, RVI can provide indirect information support for organic matter retrieval.
[0051] The spatial distribution characteristics of soil organic matter are influenced by topographic factors such as elevation, slope, and aspect, and topographic conditions affect the mode and intensity of its material cycle. Therefore, elevation, micro-topographic factors (slope, aspect, profile curvature, plane curvature), and macro-topographic factors (topographic relief, topographic roughness, and surface incision depth) are selected to invert soil organic matter content in order to improve the accuracy of the model.
[0052] In a Python environment, the matplatlib, numpy, pandas, and sklearn libraries are loaded for data visualization, multidimensional array computation, data processing, and machine learning model building, respectively. The dataset is divided into two parts: 80% is used for model training, and the remaining data is used as the test set.
[0053] The LightGBM (Light Gradient Boosting Machine) algorithm is a gradient boosting framework based on GBDT, using decision trees as base learners and suitable for classification, regression, and ranking problems. Similar to XGBoost, it fits a new decision tree using negative gradients, but unlike traditional decision trees which are based on layer splitting, LightGBM splits based on leaf nodes, reducing unnecessary splits. For data preprocessing, it introduces Gradient One-Sided Sampling (GOSS) and Equated Feature Bundling (EFB) algorithms. GOSS adjusts sample weights based on the sample gradient magnitude, improving training efficiency and model accuracy while reducing computational cost. In the LightGBM-based model constructed in this embodiment, the main parameters are as follows: max_deep = [5, 10, 15]; learning_rate is set to 0.01; num_leaves = 50. These three parameter values are the final model parameter settings obtained through experiments with a large amount of sample data; other parameters are set to default values.
[0054] Extreme Gradient Boosting (XGBoost) is an algorithm that improves the performance of gradient boosting machines, suitable for regression trees and K-classification methods. It optimizes computation, simplifies the objective function, and increases computation speed, reducing overestimation events and automatically synchronizing computation during training. The algorithm is widely used in classification, regression, and ranking tasks, utilizing parallel processing and caching to efficiently handle large-scale datasets, automatically handling missing values, controlling model complexity through L1 and L2 regularization to reduce overfitting risk, supporting flexible tree construction methods, and enhancing model generalization ability. This embodiment, through analysis of a large amount of sample data, determines the optimal parameter configuration as follows: max_deep = [2, 5, 10]; learning_rate set to 0.1; n_estimators = [50, 100, 150], with other parameters set to default values.
[0055] Random Forest (RF) is an ensemble learning method based on an ensemble of classification and regression trees (CART). It is considered to have similar or higher accuracy to AdaBoost, while being more computationally efficient. RF improves the accuracy and stability of the model by constructing multiple independent decision trees. Each tree randomly selects samples and features to reduce variance and overfitting. The final prediction is obtained through voting (classification) or averaging (regression), and feature importance is evaluated to support feature selection and model interpretation. Due to its strong adaptability, it is widely used in classification and regression problems and is one of the commonly used benchmark models. In this embodiment, parameters are adjusted during the modeling process to find the optimal model and complete the model construction. The optimal parameter configuration is determined as: max_deep = [5, 10, 15]; n_estimators = [50, 100, 150], with other parameters set to default values.
[0056] Radial Basis Function Neural Networks (RBFNNs) are a type of feedforward neural network consisting of an input layer, hidden layers, and an output layer. The hidden layers use radial basis functions such as Gaussian functions as activation functions, making them suitable for pattern recognition and nonlinear function approximation. During training, clustering algorithms are used to determine the centers of the hidden layer nodes, and linear regression is used to optimize the weights of the output layer. RBFNNs have a simple structure and fast learning speed, and are widely used in classification, regression, and time series analysis. This embodiment uses a clustering method to determine the center points and initializes them randomly. After extensive sample experiments, the number of hidden layer center points was determined to be 300, with an initial standard deviation of 1. This parameter setting resulted in optimal model training efficiency. For large-scale training datasets, interval sampling is used to improve training efficiency; the data interval is set to 500 to reduce the amount of raster data sampled.
[0057] To compare the impact of different variable combinations on soil organic matter inversion results, the inversion performance of four machine learning algorithms (LGBM, RF, XGBoost, and RBFNN) was evaluated (Table 1). The results showed that, among the models built using Modis (Band 1-10) model A, model A-4 using the XGBoost algorithm had the best inversion performance (M = 84.75%, RMSE = 5.3715 g / kg). Among the models built using Modis (Band 1-7) model B, model B-4 using the XGBoost algorithm had the best inversion performance (M = 85.51%, RMSE = 5.1923 g / kg). Under the LGBM, XGBoost, RF, and RBFNN algorithms, the highest M values in the models built using Modis (Band 1-10) were 83.04%, 84.75%, 84.22%, and 63.43%, respectively. Only RBFNN corresponds to an A-1 variable model, while the others correspond to A-4 full variable models. In the models built using Modis (Band 1-7), the highest M values were 82.39%, 85.51%, 84.65%, and 63.82%, respectively, all corresponding to model B-4 full variable models. This indicates that the addition of auxiliary variables such as spectral indices and topographic factors improved the inversion performance to varying degrees. Reducing the Modis (Band 1-7) model B in Band 8-10, compared to model A (Band 1-10), resulted in an overall RMSE reduction of 0.1-0.3, an R² increase of 0.01-0.04, and a model accuracy improvement of 0.05%-1.43%. Overall, among the four machine learning algorithms, XGBoost and RF algorithms outperformed LGBM and RBFNN algorithms in inversion performance, with XGBoost outperforming RF. Regarding variable combinations, combining spectral bands, spectral indices, and topographic factors improved inversion accuracy; models using only Bands 1-7 showed better inversion performance than models using Bands 1-10.
[0058] A model was constructed based on the B-4 variable combination. Using test set data that was not involved in the modeling, a scatter plot of the measured and inverted values of soil organic matter was drawn. Figure 1 The results show that the R-value of the XGBoost model is... 2The highest score was 0.69, with a fitted line slope of 0.66, closer to the 1:1 ideal line. The inversion results matched the actual values better, demonstrating high accuracy and stability. Looking at the residual distribution, XGBoost's residuals were concentrated and symmetrical, with most data points within the 95% confidence interval, indicating a smaller relative error in the model inversion. In contrast, LightGBM showed larger residual deviations in the low-value region and RBFNN in the high-value region, suggesting certain errors in specific organic matter content ranges. Furthermore, the XGBoost model exhibited higher stability in extreme value inversions, with data points in both high and low organic matter content ranges approaching the 1:1 ideal line. Other models deviated significantly in extreme value inversions, prone to overestimation or underestimation. Therefore, XGBoost's more concentrated data distribution and narrower confidence interval reflect its high consistency and reliability across most data points. It performed best in soil organic matter content inversions, demonstrating better generalization ability and application potential.
[0059] Soil organic matter was inverted using the optimal B-4 variable combination to obtain a spatial distribution map of soil organic matter in the study area. Figure 2 The measured soil organic matter content in this region showed significant differences across different areas. The northwest and southeast regions had higher organic matter content, ranging from 41.3013 to 61.9051 g / kg, while the southwest region had lower organic matter content, ranging from 3.6980 to 29.8511 g / kg. Comparison with measured data shows that the XGBoost model performed better in all directions, with the inversion results highly consistent with the spatial distribution of the measured data. In the medium-to-high value areas of the east and north, the inverted values were mostly green with some yellow areas, similar to the measured values. In the low value areas of the south and west, the model accurately identified blue and light blue low-value areas. Therefore, the XGBoost model performed best in terms of color and spatial distribution, outperforming the other three models.
[0060] The LightGBM model shows more green and less yellow in the median value regions of the east and north, consistent with the wide distribution of the measured data. However, in the high and low value regions of the east and south, the retrieved values deviate from the measured values, with inaccurate retrieval of red in high-value regions and blue in low-value regions. The RF model performs similarly to the XGBoost model, also showing relatively accurate yellow and green in the medium-to-high value regions of the east and north, matching the measured values well. Meanwhile, in the low value regions of the south and west, the RF model can also identify blue and light blue areas, with an overall retrieval effect close to XGBoost, especially in the color distribution of high and low value regions. The RBFNN model has a more dispersed color distribution, correctly identifying some high value regions, but overall, the color jumps are obvious, making it difficult to accurately reflect the high value distribution in the east and north, and also failing to stably reflect the low value regions in the south and west, resulting in a relatively poor match with the measured data.
[0061] Comparison of inversion value distributions ( Figure 3 The following further demonstrates the differences in coverage ratio among the four models. The measured soil organic matter content ranged from 3.6984 to 62.4968 g / kg. The LightGBM model's inversion range was 15.2146-43.3780 g / kg, covering approximately 47.8% of the measured range, exhibiting a relatively low matching degree and poor inversion accuracy in high-content areas. The XGBoost model's inversion range was 3.7595-56.7361 g / kg, with the highest coverage ratio of approximately 89.9%, providing relatively accurate inversion results across a range of low to high organic matter content, demonstrating strong inversion capability. The RF model's inversion range was between 7.9349 and 52.4452 g / kg, with a coverage rate of approximately 75.7%, showing good inversion performance and a relatively balanced spatial distribution. The RBFNN model's inversion range is 12.9940-59.9132 g / kg, with a coverage of approximately 79.8%. It is relatively sensitive in areas with high organic matter content and is suitable for inverting areas with high organic matter content. In summary, the XGBoost model performs best in soil organic matter content inversion, and its high coverage and accurate inversion capabilities provide effective prediction results for different organic matter regions.
[0062] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.
[0063] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.
Claims
1. A method for inverting soil organic matter content based on machine learning and macroscopic topographic factors, characterized in that, include: Collect soil organic matter data. Acquire remote sensing data and construct spectral indices for reflectance in each band of the multispectral data; Collect topographic data, including: elevation, micro-topographic factors, and macro-topographic factors; Multiple modeling methods were used to invert soil organic matter, and the inversion results of different models were compared and analyzed to select the optimal model.
2. The method for inverting soil organic matter content based on machine learning and macroscopic topographic factors according to claim 1, characterized in that, The specific details of collecting soil organic matter data are as follows: Sampling point locations were designed based on the spatial distribution and area proportion of each soil type. Soil samples from the top 0-20cm layer were collected using the five-point mixing method, and the organic matter content was determined using the potassium dichromate method.
3. The method for inverting soil organic matter content based on machine learning and macroscopic topographic factors according to claim 1, characterized in that, The remote sensing data used was MODIS MCD43A3 V6 as the data source, with a data period of 16 days, covering 10 spectral bands from visible light to shortwave infrared.
4. The method for inverting soil organic matter content based on machine learning and macroscopic topographic factors according to claim 1, characterized in that, Three vegetation indices were selected as modeling factors: normalized vegetation index, green normalized difference vegetation index, and ratio vegetation index.
5. The method for inverting soil organic matter content based on machine learning and macroscopic topographic factors according to claim 1, characterized in that, The micro-topographic factors include: slope, aspect, profile curvature, and planar curvature; Macro-topographic factors include: topographic relief, topographic roughness, and surface incision depth.
6. The method for inverting soil organic matter content based on machine learning and macroscopic topographic factors according to claim 1, characterized in that, In a Python environment, the matplatlib, numpy, pandas, and sklearn libraries are loaded for data visualization, multidimensional array computation, data processing, and machine learning model building, respectively. Divide the dataset into 80% for model training and the remaining data for the test set.
7. The method for inverting soil organic matter content based on machine learning and macroscopic topographic factors according to claim 1, characterized in that, The modeling methods include: LightGBM, XGBoost, RF, and RBFNN; The specific steps for retrieving soil organic matter using the LightGBM algorithm include: Data preprocessing is performed using gradient one-sided sampling algorithm and mutually exclusive feature binding algorithm. The gradient one-sided sampling algorithm adjusts the sample weights according to the sample gradient to improve training efficiency and model accuracy while reducing computation. The parameters in the constructed LightGBM-based model include: max_deep = [5, 10, 15]; learning_rate is set to 0.01; num_leaves = 50. The values of the above three parameters are the final model parameter settings obtained through experiments with a large number of sample data. Other parameters are set to default values.
8. The method for inverting soil organic matter content based on machine learning and macroscopic topographic factors according to claim 7, characterized in that, The optimal configuration parameters determined by inverting soil organic matter using the XGBoost algorithm include: max_deep = [2,5,10]; learning_rate is set to 0.1; n_estimators = [50,100,150], and other parameters are set to their default values.
9. The method for inverting soil organic matter content based on machine learning and macroscopic topographic factors according to claim 7, characterized in that, The optimal configuration parameters determined by inverting soil organic matter using the RF algorithm include: max_deep = [5, 10, 15]; n_estimators = [50, 100, 150], and other parameters are set to their default values.
10. The method for inverting soil organic matter content based on machine learning and macroscopic topographic factors according to claim 7, characterized in that, The specific steps for inverting soil organic matter using the RBFNNRF algorithm include: Clustering algorithms are used to determine the centers of hidden layer nodes, and linear regression is used to optimize the weights of the output layer. After extensive sample experiments, it was determined that the number of hidden layer center points was 300 and the initial standard deviation was 1. The model training efficiency was optimal under these parameter settings. For large-scale training datasets, training efficiency was improved by interval sampling, with the data interval set to 500 to reduce the amount of raster data sampling.