Black soil cultivated land organic carbon degradation diagnosis method based on geographic big data and AI

Abstract

The invention provides a black soil cultivated land organic carbon degradation diagnosis method based on geographic big data and AI, and the method comprises the steps: constructing a black soil area cultivated land organic carbon degradation potential influence factor data set and a diagnosis model, judging the spatial correlation of potential influence factors, training the diagnosis model based on the data set, and obtaining a diagnosis result of the black soil area cultivated land organic carbon degradation. And performing black soil area cultivated land organic carbon degradation influence factor identification based on the trained diagnosis model, and performing organic carbon degradation influence factor action mechanism analysis based on an identification result. According to the method, by utilizing high resolution and high timeliness of geographic space big data and cloud computing, multiple types of potential factors such as climate, soil, terrain and agricultural activities which influence organic carbon change of the cultivated land in the black soil area can be quickly calculated on a large-area scale, and manpower, material resources and time costs caused by large-area data collection and analysis are avoided; by using an artificial intelligence algorithm, the main control factor of organic carbon degradation of the cultivated land in the black soil area is accurately identified from multiple potential factors, and the current degradation situation can be objectively evaluated.

Description

Technical Field

[0001] The present invention relates to the technical field of organic carbon diagnosis of cultivated land in black soil areas, and in particular to a method for diagnosing organic carbon degradation in cultivated land in black soil areas based on geographic big data and AI. Background Art

[0002] Black soil is a dark-colored soil rich in organic carbon. It has high fertility and is often used for agricultural planting. In recent years, due to high-intensity development and utilization, the organic carbon in cultivated land in my country's black soil region has sharply decreased and its fertility has declined. In the current evaluation of organic carbon degradation in cultivated land in black soil regions, the main methods include soil sample collection and analysis, soil spatial mapping, and expert judgment and evaluation. Among them, large-scale soil sample collection and analysis requires a lot of manpower, material resources and time costs, which seriously affects the timeliness and dynamics of the evaluation results. In addition, simple soil spatial mapping does not fully consider the influence of factors such as climate, topography, and agricultural activities, which affects the accuracy of the evaluation results. Finally, the evaluation indicators obtained based on expert scoring rely on people's prior knowledge and cannot rule out subjective influences.

[0003] At present, geospatial big data, cloud computing and artificial intelligence have been applied in many research fields for spatial feature analysis, identification of influencing factors, etc., but this technology has not been fully applied in the evaluation of organic carbon degradation in cultivated land in black soil areas. Therefore, it is very necessary to design a diagnostic method for organic carbon degradation in cultivated land in black soil based on geospatial big data and AI. Summary of the invention

[0004] In order to overcome the shortcomings of the prior art, the purpose of the present invention is to provide a method for diagnosing organic carbon degradation of black soil farmland based on geographic big data and AI.

[0005] To achieve the above object, the present invention provides the following solutions:

[0006] The present invention provides a method for diagnosing organic carbon degradation of black soil farmland based on geographic big data and AI, comprising:

[0007] Construct a dataset of potential influencing factors of organic carbon degradation in cultivated land in black soil regions;

[0008] Determine the spatial correlation of potential influencing factors;

[0009] Construct a diagnostic model for organic carbon degradation in cultivated land in black soil regions;

[0010] Based on the dataset of potential influencing factors of organic carbon degradation in cultivated land in black soil region, the diagnostic model of organic carbon degradation in cultivated land in black soil region was trained to obtain the trained diagnostic model of organic carbon degradation in cultivated land in black soil region.

[0011] Based on the trained diagnostic model of organic carbon degradation in cultivated land in black soil region, the influencing factors of organic carbon degradation were identified;

[0012] Based on the identification results, the mechanism of action of factors affecting organic carbon degradation is analyzed.

[0013] Preferably, a dataset of potential influencing factors of organic carbon degradation in cultivated land in black soil regions is constructed, specifically:

[0014] Fit the relationship between factors and organic carbon changes, and analyze the impact of factors on organic carbon changes based on this. Among them, factors include climate factors, soil factors, terrain factors and agricultural activity factors. Climate factors include drought index, multi-year average annual precipitation, multi-year average annual temperature, multi-year average annual wind speed and multi-year average spring wind speed. Soil factors include soil bulk density, cation exchange capacity, pH, sand content, silt content, clay content, total nitrogen content, total phosphorus content, total potassium content, initial organic carbon, surface soil temperature and soil parent material. Terrain factors include digital elevation, slope aspect, slope and terrain humidity index. Agricultural activity factors include changes in planting structure and agricultural reclamation management methods.

[0015] For factors with publicly available raster data sources, raster data is obtained based on the Google Cloud computing platform, and the spatial resolution of the raster is resampled;

[0016] For the factors of vector data format, the data is converted from vector to raster based on ArcMap;

[0017] A raster dataset of uniform spatial resolution for all factors was constructed, and a dataset of potential influencing factors of organic carbon degradation in cultivated land in the black soil region was obtained.

[0018] Preferably, the spatial correlation of potential influencing factors is determined as follows:

[0019] The spatial correlation of continuous variables in potential influencing factors was analyzed based on the Pearson correlation coefficient to reduce data redundant information;

[0020] Based on the spatial distribution data of black soil cultivated land, the raster data of climate, soil, topography and agricultural activity factors were spatially masked and cut, and only the data inside the cultivated land in the black soil area was retained;

[0021] Based on ArcMap, the center point vector data of the 5-kilometer grid covering the black soil area was formed. On this basis, the values ​​of all factors and changes in organic carbon in cultivated land were batch extracted based on Arcpy to form a matrix data table.

[0022] Remove the grid center points with missing values ​​in the matrix data table, and only keep the grid center points with valid values ​​for all factors to calculate the Pearson correlation coefficient, excluding redundant variables, which is:

[0023]

[0024] Among them, cov(X,Y) represents the covariance of variables X and Y, σ X and σ Y are the standard deviations of X and Y respectively.

[0025] Preferably, a diagnostic model for organic carbon degradation in cultivated land in black soil regions is constructed, specifically:

[0026] The XGBoost model was constructed as a diagnostic model for organic carbon degradation of cultivated land in black soil regions.

[0027] Preferably, the diagnostic model for organic carbon degradation of cultivated land in black soil regions is trained based on the dataset of potential influencing factors of organic carbon degradation of cultivated land in black soil regions to obtain the trained diagnostic model for organic carbon degradation of cultivated land in black soil regions, specifically:

[0028] Based on the 10-fold cross-validation method, the dataset of potential influencing factors of organic carbon degradation in cultivated land in black soil areas was divided into ten subsets in equal amounts, and each subset was used as a validation set in turn, and the remaining nine subsets were used as training sets to construct ten combinations of training sets and validation sets. For each combination of training sets and validation sets, the XGBoost model was trained based on the training set, and the root mean square error of the model predicted on the validation set was calculated to obtain ten XGBoost models and the corresponding validation set errors. With the goal of minimizing the average validation set error of the ten models, the 10-fold cross-validation process was repeated through the grid search method to determine the optimal parameter combination of the machine learning model.

[0029] The XGBoost model parameters are the maximum depth of the decision tree max_depth and the number of decision trees n_estimators, with grids of [5, 10, 15, 20] and [40, 80, 120, 160] respectively.

[0030] Preferably, the factors affecting organic carbon degradation are identified based on the trained black soil organic carbon degradation diagnostic model, specifically:

[0031] For the ten XGBoost models, the SHAP method is used to calculate the contribution of each factor to the prediction result of each sample in the training set. The absolute value of the average contribution of a factor to all samples in the training set is taken as the importance of the factor. According to the model, ten importance results of each factor will be obtained, and the importance of the factors will be ranked according to the median of the ten importance results.

[0032] Preferably, based on the identification results, the mechanism of influencing factors of organic carbon degradation in cultivated land in black soil areas is analyzed, specifically:

[0033] For each factor, determine its value range, and take 30 values ​​at equal intervals. While keeping the values ​​of other factors unchanged, set the values ​​of the factor in the data set to these 30 values ​​in turn, and calculate the average prediction value of the model respectively, and obtain the relationship between the factor value and the average prediction result. According to the XGBoost model, ten corresponding broken lines will be obtained respectively, and their average broken line and 90% confidence interval will be calculated to obtain the partial dependence diagram, that is, the driving mechanism.

[0034] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0035] The present invention provides a method for diagnosing organic carbon degradation of black soil cultivated land based on geographic big data and AI, the method comprising constructing a dataset of potential influencing factors of organic carbon degradation of cultivated land in black soil areas, constructing a diagnostic model for organic carbon degradation of cultivated land in black soil areas, training the diagnostic model for organic carbon degradation of cultivated land in black soil areas based on the dataset of potential influencing factors of organic carbon degradation of cultivated land in black soil areas, obtaining a trained diagnostic model for organic carbon degradation of cultivated land in black soil areas, identifying factors affecting organic carbon degradation based on the trained diagnostic model for organic carbon degradation of cultivated land in black soil areas, and analyzing the mechanism of action of factors affecting organic carbon degradation of cultivated land in black soil areas based on the identification results. The present invention utilizes geographic space big data and cloud computing to eliminate the manpower, material resources and time costs caused by large-scale soil sample collection and analysis, and based on the high resolution and real-time update capabilities of geographic space big data and cloud computing, can accurately identify the organic carbon degradation area of ​​cultivated land in black soil areas, and dynamically evaluate the degradation at the same time, and utilize artificial intelligence algorithms to accurately identify the main controlling factors of organic carbon degradation of cultivated land in black soil areas, and objectively evaluate the current status of degradation. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0037] Figure 1 A flow chart of a method provided by an embodiment of the present invention;

[0038] Figure 2 A schematic diagram of a technical route provided for an embodiment of the present invention;

[0039] Figure 3 A schematic diagram of the spatial relationship between climate, soil, topography and agricultural activities;

[0040] Figure 4 This is a box plot of the Permutation feature importance obtained for the machine learning model;

[0041] Figure 5 Schematic diagram of the driving mechanism results for each factor obtained by the machine learning model. DETAILED DESCRIPTION

[0042] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0043] The purpose of this invention is to provide a diagnostic method for organic carbon degradation of black soil farmland based on geographic big data and AI, which reduces the dependence on ground surveys and expert experience, and can efficiently and intelligently identify the important influencing factors and their influencing mechanisms of organic carbon degradation in black soil areas at a regional scale.

[0044] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0045] Figure 1 A flow chart of a method provided by an embodiment of the present invention, Figure 2 Schematic diagram of the technical route of an embodiment of the present invention, such as Figure 1 and Figure 2 As shown, the present invention provides a method for diagnosing organic carbon degradation of black soil farmland based on geographic big data and AI, comprising:

[0046] Step 100: Construct a dataset of potential influencing factors of organic carbon degradation in cultivated land in black soil regions;

[0047] Step 200: Spatial correlation of potential influencing factors (continuous variables);

[0048] Step 300: constructing a diagnostic model for organic carbon degradation in cultivated land in black soil areas;

[0049] Step 400: training a diagnostic model for organic carbon degradation of cultivated land in black soil regions based on a dataset of potential influencing factors of organic carbon degradation of cultivated land in black soil regions, and obtaining a trained diagnostic model for organic carbon degradation of cultivated land in black soil regions;

[0050] Step 500: Identifying factors affecting organic carbon degradation based on the trained diagnostic model for organic carbon degradation in cultivated land in black soil areas;

[0051] Step 600: Based on the identification results, analyze the mechanism of action of factors affecting organic carbon degradation.

[0052] In step 100, a dataset of potential influencing factors of organic carbon degradation in cultivated land in black soil regions is constructed, specifically:

[0053] The relationship between multiple potential factors and organic carbon changes was fitted, and the impact of factors on organic carbon changes was analyzed based on this. Among them, the factors include climate factors, soil factors, terrain factors and agricultural activity factors. Climate factors include drought index, multi-year average annual precipitation, multi-year average annual temperature, multi-year average annual wind speed and multi-year average spring wind speed. Soil factors include soil bulk density, cation exchange capacity, soil pH, sand content, silt content, clay content, total nitrogen content, total phosphorus content, total potassium content, initial organic carbon, surface soil temperature and soil parent material. Terrain factors include digital elevation, slope aspect, slope and terrain moisture index. Agricultural activity factors include changes in planting structure and agricultural reclamation management methods.

[0054] For factors with publicly available raster data sources, raster data is obtained based on the Google Cloud computing platform, and the spatial resolution of the raster is resampled;

[0055] For the factors of vector data format, the data is converted from vector to raster based on ArcMap;

[0056] Construct a raster dataset of uniform spatial resolution for all factors;

[0057] The details are shown in Table 1;

[0058] Table 1 Potential influencing factors of soil organic carbon changes in cultivated land used in the present invention

[0059]

[0060]

[0061] In step 200, the spatial correlation of potential influencing factors (only for continuous variables) is analyzed based on the Pearson correlation coefficient to reduce data redundancy information, specifically:

[0062] Based on the spatial distribution data of black soil cultivated land, the raster data of the above climate, soil, topography and agricultural activity factors were spatially masked and cut to retain only the data inside the cultivated land in the black soil area;

[0063] Based on ArcMap, the center point vector data of the 5-kilometer grid covering the black soil area was formed. On this basis, the values ​​of all factors and changes in organic carbon in cultivated land were batch extracted based on Arcpy to form a matrix data table.

[0064] Remove the grid center points with missing values ​​in the matrix data table, and only keep the grid center points with valid values ​​for all factors to calculate the Pearson correlation coefficient and exclude redundant variables;

[0065]

[0066] Among them, cov(X,Y) represents the covariance of variables X and Y, σ X and σ Y are the standard deviations of X and Y respectively.

[0067] In step 300, a diagnostic model for organic carbon degradation in cultivated land in black soil regions is constructed, specifically:

[0068] The present invention will construct an XGBoost model as a diagnostic model for organic carbon degradation in cultivated land in black soil areas and conduct analysis. The prediction target of a decision tree trained in XGBoost is the residual predicted by the previous decision tree, so the final prediction result is determined by the sum of multiple decision trees. These models will be implemented using Python's scikit-learn library and xgboost library.

[0069] In step 400, the organic carbon degradation diagnosis model of cultivated land in black soil region is trained based on the dataset of potential influencing factors of organic carbon degradation of cultivated land in black soil region, and the trained organic carbon degradation diagnosis model of cultivated land in black soil region is obtained and analyzed, specifically:

[0070] Based on the 10-fold cross-validation method, the dataset of potential influencing factors of organic carbon degradation in cultivated land in black soil areas was divided into ten subsets in equal amounts, and each subset was used as a validation set in turn, and the remaining nine subsets were used as training sets to construct ten training set and validation set combinations. For each training set and validation set combination, the XGBoost model was trained based on the training set, and the root mean square error of the model predicted on the validation set was calculated to obtain ten XGBoost models and the corresponding validation set errors. With the goal of minimizing the average validation set error of the ten models, the 10-fold cross-validation process was repeated by the grid search method to determine the optimal parameter combination of the machine learning model. Among them, the parameters considered by the XGBoost model are the maximum depth of the decision tree max_depth and the number of decision trees n_estimators, and the grids are [5,10,15,20] and [40,80,120,160] respectively.

[0071] In step 500, the factors affecting organic carbon degradation are identified based on the trained diagnostic model for organic carbon degradation in cultivated land in black soil areas, specifically:

[0072] For the ten XGBoost models obtained from the 10-fold cross validation, the Permutation method is used to calculate the sensitivity of the prediction results to each factor on the validation set, that is, the increase in the root mean square error of the prediction after the value of the factor is randomly shuffled. Based on the ten models, ten importance results for each factor will be obtained, and the importance of the factors will be ranked according to the median of the ten importance results.

[0073] Step 600, based on the identification results, analyze the mechanism of action of factors affecting organic carbon degradation, specifically:

[0074] For each factor, determine its value range, and take 30 values ​​at equal intervals. While keeping the values ​​of other factors unchanged, set the values ​​of the factor in the data set to these 30 values ​​in turn, and calculate the average prediction value of the model respectively, and obtain the relationship between the factor value and the average prediction result. According to the ten XGBoost models, ten corresponding broken lines will be obtained respectively, and their average broken line and 90% confidence interval will be calculated to obtain the partial dependence diagram, that is, the driving mechanism.

[0075] The experimental results obtained based on the method described in this application are as follows Figure 4 As shown, Figure 4 This is a box plot of the XGBoost model Permutation feature importance, where the features are ranked according to the median of the 10 groups of feature importance obtained from the 10 models. The driving mechanism results of each factor obtained by the model are as follows Figure 5 For each machine learning model, the average of 10 groups of partial dependence graphs (driving mechanisms) obtained by 10 models is a black solid line, and the 90% confidence interval is shaded.

[0076] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0077] The principles and implementation methods of the present invention are described in this article using specific examples. The description of the above embodiments is only used to help understand the method and core idea of ​​the present invention. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A method for diagnosing organic carbon degradation in black soil farmland based on geographic big data and AI, characterized in that: include: Construct a dataset of potential influencing factors of organic carbon degradation in cultivated land in black soil regions; Determine the spatial correlation of potential influencing factors; Construct a diagnostic model for organic carbon degradation in cultivated land in black soil regions; Based on the dataset of potential influencing factors of organic carbon degradation in cultivated land in black soil region, the diagnostic model of organic carbon degradation in cultivated land in black soil region was trained to obtain the trained diagnostic model of organic carbon degradation in cultivated land in black soil region. Based on the trained diagnostic model of organic carbon degradation in cultivated land in black soil region, the influencing factors of organic carbon degradation were identified; Based on the identification results, the mechanism of action of factors affecting organic carbon degradation is analyzed.

2. The method according to claim 1, characterized in that Construct a dataset of potential influencing factors of organic carbon degradation in cultivated land in black soil regions, specifically: Fit the relationship between factors and organic carbon changes, and analyze the impact of factors on organic carbon changes based on this. Among them, factors include climate factors, soil factors, terrain factors and agricultural activity factors. Climate factors include drought index, multi-year average annual precipitation, multi-year average annual temperature, multi-year average annual wind speed and multi-year average spring wind speed. Soil factors include soil bulk density, cation exchange capacity, pH, sand content, silt content, clay content, total nitrogen content, total phosphorus content, total potassium content, initial organic carbon, surface soil temperature and soil parent material. Terrain factors include digital elevation, slope aspect, slope and terrain humidity index. Agricultural activity factors include changes in planting structure and agricultural reclamation management methods. For factors with publicly available raster data sources, raster data is obtained based on the Google Cloud computing platform, and the spatial resolution of the raster is resampled; For the factors of vector data format, the data is converted from vector to raster based on ArcMap; A raster dataset of uniform spatial resolution for all factors was constructed, and a dataset of potential influencing factors of organic carbon degradation in cultivated land in the black soil region was obtained.

3. The method according to claim 2, characterized in that Determine the spatial correlation of potential influencing factors, specifically: The spatial correlation of continuous variables in potential influencing factors was analyzed based on the Pearson correlation coefficient to reduce data redundant information; Based on the spatial distribution data of black soil cultivated land, the raster data of climate, soil, topography and agricultural activity factors were spatially masked and cut, and only the data inside the cultivated land in the black soil area was retained; Based on ArcMap, the center point vector data of the 5-kilometer grid covering the black soil area was formed. On this basis, the values ​​of all factors and changes in organic carbon in cultivated land were batch extracted based on Arcpy to form a matrix data table. Remove the grid center points with missing values ​​in the matrix data table, and only keep the grid center points with valid values ​​for all factors to calculate the Pearson correlation coefficient, excluding redundant variables, which is: Among them, cov(X,Y) represents the covariance of variables X and Y, σ X and σ Y represent the standard deviation of X and Y respectively.

4. The method according to claim 1, characterized in that: Construct a diagnostic model for organic carbon degradation in cultivated land in black soil regions, specifically: The XGBoost model was constructed as a diagnostic model for organic carbon degradation of cultivated land in black soil regions.

5. The method according to claim 4, characterized in that Based on the dataset of potential influencing factors of organic carbon degradation in cultivated land in black soil regions, the diagnostic model of organic carbon degradation in cultivated land in black soil regions was trained to obtain the trained diagnostic model of organic carbon degradation in cultivated land in black soil regions, which is as follows: Based on the 10-fold cross-validation method, the dataset of potential influencing factors of organic carbon degradation in cultivated land in black soil areas was divided into ten subsets in equal amounts, and each subset was used as a validation set in turn, and the remaining nine subsets were used as training sets to construct ten combinations of training sets and validation sets. For each combination of training sets and validation sets, the XGBoost model was trained based on the training set, and the root mean square error of the model predicted on the validation set was calculated to obtain ten XGBoost models and the corresponding validation set errors. With the goal of minimizing the average validation set error of the ten models, the 10-fold cross-validation process was repeated through the grid search method to determine the optimal parameter combination of the machine learning model. The XGBoost model parameters are the maximum depth of the decision tree max_depth and the number of decision trees n_estimators, with grids of [5, 10, 15, 20] and [40, 80, 120, 160] respectively.

6. The method according to claim 5, characterized in that Based on the trained black soil organic carbon degradation diagnostic model, the influencing factors of organic carbon degradation are identified, specifically: For the ten XGBoost models, the SHAP method is used to calculate the contribution of each factor to the prediction result of each sample in the training set. The absolute value of the average contribution of a factor to all samples in the training set is taken as the importance of the factor. According to the model, ten importance results of each factor will be obtained, and the importance of the factors will be ranked according to the median of the ten importance results.

7. The method according to claim 6, characterized in that Based on the identification results, the mechanism of influencing factors of organic carbon degradation in cultivated land in black soil regions was analyzed, specifically: For each factor, determine its value range, and take 30 values ​​at equal intervals. While keeping the values ​​of other factors unchanged, set the values ​​of the factor in the data set to these 30 values ​​in turn, and calculate the average prediction value of the model respectively, and obtain the relationship between the factor value and the average prediction result. According to the XGBoost model, ten corresponding broken lines will be obtained respectively, and their average broken line and 90% confidence interval will be calculated to obtain the partial dependence diagram, that is, the driving mechanism.

Images