A soil carbon component prediction method based on logarithmic ratio transformation and memory learning model

Abstract

The present invention relates to a soil carbon component prediction method based on logarithmic ratio transformation and memory learning model, which belongs to the field of soil detection technology, including: performing logarithmic ratio transformation on the content of MAOC component and the content of POC component to obtain soil carbon component logarithmic ratio transformation data; obtaining soil MIR spectrum data; generating a training set and a validation set using the soil MIR spectrum data obtained by the acquired soil carbon component logarithmic ratio transformation data; using the training set, establishing a regression model of soil spectral characteristics and soil carbon components based on a memory learning algorithm, and applying it to the validation set to obtain a prediction result; performing an inverse logarithmic ratio transformation on the prediction result to obtain a predicted value of the content of two organic carbon components, comparing the predicted value with the measured values ​​of the two organic carbon components, evaluating the accuracy of the regression model prediction, and using the regression model to predict the soil samples collected in real time. The prediction accuracy of the soil carbon component based on a large-scale soil spectrum database can be effectively improved.

Description

Technical Field

[0001] The invention relates to a soil carbon component prediction method based on logarithmic ratio transformation and memory learning model, and belongs to the technical field of soil detection. Background Art

[0002] Soil organic carbon (SOC) is a core indicator for measuring soil fertility, and its dynamic changes play a key role in agricultural productivity, ecosystem stability and climate regulation. The chemical composition and structure of SOC are complex, and it is composed of carbon components with different stabilities, decomposition rates and functions. In recent years, more and more studies have divided SOC into mineral-associated organic carbon (MAOC) and particulate organic carbon (POC) components with different structures, compositions and functions, which provides a new perspective for in-depth understanding of the stability and storage mechanism of soil carbon pools and their diverse ecological functions in soil.

[0003] The detection method of MAOC and POC content in soil is mainly based on the traditional physical screening method in the laboratory. After screening, the soil components with a particle size of less than 63 μm (i.e., clay particles + soil particles) are used to determine MAOC. This component is tightly bound to soil minerals, has high stability, and can be stored in the soil for a long time; the components larger than 63 μm are used to determine POC. This component is composed of incompletely decomposed plant residues and is an unstable short-term carbon pool that is easily affected by environmental disturbances. At present, the soil carbon component detection technology based on traditional physical screening methods has the disadvantages of low efficiency and high cost, which limits the study of the large-scale spatiotemporal evolution of soil carbon components. Therefore, it is urgent to establish a high-throughput detection method for soil MAOC and POC to strengthen the cross-scale research on the structure and function of soil carbon pools.

[0004] In recent years, Fourier transform infrared spectroscopy sensing technology has been widely used in the field of quantitative soil science, and mid-infrared (MIR) spectral inversion modeling has been used to achieve rapid and high-precision prediction of soil properties. For soil carbon components, soil MIR spectra can effectively capture the characteristic absorption peaks of soil organic components and reflect the molecular vibration information of these components. Therefore, establishing a physical response model between MIR and soil carbon components is conducive to the acquisition of large-scale and high-density soil carbon component data. At present, there is little research on spectral prediction methods for soil carbon components, and the existing problems can be summarized into two aspects:

[0005] First, previous studies have all conducted independent prediction modeling for different soil carbon components. The prediction accuracy of the modeling results for MAOC is high, but the prediction accuracy for POC is not ideal. This is because POC is a highly dynamic and unstable carbon component, and its dynamic changes are particularly affected by extraterrestrial dynamic processes such as soil erosion and tillage. In addition, the prediction results of separate modeling will cause the sum of the MAOC and POC percentages of some samples to be far greater than 100%, which is inconsistent with the reality that it is compositional data.

[0006] Secondly, most previous studies have used global modeling to perform spectral prediction modeling on soil properties in a specific study area. This "global modeling" approach is suitable for small-scale research, but when the spatial scale becomes larger, the mapping relationship between soil properties and spectral absorption characteristics becomes complicated, resulting in a lower prediction accuracy of the global prediction model for local variations in soil carbon components. Summary of the invention

[0007] In view of the problems that traditional detection methods of soil carbon components are cumbersome and the POC component spectral prediction model has low accuracy, the present invention provides a soil carbon component prediction method based on logarithmic ratio transformation and memory learning model.

[0008] The present invention is achieved in this way.

[0009] A soil carbon component prediction method based on logarithmic ratio transformation and memory learning model, the method comprising:

[0010] S1: Collect soil samples, conduct physical screening experiments on SOC components, and use carbon and nitrogen analyzers to determine the content of MAOC components and POC components;

[0011] S2 The content of MAOC component and the content of POC component were logarithmically transformed to obtain the logarithmic ratio transformation data of soil carbon component;

[0012] S3 uses Fourier transform infrared spectrometer to obtain soil MIR (infrared) spectral data, and performs preprocessing on the soil MIR spectral data including resampling, smoothing and scatter correction;

[0013] S4 generating a training set and a validation set using the soil carbon component logarithmic ratio transformation data obtained in step S2 and the soil MIR spectrum data obtained in step S3;

[0014] S5. Using the training set in step S4, taking the soil MIR spectral data as the independent variable and the logarithmic ratio transformation result of the soil carbon component as the dependent variable, a regression model of soil spectral characteristics and soil carbon components is established by using a memory learning algorithm (MBL), and applied to the validation set to obtain prediction results;

[0015] S6: performing an inverse logarithmic ratio transformation on the prediction result in step S5 to obtain the predicted values ​​of the content of the MAOC component and the content of the POC component, comparing the predicted values ​​with the measured values ​​of the two organic carbon components, and evaluating the accuracy of the regression model prediction;

[0016] S7 uses regression models to predict the content of MAOC components and POC components in soil samples collected in real time.

[0017] Furthermore, S1 specifically includes:

[0018] A 0.5% sodium hexametaphosphate solution was added to the air-dried soil sample that was sieved to <2 mm, and the soil sample was placed on a shaker for oscillation and dispersion. The dispersed soil sample was rinsed with deionized water on a 63 μm sieve, and the components <63 μm were collected for the determination of the MAOC component, and the components >63 μm were collected for the determination of the POC component. The two parts of the soil samples were dried to constant weight. After screening, the two components of the soil samples were ground through a 150 μm soil sieve, and hydrochloric acid with a mass percentage concentration of 10% was used to remove inorganic carbon. The content of the MAOC component and the content of the POC component were determined using a carbon-nitrogen analyzer, respectively.

[0019] Further, performing logarithmic ratio transformation on the content of the MAOC component and the content of the POC component includes: the logarithmic ratio transformation formula is:

[0020] ,

[0021] in, It is the logarithmic ratio function of the content of MAOC component and the content of POC component.

[0022] Furthermore, S5 specifically includes:

[0023] S51 uses the memory learning algorithm to perform spectral prediction modeling of soil carbon components. For each sample to be predicted in the validation set, a training subset is selected in the training set according to the spectral similarity calculation;

[0024] S52 The process of selecting training subsets by using spectral similarity calculation is carried out through the k-nearest neighbor algorithm, that is, a subset consisting of k training samples is selected for each sample to be predicted, and a regression model is established for each sample to be predicted using the weighted average partial least squares method. The maximum value of k is less than the total number of samples in the training set, and the minimum value is set to 20. Within the k value variation range, the nearest neighbor cross-validation (NNv) method is used to evaluate the root mean square error of the regression model corresponding to different k values. The k value corresponding to the regression model with the smallest root mean square error is selected as the optimal regression model for obtaining the final prediction result. .

[0025] Furthermore, the spectral similarity calculation includes:

[0026] S511 uses principal component analysis to reduce the dimension of soil MIR spectral data used for training set and establishes the projection model as follows:

[0027] ,

[0028] in The soil MIR spectral data input for the training set was decomposed into two matrices in principal component analysis: is the score matrix, is the loading matrix, represents the reconstruction error;

[0029] S512 through load matrix The soil MIR spectral data of the sample to be predicted Project it onto the projection space of the same principal component to generate the soil MIR spectral data of the sample to be predicted The score matrix :

[0030] ,

[0031] in, for The transposed matrix of

[0032] S513 calculates the Mahalanobis distance between the sample to be predicted and each sample in the training set in the projection space, and uses the Mahalanobis distance as the evaluation standard for spectral similarity. The Mahalanobis distance is based on the observed value of the sample to be predicted. In the score matrix The projected score vector in and training set sample observations Projected score vector in the score matrix S The calculation results are:

[0033] ,

[0034] in is the number of principal components selected using the cumulative variance method, is the projection score vector and The inverse matrix of the covariance matrix is yes The transposed matrix of .

[0035] Furthermore, the formula for the inverse logarithmic ratio transformation is:

[0036] ,

[0037] ,

[0038] in, is the total soil organic carbon content of the soil sample, is the predicted value of mineral-bound organic carbon content, is the predicted value of particulate organic carbon content. Compared with the prior art, the present invention has the following beneficial effects:

[0039] Compared with the traditional laboratory soil organic carbon physical screening method, the present invention can quickly and non-destructively obtain high-precision MAOC component content and POC component content data. The two soil carbon components are regarded as compositional data for predictive modeling through the logarithmic ratio transformation method, which can effectively improve the POC prediction accuracy while keeping the sum of the two components at 100%, thereby achieving the purpose of reducing the overall prediction error and eliminating the abnormal values ​​generated by the prediction. The local regression model based on the memory learning algorithm fully considers the nonlinear relationship between the soil MIR spectral characteristics and the carbon components, and locally simplifies the global complex relationship between soil properties and spectral characteristics, which can effectively improve the soil carbon component prediction accuracy based on the large-scale soil spectral database, and provide a convenient and reliable data acquisition method for research on soil carbon cycle and soil degradation. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 A flow chart of a soil carbon component prediction method based on logarithmic ratio transformation and memory learning model provided in an embodiment of the present invention;

[0041] Figure 2 The principal component analysis provided in the embodiment of the present invention distinguishes the soil spectral variation graphs of the training set and the validation set;

[0042] Figure 3 A schematic diagram of a memory learning model provided by an embodiment of the present invention;

[0043] Figure 4 A graph showing cross-validation results of training subsets of different sizes for the memory learning model provided in an embodiment of the present invention;

[0044] Figure 5 Scatter plots of (a) mineral-bound organic carbon and (b) particulate organic carbon measured values ​​and model predicted values ​​provided in embodiments of the present invention. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0046] like Figure 1As shown, a soil carbon component prediction method based on logarithmic ratio transformation and memory learning model includes the following steps:

[0047] S1 collects soil samples, conducts physical screening experiments on SOC components, and uses a carbon-nitrogen analyzer to determine the content of MAOC components and the content of POC components: 200 samples of cultivated land surface soil were collected in this embodiment. The naturally air-dried soil was fully mixed and passed through a 2 mm soil sieve. 8g of soil sample <2 mm was added to 40 ml of a 0.5% sodium hexametaphosphate solution by mass percentage, and the soil was completely dispersed by shaking at 200 rpm on a reciprocating shaker for 18 hours. The dispersed sample was then rinsed on a 63 μm sieve with deionized water, and components <63 μm were collected for determining the content of MAOC components, and components >63 μm were collected for determining the content of POC components. Both components were dried in a 60°C oven to constant weight, and the samples were weighed. A mass recovery rate of more than 97% was considered acceptable. After the SOC components were screened, the soil samples of the two components were ground through a 150 μm soil sieve, and the inorganic carbon was removed using hydrochloric acid with a mass percentage concentration of 10%. The content of the MAOC component and the content of the POC component were determined using a carbon-nitrogen analyzer. The total SOC is the sum of the contents of the POC and MAOC components.

[0048] S2 Introducing the logarithmic ratio transformation method to process the data of MAOC components and POC components can combine the two components into a whole for predictive modeling while retaining their inherent structure and internal relationship. This modeling strategy can not only improve the POC prediction accuracy, but also keep the sum of the two components at 100%, thereby achieving the purpose of reducing the overall prediction error and eliminating the outliers generated by the prediction. The measured content data of the MAOC component and the measured content data of the POC component are converted into logarithmic ratios using the following formula:

[0049] ,

[0050] in, It is the logarithmic ratio function of the content of MAOC component and the content of POC component.

[0051] S3 Scan the soil sample with a Fourier transform infrared spectrometer to obtain soil MIR spectral data, and perform resampling, smoothing and scatter correction preprocessing on the soil MIR spectral data to achieve the purpose of denoising and improving the quality of soil MIR spectral data; In this embodiment, the soil sample is scanned with an ALPHA II Fourier transform infrared spectrometer from Bruker, Germany, at a resolution of 1 cm -1 The resolution of 4000 to 400 cm -1The mid-infrared spectrum of the soil was obtained by scanning 24 times. The average value of 24 repeated scans was taken as the soil MIR spectrum data of each sample to improve the signal-to-noise ratio. Before the spectrum scan, the background value was collected for data correction. The spectrum data scanning, recording and correction were performed using Opus 8.1 software;

[0052] The original soil MIR spectral data were preprocessed to retain the 4000-600 cm -1 Spectral range, according to 16 cm -1 The spectral data were resampled by 1000 nm interval units; then the Savitzky-Golay filter was used with a window size of 15 cm -1 The spectral curve was smoothed by a second-order polynomial to reduce baseline variations; finally, multivariate scattering correction was performed.

[0053] S4 Based on the soil MIR spectral data obtained in step S3, the conditional Latin hypercube sampling algorithm is used to divide the entire data set into a training set (80%, 160) and a validation set (20%, 40) in a ratio of 8:2, as shown in Figure 2 As shown in Figure 2, the principal component analysis results show that the soil MIR spectral data of the training set and the validation set are highly similar.

[0054] S5 Utilize the 160 training set samples in step S4, take the soil MIR spectral data as the independent variable, and the logarithmic ratio transformation result of the soil carbon component as the dependent variable, and use the memory learning algorithm to establish a regression model of soil spectral characteristics and soil carbon components, and apply it to 40 validation set samples to obtain prediction results. The present invention uses a memory learning algorithm to select a training subset with the highest spectral feature similarity from the training set for each sample to be predicted, and establishes a regression method of local (local) prediction models one by one. The essence of this function approximation method is to locally simplify the global complex relationship between soil properties and spectral characteristics, so as to achieve the purpose of improving the prediction accuracy of soil carbon component spectra, and is particularly suitable for sample sets with strong spatial heterogeneity of soil properties. Figure 3 This is a schematic diagram of the MBL model, showing the core process of the method. For each validation set sample point, the MBL model selects several samples with the closest spectral features from the training set as the nearest neighbor value to fit and construct a specific local regression model. For example, Figure 3 The spectral data of 160 training set samples and 40 validation set samples were reduced in dimension and projected into space. Three validation set samples were taken as examples to show their respective nearest neighbor sample selections.

[0055] The specific steps are as follows:

[0056] S51 Spectral prediction modeling of soil carbon components is performed using a memory learning algorithm. For each sample to be predicted in the validation set, a training subset is selected in the training set according to the spectral similarity principle. The calculation steps for the spectral similarity between the sample to be predicted and the sample in the training set include:

[0057] S511 uses the soil MIR spectral data of the training set and the logarithmic ratio transformation result data of the soil carbon component, and uses principal component analysis to reduce the dimension of the spectral data and establish a projection model as follows:

[0058] ,

[0059] in The soil MIR spectral data input for the training set was decomposed into two matrices in principal component analysis: is the score matrix, is the loading matrix, represents the reconstruction error;

[0060] S512 through load matrix The soil MIR spectral data of the sample to be predicted Project it onto the projection space of the same principal component to generate the soil MIR spectral data of the sample to be predicted The score matrix :

[0061] ,

[0062] in, for The transposed matrix of

[0063] S513 calculates the Mahalanobis distance between the sample to be predicted and each sample in the training set in the projection space, and uses this as an evaluation standard for spectral similarity. The Mahalanobis distance is based on the observed value of the sample to be predicted. In the score matrix The projected score vector in and training set sample observations Projected score vector in the score matrix S The calculation results are:

[0064] ,

[0065] in is the number of principal components selected using the cumulative variance method, is the projection score vector and The inverse matrix of the covariance matrix is yes The transposed matrix of .

[0066] S52 The process of selecting training subsets by using spectral similarity calculation is carried out by the k-nearest neighbor algorithm, that is, a subset consisting of k training samples is selected for each sample to be tested, and a model is established for each sample to be predicted using the weighted average partial least squares method. The maximum and minimum values ​​of k are set to 160 and 20, respectively, with an interval of 10. Within the range of k value variation, the nearest neighbor cross validation (NNv) method is used to evaluate the root mean square error (RMSE) of the regression model corresponding to different k values, as shown in Figure 4 As shown in Figure 2, when the number of samples in the selected training subset is 100, the RMSE is the smallest, so k=100 is selected as the optimal model to obtain the prediction results for 40 validation set samples. .

[0067] S6: Perform an inverse logarithmic ratio transformation on the prediction results in step S5 to obtain the predicted values ​​of MAOC and POC content, and compare the predicted values ​​with the actual measured values ​​of MAOC and POC of the 40 validation set samples obtained in step S1 to evaluate the prediction accuracy of the model.

[0068] The specific steps are as follows:

[0069] S61 To obtain the predicted values ​​of MAOC and POC as non-converted values, the following formula is used to calculate the predicted results. Perform an inverse log-ratio transformation:

[0070] ,

[0071]

[0072] where SOC is the total soil organic carbon content of the soil sample; is the predicted value of mineral-bound organic carbon content, is the predicted value of particulate organic carbon content;

[0073] What the S62 will gain and The predicted results are compared with the corresponding measured values ​​in step S1, and the coefficient of determination (R 2 The prediction accuracy of the regression model was evaluated by three parameters: R ), root mean square error (RMSE) and the ratio of the standard deviation of the validation set to the standard deviation of the prediction set (RPD). 2 It is used to indicate the proportion of the variance of the predicted value of soil carbon components explained by soil spectra in the regression model. The closer the value is to 1, the better the fit between the actual data and the predicted data. RMSE and RPD are used to evaluate the prediction accuracy and stability of the regression model. The lower the RMSE value and the higher the RPD value, the better the performance of the regression model. The formula is as follows:

[0074] ,

[0075] ,

[0076] ,

[0077] in yes The measured value of the point, yes The predicted value of the point, is the average of the measured values, is the sample size.

[0078] like Figure 5 As shown in FIG. 1 , the predicted values ​​and measured values ​​of the two soil carbon components are evenly distributed on or on both sides of the 1:1 line, indicating that the method of the present invention can be used for the quantitative detection of MAOC components and POC components. Figure 5 As shown in (a) in the figure, the determination coefficient R of MAOC component prediction is 2 is 0.97, the root mean square error RMSE is 0.07 g / kg, and the ratio of standard deviation to the standard deviation of the prediction set RPD is 6.3, indicating that the model has high prediction accuracy; Figure 5 As shown in (b), the determination coefficient R of POC component prediction 2 The root mean square error (RMSE) is 0.07 g / kg, and the ratio of the standard deviation to the standard deviation of the prediction set (RPD) is 2.36. The prediction accuracy of the model is relatively ideal.

[0079] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A soil carbon component prediction method based on logarithmic ratio transformation and memory learning model, characterized in that: The method includes: S1: Collect soil samples, conduct physical screening experiments on SOC components, and use carbon and nitrogen analyzers to determine the content of MAOC components and POC components; S2 The content of MAOC component and the content of POC component were logarithmically transformed to obtain the logarithmic ratio transformation data of soil carbon component; the logarithmic ratio transformation formula is: ， in, It is the logarithmic ratio function of the content of MAOC component and the content of POC component; S3 uses Fourier transform infrared spectrometer to obtain soil MIR spectral data, and performs preprocessing including resampling, smoothing and scatter correction on the soil MIR spectral data; S4 generates a training set and a validation set using the soil carbon component logarithmic ratio transformation data obtained in step S2 and the soil MIR spectrum data obtained in step S3; S5 uses the training set in step S4, takes the soil MIR spectral data as the independent variable, and the logarithmic ratio transformation result of the soil carbon component as the dependent variable, establishes a regression model of soil spectral characteristics and soil carbon components by using a memory learning algorithm, and applies it to the validation set to obtain prediction results; S6 performs an inverse logarithmic ratio transformation on the prediction result in step S5 to obtain the predicted values ​​of the content of the MAOC component and the content of the POC component, compares the predicted values ​​with the measured values ​​of the two organic carbon components, and evaluates the accuracy of the regression model prediction; The formula for the inverse log ratio transformation is: ， ， in, is the total soil organic carbon content of the soil sample, is the predicted value of mineral-bound organic carbon content, is the predicted value of particulate organic carbon content; The final prediction result; S7 uses regression models to predict the content of MAOC components and POC components in soil samples collected in real time.

2. The soil carbon component prediction method based on logarithmic ratio transformation and memory learning model according to claim 1 is characterized in that: S1 specifically includes: A 0.5% sodium hexametaphosphate solution was added to the air-dried soil sample that was sieved to <2 mm, and the soil sample was placed on a shaker for oscillation and dispersion. The dispersed soil sample was rinsed with deionized water on a 63 μm sieve, and the components <63 μm were collected for the determination of the MAOC component, and the components >63 μm were collected for the determination of the POC component. The two parts of the soil samples were dried to constant weight. After screening, the two components of the soil samples were ground through a 150 μm soil sieve, and hydrochloric acid with a mass percentage concentration of 10% was used to remove inorganic carbon. The content of the MAOC component and the content of the POC component were determined using a carbon-nitrogen analyzer, respectively.

3. The soil carbon component prediction method based on logarithmic ratio transformation and memory learning model according to claim 1 is characterized in that: S5 specifically includes: S51 uses the memory learning algorithm to perform spectral prediction modeling of soil carbon components. For each sample to be predicted in the validation set, a training subset is selected in the training set according to the spectral similarity calculation; S52 The process of selecting a training subset using spectral similarity calculation includes: using the k-nearest neighbor algorithm, that is, selecting a subset consisting of k training samples for each sample to be predicted, and using the weighted average partial least squares method to establish a regression model for each sample to be predicted, the maximum value of k is less than the total number of samples in the training set, and the minimum value is set to 20. Within the k value variation range, the nearest neighbor cross-validation method is used to evaluate the root mean square error of the regression model corresponding to different k values. The k value corresponding to the regression model with the smallest root mean square error is selected as the optimal regression model to obtain the final prediction result. .

4. The soil carbon component prediction method based on logarithmic ratio transformation and memory learning model according to claim 3 is characterized in that: Spectral similarity calculation includes: S511 uses principal component analysis to reduce the dimension of soil MIR spectral data used for training set and establishes the projection model as follows: ， in The soil MIR spectral data input for the training set was decomposed into two matrices in principal component analysis: is the score matrix, is the loading matrix, represents the reconstruction error; S512 through load matrix The soil MIR spectral data of the sample to be predicted will be used Project it onto the projection space of the same principal component to generate the soil MIR spectral data of the sample to be predicted The score matrix : ， in, for The transposed matrix of S513 calculates the Mahalanobis distance between the sample to be predicted and each sample in the training set in the projection space, and uses the Mahalanobis distance as the evaluation standard for spectral similarity. The Mahalanobis distance is based on the observed value of the sample to be predicted. In the score matrix The projected score vector in and training set sample observations Projected score vector in the score matrix S The calculation results are: ， in is the number of principal components selected using the cumulative variance method, is the projection score vector and The inverse matrix of the covariance matrix is yes The inverse matrix of .

Images