A soil organic carbon prediction method based on an attention-enhanced TabM model
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- GUILIN UNIVERSITY OF TECHNOLOGY
- Filing Date
- 2026-04-01
- Publication Date
- 2026-07-21
Smart Images
Figure CN122432478A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of remote sensing information processing, geographic information systems and artificial intelligence, and specifically to a method for predicting soil organic carbon based on the attention-enhanced TabM model, which is applicable to the estimation of regional-scale soil organic carbon content and spatial distribution analysis under the condition of fusion of multi-source remote sensing data and environmental factors. Background Technology
[0002] Soil organic carbon (SOC) is an important indicator for measuring soil quality and ecosystem carbon cycling. Its spatial distribution characteristics are of great significance for regional carbon storage assessment, land management, and ecological environment regulation. Traditional methods for measuring soil organic carbon mainly rely on field sampling and laboratory analysis. Although these methods are highly accurate, they are costly and time-consuming when applied to large-scale areas, making it difficult to achieve continuous dynamic monitoring.
[0003] With the development of multi-source remote sensing technology, soil organic carbon inversion using optical and radar remote sensing data has become a research hotspot. However, multi-source remote sensing data are usually characterized by high dimensionality, multiple scales, and high redundancy. Traditional machine learning methods struggle to fully model complex nonlinear relationships when processing high-dimensional continuous tabular data, and have limited ability to express the interaction relationships between features, thus affecting prediction accuracy and stability.
[0004] To address the above issues, this invention unifies the processing of multi-source remote sensing data and environmental factors. Based on feature construction and screening, it introduces ensemble learning to fuse the prediction results of multiple models, ultimately forming a soil organic carbon inversion system based on multi-source remote sensing data and ensemble learning, thereby improving the stability and applicability of soil organic carbon inversion results. Summary of the Invention
[0005] The purpose of this invention is to provide a soil organic carbon prediction method based on the attention-enhanced TabM model. By performing unified preprocessing on multi-source remote sensing data and environmental factors, and introducing a feature interaction attention mechanism and an attention weighted aggregation mechanism on the basis of the TabM piecewise linear embedding structure, the method can model complex nonlinear relationships and improve the accuracy and stability of soil organic carbon prediction.
[0006] To achieve the above objectives, the technical solution adopted by the present invention includes the following steps: Step S1. Data acquisition and preprocessing: Acquire environmental factor data corresponding to soil sampling points in the study area and multi-source remote sensing data matching their spatial location and sampling time. After preprocessing the multi-source remote sensing data, perform spatial matching and feature extraction, and perform quantile normalization on all input features to divide the training set and test set. Step S2. Construct piecewise linear embedding based on the TabM framework: Calculate the boundary parameters of the quantile intervals of each feature based on the training set, divide each feature into K intervals, generate interval weight vectors through piecewise linear interpolation, and obtain the embedding vector through learnable mapping; Step S3. Feature Interaction Enhancement: Based on the feature embedding matrix, a multi-head self-attention feature interaction module is constructed to model the interaction between different environmental factors and obtain the interaction-enhanced feature representation; Step S4. Attention-weighted aggregation: Attention-weighted aggregation is performed on the interaction enhancement feature representation. By calculating the importance weight of each feature and performing a weighted sum, a sample-level aggregated representation vector is formed. Step S5. Regression Prediction: Input the sample-level aggregated representation vector into the multilayer perceptron regression network for prediction calculation, and output the predicted value of soil organic carbon; Step S6. Model Training and Parameter Optimization: Construct the mean squared error loss function, use the AdamW optimization algorithm to iteratively update the model parameters, and combine the adaptive learning rate decay strategy and early stopping mechanism to complete model training.
[0007] The data acquisition and preprocessing step S1 includes the following steps: Step S11: This study collected environmental factor information from on-site soil sample points and, in conjunction with cloud-based remote sensing platforms such as Google Earth Engine, simultaneously acquired multi-source satellite remote sensing data that perfectly matched the geographic coordinates and sampling time of the sample points. The remote sensing data used included optical imagery and synthetic aperture radar data.
[0008] Step S12: Preprocess the multi-source remote sensing data, including cloud and shadow removal, atmospheric correction and time synthesis processing for optical remote sensing data, and radiometric correction, geometric correction and terrain correction processing for radar remote sensing data, in order to eliminate observation errors and terrain effects. Step S13: Based on the spatial location of the soil sampling points, perform spatial matching and feature extraction on the preprocessed multi-source remote sensing data to obtain remote sensing feature data corresponding to each sampling point, and use it together with environmental factor data as the data basis for subsequent analysis.
[0009] Step S2, which constructs a piecewise linear embedding based on the TabM framework, includes the following steps: Step S21: Calculate the quantile interval boundary parameters for each input feature on the training set, and divide each feature into... 1 consecutive interval, of which The number of segments; Step S22: For any feature value, perform linear interpolation calculation based on its interval to generate a value of length [length missing]. interval weight vector ; Step S23: Using the learnable weight matrix With bias vector By performing a linear mapping on the interval weight vector, the feature embedding vector is obtained: Step S24: Combine all feature embedding vectors to form a feature embedding matrix for subsequent feature interaction modeling.
[0010] The feature interaction enhancement step S3 includes the following steps: Step S31: Map the feature embedding matrix to the query matrix through a linear transformation. Key matrix Sum matrix ,in , , , , , The learnable parameter matrix; Step S32: Calculate the interaction weights between features using the following formula: in This represents the correlation matrix between different feature embeddings. Scaling factor The function is used to normalize the relevance, obtain the attention weights of each feature on other features, and apply the weights to the value matrix. The weighted result is obtained; Step S33: Fuse the attention-weighted result with the original embedding representation to obtain the interaction-enhanced feature representation E': in This represents an enhanced embedding representation that incorporates dynamic dependencies between features, thereby enabling dynamic correlation modeling between different environmental factors; Step S4, attention-weighted aggregation, includes the following steps: Step S41: Assess the importance of the interactive enhancement feature representations to obtain the weight coefficients of each feature. ,in Indicates the first The contribution of each environmental factor to the prediction of the current sample is determined by the weights calculated by the scoring function and then normalized. Step S42: Perform a weighted summation according to the following formula: Obtain the sample-level representation vector ,in For the number of features, For the first The interaction-enhanced embedding representation of each feature, steps; The regression prediction step S5 includes the following steps: Step S51: Input the sample-level representation vector Z into the multilayer perceptron regression network.
[0011] Step S52: Perform feature abstraction and transformation through multi-layer linear mapping and non-linear activation functions.
[0012] Step S53: Obtain a single continuous output value through linear mapping of the output layer. This serves as a prediction result for soil organic carbon.
[0013] The model training and parameter optimization step S5 includes the following steps: Step S61: Construct the mean squared error loss function in This indicates the number of samples used in the training process. For the first The actual soil organic carbon observation value of each sample The loss function is the corresponding predicted value output by the model. It is used to measure the overall prediction bias of the model and serve as the optimization objective by calculating the square of the error between the predicted value and the true value and taking the average. Step S62: Iteratively update the model parameters using the AdamW optimization algorithm.
[0014] Step S63: Combine the adaptive decay strategy of learning rate to dynamically adjust the learning rate according to the change of validation set loss.
[0015] Step S64: Set an early stopping mechanism to terminate training when the validation set loss does not decrease in several consecutive training rounds, thereby completing model training.
[0016] The present invention has the following beneficial effects and advantages: (1) By integrating multi-source remote sensing information and environmental factor data, the soil organic carbon inversion model has been enhanced to represent and adapt to heterogeneous surface environments. (2) The continuous numerical features are piecewise linearly mapped by the TabM piecewise linear embedding structure, which enhances the nonlinear expressive ability while maintaining the continuity of the original features, so that high-dimensional tabular data can be represented in a unified embedding space. (3) By constructing a multi-head self-attention mechanism, the correlation between different environmental factors is modeled, dynamic interaction and information fusion between features are realized, and the model's ability to express the nonlinear dependency structure of multi-source data is enhanced. (4) This method can provide a stable and reliable technical solution for the spatial distribution of soil organic carbon at the regional scale and has good application prospects. Attached Figure Description
[0017] Figure 1 This is a flowchart illustrating the steps of a soil organic carbon prediction method based on the attention-enhanced TabM model of the present invention.
[0018] Figure 2 This is a flowchart illustrating the implementation of the present invention.
[0019] Figure 3 This is a block diagram of the attention-enhanced TabM model proposed in this invention.
[0020] Marked in the image: S1 - Data Acquisition and Preprocessing; S2 - Construction of Piecewise Linear Embeddings Based on the TabM Framework; S3 - Feature Interaction Enhancement; S4 - Attention-Weighted Aggregation; S5 - Regression Prediction; S6 - Model Training and Parameter Optimization. Detailed Implementation
[0021] Example: like Figure 1 As shown, the technical solution of the present invention includes six steps: data acquisition and preprocessing, constructing piecewise linear embedding based on the TabM framework, feature interaction enhancement, attention-weighted aggregation, regression prediction, model training and parameter optimization.
[0022] Step S1, data acquisition and preprocessing, involves acquiring multi-source remote sensing data corresponding to the spatial location and sampling time of soil sampling points on a cloud computing platform. This multi-source remote sensing data includes optical and radar remote sensing data. The optical remote sensing data undergoes cloud and shadow removal, atmospheric correction, and time-series synthesis processing. The radar remote sensing data undergoes radiometric, geometric, and topographic correction processing to eliminate observation errors and topographic influences. Based on the spatial coordinates of the soil sampling points, spatial matching and feature extraction are performed on the preprocessed multi-source remote sensing data to obtain remote sensing feature data corresponding to each sampling point. This data, along with environmental factor data, constitutes the input dataset for soil organic carbon inversion. Subsequently, a quantile mapping function is calculated based on the feature distribution of the training set. Quantile normalization is applied to all input features, mapping them to a normal distribution space. The same mapping parameters are applied to the test set features to ensure data distribution consistency and model training stability. Step S2 involves constructing piecewise linear embeddings based on the TabM framework: After data preprocessing, a piecewise linear embedding module is constructed based on the TabM model; firstly, for each feature on the training set... Calculate the boundary parameters of its quantile intervals, assuming each feature is divided into 1 / 2 quantile intervals. There are several intervals, among which This indicates the number of segments, used to control the precision and expressive power of feature discretization; for any feature value, linear interpolation is performed based on its interval to generate a segment of length [length missing]. interval weight vector Subsequently, a learnable weight matrix was used. With bias vector Perform a linear mapping to obtain the embedding vector: And construct the feature embedding matrix: in For the sample size, For the number of features, For the embedded dimension.
[0023] Step S3, Feature Interaction Enhancement, involves constructing a multi-head self-attention mechanism based on the feature embedding matrix obtained from piecewise linear embedding, mapping the embedding matrix to the query matrix through linear transformations. Key matrix Sum matrix ,in , , , , , For learnable parameter matrix, For the embedding dimension; the interaction weights between features are calculated using the following formula: in This represents the correlation matrix between different feature embeddings. Scaling factor The function is used to normalize the relevance, obtain the attention weights of each feature on other features, and apply the weights to the value matrix. The weighted result is obtained; finally, the interaction-enhanced feature representation is obtained: in This represents an enhanced embedding representation that integrates dynamic dependencies between features, thereby enabling adaptive correlation modeling between different environmental factors and improving the model's ability to express complex nonlinear coupling relationships. Step S4, attention-weighted aggregation, involves scoring the importance of the interaction-enhanced feature representations and assigning weight coefficients to each feature. ,in Indicates the first The contribution of each environmental factor to the prediction of the current sample is determined by the weights calculated from the scoring function and then normalized; subsequently, the formula is applied: The sample-level representation vector is obtained by weighted summation of the feature embedding vectors. ,in For the number of features, For the first The interactive enhanced embedding representation of each feature, the weighted summation process is essentially a selective aggregation of different feature information according to feature importance, thereby realizing feature information compression and key feature enhancement, improving the model's expressive power and interpretability for soil organic carbon prediction; Step S5, regression prediction, involves using the attention-weighted aggregation to obtain the sample-level representation vector. The sample-level representation vector is fed into a multilayer perceptron regression network for mapping calculation. Specifically, the sample-level representation vector is first mapped to a high-dimensional feature space through a first linear transformation and then normalized and nonlinearly activated to extract high-order regression features. It is then further reconstructed and abstracted through a second linear transformation, normalization, and nonlinear activation. Finally, the predicted value of soil organic carbon is obtained as a single continuous output through the linear regression mapping of the output layer, thereby realizing end-to-end regression prediction from the sample-level representation vector to the soil organic carbon inversion result.
[0024] Step S6, model training and parameter optimization, involves constructing the mean squared error loss function. in This indicates the number of samples used in the training process. For the first The actual soil organic carbon observation value of each sample The loss function is used to measure the overall prediction bias of the model and serve as the optimization objective by calculating the square of the error between the predicted value and the true value and taking the average. The AdamW optimization algorithm is used to iteratively update the model parameters, and the model training is completed by combining the adaptive decay strategy of the learning rate and the early stopping mechanism.
[0025] like Figure 2The flowchart shown above illustrates the implementation process of this invention, which includes three stages: model initialization, training loop, and validation optimization. First, data acquisition and preprocessing are performed, and the data is divided into training set and validation set. All learnable parameters are randomly initialized, and a loss function, optimizer, and early stopping control mechanism are constructed. Then, an epoch-based iterative training phase begins. In each training epoch, the model is first set to training mode and the cumulative training error is initialized. The training set is then traversed in batches, and for each batch, a complete forward propagation, loss calculation, backpropagation, and parameter update process is performed. This allows the embedding parameters, attention weights, and regression layer parameters of the attention-enhanced TabM model to be continuously optimized under the influence of training set errors. After all batches in the current epoch are trained, the average training error is calculated, and the model is switched to evaluation mode. Forward inference is performed using the test set to verify the error, determining whether the current model performance is better than the historical best result. If it is, the optimal model parameter state is updated; otherwise, the original optimal state is maintained. This training and verification process is repeated within the set training epochs until the stopping condition is met, at which point the optimal model parameters are loaded and the final prediction result is output, thus completing the overall training and optimization process for soil organic carbon prediction based on the attention-enhanced TabM model.
[0026] like Figure 3 As shown, the block diagram of the proposed attention-enhanced TabM model illustrates the overall forward propagation structure of the attention-enhanced TabM model. Its computational flow can be represented as follows: First, input the feature matrix: in Indicates the number of samples. This represents the number of environmental factors; subsequently, in the piecewise linear embedding module, each feature is divided into quantile intervals and linearly interpolated, and an embedding transformation is achieved through learnable mapping parameters to obtain the feature embedding matrix: in For the embedding dimension; in the feature interaction module, a linear transformation is performed on the embedding matrix to generate the query matrix, key matrix, and value matrix, i.e. , , The weights between features are calculated through an attention mechanism. The interaction-enhanced feature representation is obtained as follows: Subsequently, in the attention-weighted aggregation module, importance weights are assigned to each feature, and then... Perform a weighted summation to form a sample-level representation vector: Finally, in the regression output module, the multilayer perceptron mapping function is used. Prediction results obtained: This completes the end-to-end nonlinear modeling process from the input of original environmental factors to the output of predicted soil organic carbon values.
[0027] The preferred embodiments of the present invention disclosed above are merely illustrative of the invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the invention to any specific implementation. Clearly, many modifications and variations can be made based on the content of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the invention, thereby enabling those skilled in the art to better understand and utilize the invention.
Claims
1. A method for predicting soil organic carbon based on the attention-enhanced TabM model, characterized in that, The steps include: Step S1. Data acquisition and preprocessing: After preprocessing the environmental factor data corresponding to the soil sampling points in the study area, as well as various remote sensing feature data that match their spatial coordinates and sampling time, the features are normalized by quantiles and divided into training set and test set. Step S2. Construct piecewise linear embedding based on the TabM framework: Calculate the boundary parameters of the quantile intervals of each feature on the training set, divide each feature into K intervals, and generate the feature embedding matrix through piecewise linear interpolation encoding; Step S3. Feature Interaction Enhancement: Based on the feature embedding matrix, a multi-head self-attention feature interaction module is constructed to model the interaction between different environmental factors and obtain the interaction-enhanced feature representation; Step S4. Attention-weighted aggregation: Attention-weighted aggregation is performed on the interaction enhancement feature representation. By calculating the importance weight of each feature and performing a weighted sum, a sample-level aggregated representation vector is formed. Step S5. Regression Prediction: Input the sample-level aggregated representation vector into the multilayer perceptron regression network for prediction calculation, and output the predicted value of soil organic carbon; Step S6. Model Training and Parameter Optimization: Construct the mean squared error loss function, use the AdamW optimization algorithm to iteratively update the model parameters, and combine the adaptive learning rate decay strategy and early stopping mechanism to complete model training.
2. The soil organic carbon prediction method based on the attention-enhanced TabM model according to claim 1 is characterized in that, The data acquisition and preprocessing method is used to acquire multi-source remote sensing data corresponding to the spatial location and sampling time of soil sampling points on a cloud computing platform. This multi-source remote sensing data includes optical remote sensing data and radar remote sensing data. The optical remote sensing data undergoes cloud and shadow removal, atmospheric correction, and time-series synthesis processing. The radar remote sensing data undergoes radiometric correction, geometric correction, and topographic correction processing to eliminate observation errors and topographic influences. Based on the spatial coordinates of the soil sampling points, spatial matching and feature extraction are performed on the preprocessed multi-source remote sensing data to obtain remote sensing feature data corresponding to each sampling point. This data, along with environmental factor data, constitutes the input dataset for soil organic carbon inversion. Subsequently, a quantile mapping function is calculated based on the feature distribution of the training set. Quantile normalization is applied to all input features, mapping them to a normal distribution space. The same mapping parameters are applied to the test set features to ensure data distribution consistency and model training stability.
3. The soil organic carbon prediction method based on the attention-enhanced TabM model according to claim 1 is characterized in that, The step of constructing piecewise linear embeddings based on the TabM framework is used to construct a piecewise linear embedding module based on the TabM model after data preprocessing; firstly, for each feature on the training set... Calculate the boundary parameters of its quantile intervals, assuming each feature is divided into 1 / 2 quantile intervals. There are several intervals, among which This indicates the number of segments, used to control the precision and expressive power of feature discretization; for any feature value, linear interpolation is performed based on its interval to generate a segment of length [length missing]. interval weight vector Subsequently, a learnable weight matrix was used. With bias vector Perform a linear mapping to obtain the embedding vector: And construct the feature embedding matrix: in For the sample size, For the number of features, For the embedded dimension.
4. The soil organic carbon prediction method based on the attention-enhanced TabM model according to claim 1 is characterized in that, The feature interaction enhancement method is used to construct a multi-head self-attention mechanism based on the feature embedding matrix obtained by piecewise linear embedding, and to map the embedding matrix to the query matrix through linear transformation. Key matrix Sum matrix ,in , , , , , For learnable parameter matrix, For the embedding dimension; the interaction weights between features are calculated using the following formula: in This represents the correlation matrix between different feature embeddings. Scaling factor The function is used to normalize the relevance, obtain the attention weights of each feature on other features, and apply the weights to the value matrix. The weighted result is obtained; finally, the interaction-enhanced feature representation is obtained: in This represents an enhanced embedding representation that incorporates dynamic dependencies between features, thereby enabling adaptive correlation modeling between different environmental factors and improving the model's ability to express complex nonlinear coupling relationships.
5. The soil organic carbon prediction method based on the attention-enhanced TabM model according to claim 1 is characterized in that, The attention-weighted aggregation method is used to score the importance of interaction-enhanced feature representations and assign weight coefficients to each feature. ,in Indicates the first The contribution of each environmental factor to the prediction of the current sample is determined by the weights calculated from the scoring function and then normalized; subsequently, the formula is applied: The sample-level representation vector is obtained by weighted summation of the feature embedding vectors. ,in For the number of features, For the first The interactive enhanced embedding representation of each feature, the weighted summation process is essentially a selective aggregation of different feature information according to feature importance, thereby realizing feature information compression and key feature enhancement, improving the model's expressive power and interpretability for soil organic carbon prediction.
6. The soil organic carbon prediction method based on the attention-enhanced TabM model according to claim 1 is characterized in that, The regression prediction method is used to aggregate the sample-level representation vector obtained by attention weighting. The sample-level representation vector is fed into a multilayer perceptron regression network for mapping calculation. Specifically, the sample-level representation vector is first mapped to a high-dimensional feature space through a first linear transformation and then normalized and nonlinearly activated to extract high-order regression features. It is then further reconstructed and abstracted through a second linear transformation, normalization, and nonlinear activation. Finally, the predicted value of soil organic carbon is obtained as a single continuous output through the linear regression mapping of the output layer, thereby realizing end-to-end regression prediction from the sample-level representation vector to the soil organic carbon inversion result.
7. The soil organic carbon prediction method based on the attention-enhanced TabM model according to claim 1 is characterized in that, The model training and parameter optimization method described above is used to construct the mean squared error loss function. in This indicates the number of samples used in the training process. For the first The actual soil organic carbon observation value of each sample The loss function is used to measure the overall prediction bias of the model and serve as the optimization objective by calculating the square of the error between the predicted value and the true value and taking the average. The AdamW optimization algorithm is used to iteratively update the model parameters, and the model training is completed by combining the adaptive decay strategy of the learning rate and the early stopping mechanism.