Soil temperature inversion method and system considering hysteresis effect and feature importance

Abstract

The invention belongs to the technical field of soil temperature inversion, and relates to a soil temperature inversion method and system considering a hysteresis effect and feature importance. Constructing a basic input data set of the to-be-inverted region; constructing a lag structure input set of the multi-source driving variables, and generating a plurality of input variables of time-lag versions; evaluating the feature importance degree of each input variable; screening a dominant variable set; reconstructing a final input set, and training to obtain an optimized soil temperature inversion model; obtaining a soil temperature estimation result of the target area at the target moment; and performing precision evaluation on the optimized soil temperature inversion model by using verification data, and adjusting a lag structure of an input variable. According to the invention, by introducing an input mechanism of a lagging structure, the inversion precision is significantly improved; a feature importance evaluation mechanism is introduced, key variables are dynamically selected as input features, efficient modeling is realized in a variable limited scene, and the flexibility and generalization ability of the model under different earth surface types and meteorological conditions are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of soil temperature inversion, and specifically relates to a soil temperature inversion method and system considering hysteresis effect and feature importance. Background Art

[0002] Soil temperature is extremely crucial for crop growth, ecosystem stability, and climate research. Currently, there are mainly two ways to obtain soil temperature: ground station observation and remote sensing inversion. Ground station observation can provide relatively accurate data, but the spatial coverage is limited. In large-area monitoring, it is difficult to comprehensively reflect the distribution of soil temperature. For example, in vast farmlands or complex terrain areas, sparse stations lead to the lack of soil temperature data in many places. Moreover, the construction and maintenance costs of stations are relatively high, which limits the increase in the number of stations and cannot meet the needs of large-scale and refined research.

[0003] Although remote sensing inversion can achieve large-area soil temperature measurement, due to the influence of factors such as vegetation cover, cloud occlusion, and observation angle, the obtained data is often the mixed temperature of multiple underlying surface components, with poor accuracy. At the same time, cloud occlusion will cause data loss, reducing the integrity and continuity of the data.

[0004] To solve these problems, using machine learning methods to combine multi-source data such as meteorology and remote sensing to estimate soil temperature has become a new direction. There are two major problems in existing machine learning methods when modeling soil temperature: one is ignoring the variable hysteresis effect. The change of soil temperature lags behind driving factors such as air temperature and radiation. Modeling only with current moment data will make the prediction results inaccurate; the other is not distinguishing the variable importance. Directly inputting all variables into the model not only increases the computational burden but also reduces the model stability due to redundant information.

[0005] Therefore, the present invention proposes a new soil temperature inversion method, fully considering the variable hysteresis effect and feature importance to improve the inversion accuracy and enhance the model performance. Summary of the Invention

[0006] To solve the above technical problems, the present invention provides a soil temperature inversion method and system considering hysteresis effect and feature importance.

[0007] In the first aspect, the present invention provides a soil temperature inversion method considering hysteresis effect and feature importance, including: Constructing a basic input data set for the area to be inverted; the basic input data set includes historical time series of soil temperature labels and multi-source driving variables; Extracting time series data of the current period and the lag period of multi-source driving variables, constructing a lag structure input set of multi-source driving variables, and generating several time-delay versions of input variables; Construct a supervised learning model, and use the time-lagged version of the input variables and the soil temperature labels to train the supervised learning model to obtain an initial inversion model, and evaluate the feature importance of each input variable; Screen the input variables according to the feature importance to obtain a set of dominant variables; Obtain the time-lagged version of the input variables in the set of dominant variables, reconstruct the final input set, and train the initial inversion model to obtain an optimized soil temperature inversion model; Use the optimized soil temperature inversion model to perform soil temperature inversion to obtain the soil temperature estimation result at the target time in the target area; Use the original measured soil temperature data as verification data, evaluate the accuracy of the optimized soil temperature inversion model using the verification data, and adjust the selection strategy of the input variables according to the accuracy evaluation result.

[0008] In a second aspect, the present invention provides a soil temperature inversion system considering hysteresis effect and feature importance, including a first data set construction unit, an input variable generation unit, a feature importance evaluation unit, a screening unit, an input reconstruction and model training unit, an inversion unit, and an accuracy evaluation and adjustment unit; The first data set construction unit is used to construct a basic input data set for the area to be inverted; the basic input data set includes historical time series of soil temperature labels and multi-source driving variables; The input variable generation unit is used to extract the time series data of the current period and the lag period of the multi-source driving variables, construct a lag structure input set of the multi-source driving variables, and generate several time-lagged versions of the input variables; The feature importance evaluation unit is used to construct a supervised learning model, and use the time-lagged version of the input variables and the soil temperature labels to train the supervised learning model to obtain an initial inversion model, and evaluate the feature importance of each input variable; The screening unit is used to screen the input variables according to the feature importance to obtain a set of dominant variables; The input reconstruction and model training unit is used to obtain the time-lagged version of the input variables in the set of dominant variables, reconstruct the final input set, and train the initial inversion model to obtain an optimized soil temperature inversion model; The inversion unit is used to use the optimized soil temperature inversion model to perform soil temperature inversion to obtain the soil temperature estimation result at the target time in the target area; The accuracy evaluation and adjustment unit is used to use the original measured soil temperature data as verification data, evaluate the accuracy of the optimized soil temperature inversion model using the verification data, and adjust the selection strategy of the input variables according to the accuracy evaluation result.

[0009] Based on the above technical solutions, the present invention can also be improved as follows.

[0010] Furthermore, a basic input data set for the area to be inverted is constructed, including: collecting soil temperature observation data in the target period and for a period of time before the target period in the area to be inverted, and time series data of multi-source driving variables corresponding to the soil temperature observation data.

[0011] Furthermore, the multi-source driving variables include air temperature, precipitation, solar radiation, wind speed, soil moisture and vegetation index; the data sources include ground meteorological stations, remote sensing products and reanalysis data.

[0012] Furthermore, when generating several time-lagged versions of the input variables, based on statistical data, calculate the mean value of the lag duration of each multi-source driving variable in different lag periods, and select the lag duration corresponding to the smallest difference between the soil temperature observation data and the predicted soil temperature value as the lag duration of the input variable.

[0013] Furthermore, the supervised learning model is one of a random forest model, a gradient boosting tree model and a neural network model.

[0014] Furthermore, evaluating the feature importance of each input variable includes: evaluating the importance of each input variable to the soil temperature prediction result through the variable contribution index generated during the training process of the initial inversion model.

[0015] Furthermore, the variable contribution index is the split gain or attention weight or SHAP value or Pearson correlation coefficient.

[0016] Furthermore, calculate the error index based on the validation data and the soil temperature inversion result, and evaluate the accuracy of the optimized soil temperature inversion model.

[0017] Furthermore, adjusting the selection strategy of input variables includes: adjusting the number of input variables, the lag time length of input variables and the screening threshold of feature importance; when screening input variables according to feature importance, set the screening threshold of feature importance, and retain the input variables with feature importance greater than the screening threshold; the feature importance is used to characterize the contribution degree of input variables to soil temperature prediction.

[0018] The beneficial effects of the present invention are as follows: (1) By introducing an input mechanism with a lag structure, the present invention makes up for the information loss problem that current input variables are difficult to reflect the current soil thermal state, effectively enhances the model's ability to depict the soil thermal evolution process, and thus significantly improves the inversion accuracy; (2) The present invention introduces a feature importance evaluation mechanism to enhance the pertinence of input selection and the interpretability of the model, quantify the contribution degrees of different variables, and dynamically select key variables as input features during the modeling process. On the one hand, it can achieve efficient modeling in scenarios with limited variables, and on the other hand, it improves the flexibility and generalization ability of the method under different surface types and meteorological conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is the schematic diagram of the soil temperature inversion method considering hysteresis effect and feature importance provided by Embodiment 1 of the present invention; Figure 2 It is the system block diagram of the soil temperature inversion system considering hysteresis effect and feature importance provided by Embodiment 2 of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0020] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. Generally, the components of the embodiments of the present invention described and illustrated herein can be arranged and designed in various different configurations.

[0021] Embodiment 1 As an embodiment, as shown in the attached Figure 1 figure, to solve the above technical problems, the present embodiment provides a soil temperature inversion method considering hysteresis effect and feature importance, including: Construct a basic input data set for the area to be inverted; the basic input data set includes historical time series of soil temperature labels and multi-source driving variables; Extract the time series data of the current period and the lag period of the multi-source driving variables, construct a lag structure input set of the multi-source driving variables, and generate several time-lagged versions of the input variables; Construct a supervised learning model, use the time-lagged versions of the input variables and the soil temperature labels to train the supervised learning model to obtain an initial inversion model, and evaluate the feature importance of each input variable; Screen the input variables according to the feature importance to obtain a set of dominant variables; Obtain the time-lagged versions of the input variables in the set of dominant variables, reconstruct the final input set, and train the initial inversion model to obtain an optimized soil temperature inversion model; Use the optimized soil temperature inversion model to perform soil temperature inversion to obtain the soil temperature estimation result at the target time in the target area; Take the measured data of the original soil temperature as verification data, use the verification data to evaluate the accuracy of the optimized soil temperature inversion model, and adjust the selection strategy of input variables according to the accuracy evaluation results.

[0022] The present invention proposes a soil temperature inversion method that simultaneously considers the lag effect and feature importance of input variables, which can improve the physical consistency of the supervised learning model for the soil heat response process, the effectiveness of the input structure, and the prediction accuracy under diverse environmental conditions.

[0023] The present invention significantly improves the soil temperature inversion accuracy by introducing an input mechanism with a lag structure. Most traditional soil temperature modeling methods only use input variables at the current moment, ignoring the lag response characteristics of soil temperature to external environmental drivers. Especially in the context of rapid changes in air temperature and radiation, the model prediction error increases significantly. The present invention constructs a variable lag structure and introduces input variables at several past moments (such as air temperature, solar radiation, and soil moisture, etc.) into the modeling process, making up for the information loss problem that current input variables are difficult to reflect the current soil heat state, effectively enhancing the model's ability to depict the soil heat evolution process, and thus significantly improving the inversion accuracy.

[0024] The present invention introduces a variable importance evaluation mechanism to improve the pertinence of input variable selection and the interpretability of the model. In traditional methods, all variables are input into the model without effective screening, which not only introduces redundant information but also increases the computational cost and training difficulty. The present invention embeds a feature importance evaluation, quantifies the contribution degrees of different variables using built-in or external interpretation methods of the model, and dynamically selects key variables as input features during the modeling process. On the one hand, it can achieve efficient modeling in scenarios with limited variables, and on the other hand, it improves the flexibility and generalization ability of the method under different land surface types and meteorological conditions.

[0025] The present invention also proposes a selection strategy for input variables, that is, screening the input variables, and according to the importance evaluation of the input variables, selecting the variable combination with the best prediction effect and removing the input variables that are ineffective or have a negative impact on the prediction results.

[0026] Optionally, construct a basic input data set for the area to be inverted, including: collecting soil temperature observation data in the target period and for a period of time before the target period in the area to be inverted, and time series data of multi-source driving variables corresponding to the soil temperature observation data.

[0027] Optionally, the multi-source driving variables include air temperature, precipitation, solar radiation, wind speed, soil moisture, and vegetation index; the data sources include ground meteorological stations, remote sensing products, and reanalysis data.

[0028] Optionally, when generating multiple time-lagged versions of the input variables, calculate the mean lag duration of each multi-source driving variable in different lag periods based on statistical data, and select the lag duration corresponding to the smallest difference between the soil temperature observation data and the predicted soil temperature value as the lag duration of the input variable.

[0029] Optionally, the supervised learning model is one of a random forest model, a gradient boosting tree model, and a neural network model.

[0030] Fit the non-linear mapping relationship between the input variables and the soil temperature label through the supervised learning model.

[0031] Optionally, evaluating the feature importance of each input variable includes: evaluating the importance of each input variable to the soil temperature prediction result through the variable contribution index generated during the training process of the initial inversion model.

[0032] The feature importance refers to the variable contribution index generated during the training process of the initial inversion model, and evaluates the importance of each input variable to the soil temperature inversion result.

[0033] Optionally, the variable contribution index is the split gain or the attention weight or the SHAP value (SHapley Additive exPlanations, Shapley additive explanation value or Pearson correlation coefficient).

[0034] Specifically, the feature importance is determined based on the supervised learning model, such as the split gain of the tree model, the attention weight of the neural network, and the SHAP value of the posterior analysis method, etc.

[0035] During the construction of the tree model, the data is continuously split to reduce the impurity of the nodes and improve the classification or prediction ability of the model for the data. The split gain indicators are such as the Gini index and the information gain. When training the tree model, a feature is selected for splitting each time, and the gain value after splitting different features is calculated. For example, for a classification problem, the Gini index is used to calculate the change in node impurity after splitting each feature. The greater the decrease in impurity, the greater the split gain, indicating that the feature is more effective in distinguishing different categories and contributes more to the model. After training is completed, count the number of times each feature is used for splitting in all trees, and the total gain brought by these splits. The more times a feature is used for splitting and the greater the total split gain, the higher the feature importance.

[0036] In a neural network model with an attention mechanism, attention weights can be used to measure the importance of features. Taking the Transformer model in natural language processing as an example, the self-attention mechanism can dynamically assign different weights to each position in the input sequence. During the training process, the model automatically learns the weights so that important features receive greater weights. For the input text data, each word vector interacts and calculates with other word vectors to generate attention scores, and then the attention scores are transformed into attention weights through the softmax function. These weights reflect the importance of each word for the current position representation. Summing up the attention weights of this feature at all positions can obtain the importance measure of this feature. The higher the weight, the more attention the feature receives when the model processes information, and the greater the impact on the model output.

[0037] SHAP values are based on the concept of Shapley values in cooperative game theory, considering all possible feature combinations and assigning an importance value to each feature. When calculating SHAP values, the model needs to be evaluated multiple times first. For a dataset containing n features, all possible feature subsets need to be considered. For each feature subset, calculate the difference between the prediction result of the model on this subset and the prediction result on the empty set. This difference is the contribution of this feature subset to the model prediction, and then the contribution is allocated to each feature in the subset through methods such as sampling approximation. Repeat this process until the contributions of all features in all possible subsets are calculated, and finally obtain the SHAP value of each feature. The larger the SHAP value, the greater the impact of this feature on the model prediction result, and the higher the feature importance.

[0038] Optionally, calculate the error index based on the validation data and the soil temperature inversion result to evaluate the accuracy of the optimized soil temperature inversion model.

[0039] Error indices such as root mean square error, mean absolute error, and coefficient of determination, etc. When calculating the root mean square error, first find the square of the difference between the predicted value and the measured value of each sample point, accumulate these squared values and divide by the total number of samples to obtain the mean square error, and then take the square root to get the root mean square error value. The smaller the root mean square error value, the closer the model predicted value is to the measured value, and the higher the prediction accuracy of the model. When calculating the mean absolute error, first calculate the absolute value of the difference between the predicted value and the measured value of each sample point, then accumulate these absolute values and divide by the total number of samples. The smaller the mean absolute error value, the smaller the average error between the model predicted value and the measured value, and the better the model prediction effect. The closer the coefficient of determination is to 1, the better the fitting effect of the model to the data, the higher the proportion of the dependent variable change that the model can explain, that is, the stronger the prediction ability of the model; the closer the coefficient of determination is to 0, the worse the fitting effect of the model to the data, and the weaker the model prediction ability.

[0040] Optionally, the selection strategy for adjusting the input variables includes: adjusting the number of input variables, the lag time length of the input variables, and the screening threshold of the feature importance; when screening the input variables according to the feature importance, setting the screening threshold of the feature importance, and retaining the input variables with feature importance greater than the screening threshold; the feature importance is used to characterize the contribution degree of the input variables to the soil temperature prediction.

[0041] Adjusting the number of input variables, the lag time length of the input variables, and the screening threshold of the feature importance based on the verification results can further improve the adaptability and robustness of the soil temperature inversion model under different climate zones, surface types, or observation densities.

[0042] Embodiment 2 Based on the same principle as the method shown in Embodiment 1 of the present invention, as shown in the appendix Figure 2 In the embodiments of the present invention, a soil temperature inversion system considering the lag effect and feature importance is also provided, including a first data set construction unit, an input variable generation unit, a feature importance evaluation unit, a screening unit, an input reconstruction and model training unit, an inversion unit, and an accuracy evaluation and adjustment unit; The first data set construction unit is used to construct a basic input data set for the area to be inverted; the basic input data set includes historical time series of soil temperature labels and multi-source driving variables; The input variable generation unit is used to extract the time series data of the current period and the lag period of the multi-source driving variables, construct a lag structure input set of the multi-source driving variables, and generate several time-delay versions of the input variables; The feature importance evaluation unit is used to construct a supervised learning model, train the supervised learning model with the time-delay versions of the input variables and the soil temperature labels to obtain an initial inversion model, and evaluate the feature importance of each input variable; The screening unit is used to screen the input variables according to the feature importance to obtain a set of dominant variables; The input reconstruction and model training unit is used to obtain the time-delay versions of the input variables in the set of dominant variables, reconstruct the final input set, and train the initial inversion model to obtain an optimized soil temperature inversion model; The inversion unit is used to perform soil temperature inversion using the optimized soil temperature inversion model to obtain the soil temperature estimation result at the target time in the target area; The accuracy evaluation and adjustment unit is used to use the original measured soil temperature data as verification data, evaluate the accuracy of the optimized soil temperature inversion model using the verification data, and adjust the selection strategy of the input variables according to the accuracy evaluation result.

[0043] Optionally, a basic input data set for the area to be inverted is constructed, including: collecting soil temperature observation data in the target period and a period of time before the target period for the area to be inverted, and time series data of multi-source driving variables corresponding to the soil temperature observation data.

[0044] Optionally, the multi-source driving variables include air temperature, precipitation, solar radiation, wind speed, soil humidity and vegetation index; the data sources include ground meteorological stations, remote sensing products and reanalysis data.

[0045] Optionally, when generating several time-lagged versions of the input variables, based on statistical data, calculate the mean value of the lag duration of each multi-source driving variable in different lag periods, and select the lag duration corresponding to the smallest difference between the soil temperature observation data and the predicted soil temperature value as the lag duration of the input variables.

[0046] Optionally, the supervised learning model is one of a random forest model, a gradient boosting tree model and a neural network model.

[0047] Optionally, evaluating the feature importance of each input variable includes: evaluating the importance of each input variable to the soil temperature prediction result through the variable contribution index generated during the training process of the initial inversion model.

[0048] Optionally, the variable contribution index is the split gain or attention weight or SHAP value or Pearson correlation coefficient.

[0049] Optionally, calculate the error index based on the validation data and the soil temperature inversion result, and evaluate the accuracy of the optimized soil temperature inversion model.

[0050] Optionally, adjusting the selection strategy of the input variables includes: adjusting the number of input variables, the lag time length of the input variables, and the screening threshold of the feature importance; when screening the input variables according to the feature importance, set the screening threshold of the feature importance, and retain the input variables with feature importance greater than the screening threshold; the feature importance is used to characterize the contribution of the input variables to the soil temperature prediction.

[0051] The above are only the preferred embodiments of the present invention and are not used to limit the present invention. For those skilled in the art, the present invention can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A soil temperature inversion method considering hysteresis effect and feature importance, characterized in that, Including: Constructing a basic input data set for the area to be inverted; The basic input data set includes historical time series of soil temperature labels and multi-source driving variables; Extracting time series data of the current period and lag period of multi-source driving variables, constructing a lag structure input set of multi-source driving variables, and generating several time-lagged versions of input variables; Constructing a supervised learning model, training the supervised learning model using the time-lagged versions of input variables and soil temperature labels to obtain an initial inversion model, and evaluating the feature importance of each input variable; Screening the input variables according to the feature importance to obtain a set of dominant variables; Obtaining the time-lagged versions of input variables in the set of dominant variables, reconstructing the final input set, and training the initial inversion model to obtain an optimized soil temperature inversion model; Using the optimized soil temperature inversion model to perform soil temperature inversion to obtain the estimated result of soil temperature at the target time in the target area; Taking the original measured soil temperature data as verification data, evaluating the accuracy of the optimized soil temperature inversion model using the verification data, and adjusting the selection strategy of input variables according to the accuracy evaluation result.

2. The soil temperature inversion method considering hysteresis effect and feature importance according to claim 1, characterized in that Constructing a basic input data set for the area to be inverted, including: collecting soil temperature observation data in the target period and a period of time before the target period in the area to be inverted and time series data of multi-source driving variables corresponding to the soil temperature observation data.

3. The soil temperature inversion method considering hysteresis effect and feature importance according to claim 1, characterized in that The multi-source driving variables include air temperature, precipitation, solar radiation, wind speed, soil humidity and vegetation index; the data sources include ground meteorological stations, remote sensing products and reanalysis data.

4. The soil temperature inversion method considering hysteresis effect and feature importance according to claim 1, characterized in that, When generating several time-lagged versions of input variables, calculate the mean value of the lag duration of each multi-source driving variable in different lag periods based on statistical data, and select the lag duration with the smallest difference between the soil temperature prediction value corresponding to the soil temperature observation data and the mean value as the lag duration of the input variable.

5. The soil temperature inversion method considering hysteresis effect and feature importance according to claim 1, characterized in that The supervised learning model is one of a random forest model, a gradient boosting tree model and a neural network model.

6. The soil temperature inversion method considering hysteresis effect and feature importance according to claim 1, characterized in that Evaluating the feature importance of each input variable includes: evaluating the importance of each input variable to the soil temperature prediction result through the variable contribution index generated during the training process of the initial inversion model.

7. The soil temperature inversion method considering hysteresis effect and feature importance according to claim 6, characterized in that, The variable contribution index is the split gain or attention weight or SHAP value or Pearson correlation coefficient.

8. The soil temperature inversion method considering hysteresis effect and feature importance according to claim 1, characterized in that Calculating an error index based on the verification data and the soil temperature inversion result, and evaluating the accuracy of the optimized soil temperature inversion model.

9. The soil temperature inversion method considering hysteresis effect and feature importance according to claim 1, characterized in that Adjusting the selection strategy of input variables includes: adjusting the number of input variables, the lag time length of input variables and the screening threshold of feature importance; when screening input variables according to feature importance, setting the screening threshold of feature importance and retaining input variables with feature importance greater than the screening threshold; the feature importance is used to characterize the contribution degree of input variables to soil temperature prediction.

10. A soil temperature inversion system considering hysteresis effect and feature importance, characterized in that, Including a first data set construction unit, an input variable generation unit, a feature importance evaluation unit, a screening unit, an input reconstruction and model training unit, an inversion unit, and an accuracy evaluation and adjustment unit; The first data set construction unit is used to construct a basic input data set for the area to be inverted; the basic input data set includes historical time series of soil temperature labels and multi-source driving variables; An input variable generation unit, configured to extract time series data of the current period and the lag period of multi-source driving variables, construct a lag structure input set of the multi-source driving variables, and generate several time-delay versions of input variables; A feature importance evaluation unit, configured to construct a supervised learning model, train the supervised learning model with the time-delay versions of input variables and soil temperature labels to obtain an initial inversion model, and evaluate the feature importance of each input variable; A screening unit, configured to screen the input variables according to the feature importance to obtain a set of dominant variables; An input reconstruction and model training unit, configured to obtain the time-delay versions of input variables in the set of dominant variables, reconstruct a final input set, and train the initial inversion model to obtain an optimized soil temperature inversion model; An inversion unit, configured to perform soil temperature inversion by using the optimized soil temperature inversion model to obtain an estimation result of the soil temperature at the target time in the target area; An accuracy evaluation and adjustment unit, configured to use the original measured soil temperature data as verification data, evaluate the accuracy of the optimized soil temperature inversion model by using the verification data, and adjust the selection strategy of input variables according to the accuracy evaluation result.

Images