Soil humidity prediction method based on external climate environment

Abstract

The invention discloses a soil humidity prediction method based on an external climatic environment. The method comprises the following steps: 1) collecting humidity data, evaporation data and rainfall data of soil at different depths and time; 2) data preprocessing and data standardization: 2-1) abnormal value elimination: identifying abnormal values of the data by adopting a box graph method, analyzing and processing the data by adopting an outlier algorithm based on neighbor judgment, identifying the abnormal values, and then eliminating commonly identified abnormal points; 2-2) carrying out spline interpolation on the data of which the abnormity is eliminated; 2-3) standardizing the data; and 3) predicting the soil humidity by using a gray-LSTM prediction model: specifically, processing historical data by using the gray model, generating predicted values of each time point in the future, then correcting the predicted values by using the LSTM model, and outputting a final predicted value. The soil humidity prediction method is high in accuracy.

Description

Technical Field

[0001] The invention relates to a soil moisture prediction method based on external climate environment applicable to grassland pastures. Background Art

[0002] Soil moisture is one of the key indicators for assessing the health of an ecosystem, especially in vast grassland ecosystems. It not only affects plant growth and ecological balance, but is also the basis for maintaining biodiversity and ecological service functions. Soil moisture is also affected by a variety of external climate and environmental factors, including but not limited to precipitation, evaporation, temperature, wind speed, etc. These factors interact with each other and jointly determine the dynamic changes in soil moisture. For example, higher precipitation can increase soil moisture content, while high temperatures and strong winds can accelerate water evaporation and reduce soil moisture. Therefore, understanding how these external climate and environmental factors affect soil moisture is crucial to the effective management and protection of ecosystems.

[0003] In the field of soil moisture monitoring and prediction, researchers rely on a variety of methods and technologies to obtain accurate data and make effective predictions. However, each method is accompanied by specific challenges and limitations. Although direct instrument measurement and remote sensing technology provide immediate or large-area information on soil moisture, data quality is often limited due to the dynamic complexity of soil moisture and the potential errors of monitoring equipment. Empirical models are widely used due to their simplicity, but the scope of application of model parameters is limited, which restricts their versatility. Time series models have strong predictive capabilities in theory, but in practice they require a large amount of historical data and consideration of multiple variables, which is often difficult to achieve in reality. Neural network models predict future soil moisture changes based on historical data, but their predictive capabilities are limited when data is insufficient. Grey models are simple to calculate, but have limited processing capabilities for nonlinear relationships, which reduces the flexibility and accuracy of predictions.

[0004] In view of the above background, the present invention aims to propose a new soil moisture prediction method suitable for grassland pastures and improve the accuracy of the prediction results. Summary of the invention

[0005] The purpose of the present invention is to provide a new soil moisture prediction method suitable for grassland pastures and to improve the accuracy of the prediction results.

[0006] The object of the present invention is achieved through the following technical solution: a soil moisture prediction method based on external climate environment, comprising the following steps:

[0007] 1) Collect soil moisture data, evaporation data and precipitation data at different depths and times (evaporation and precipitation data are uniformly measured based on surface conditions);

[0008] 2) Data preprocessing and data standardization

[0009] 2-1) Removal of outliers

[0010] The box plot method is used to identify outliers in the data, and then the local distance-based outlier factor (LDOF) algorithm based on nearest neighbor judgment is used to analyze and process the data, identify outliers, and then eliminate the commonly recognized outliers.

[0011] 2-2) Perform spline interpolation on the data after removing the abnormalities;

[0012] 2-3) Standardize the data;

[0013] 3) Predict soil moisture using the Grey-LSTM prediction model

[0014] Specifically, the grey model is used to process historical data and generate prediction values ​​for future time points, and then the LSTM model is used to correct the prediction values ​​and output the final prediction values.

[0015] The LSTM model training uses the predicted values ​​of the grey model as input features and the corresponding actual observed data as target output, so that the LSTM model learns how to map from the predicted values ​​of the grey model to the actual observed values.

[0016] It is recommended to use the minimum mean square error (MSE) as the loss function for LSTM model training.

[0017] The Adam optimizer is recommended for LSTM model training.

[0018] In the box plot method, there is an upper limit Q U +k(Q U -Q L ), lower limit Q L -k(Q U -Q L ), where Q U represents the upper quartile, Q L represents the lower quartile, k is the calibration coefficient, which is generally taken as 1.5. When the data exceeds the upper and lower limits, it can be determined that the data is an outlier.

[0019] In the face of the large climate fluctuations in grassland pastures, simple data processing methods are difficult to distinguish abnormal data from the results of complex environmental changes. The present invention first uses the box plot method to identify the outliers of the data, and then uses the outlier algorithm based on nearest neighbor judgment to verify the outliers, remove the commonly identified outliers, and strive to ensure the comprehensiveness and reliability of the data, and reduce the errors brought by the subsequent prediction.

[0020] Then, the present invention combines the respective advantages of the grey model and the LSTM model, obtains the predicted value through the grey model, and corrects the predicted value through the LSTM model, thereby further improving the accuracy of the prediction result.

[0021] The advantages of the present invention are:

[0022] 1) The method of the present invention utilizes a combination of the box plot method and the LDOF algorithm to identify and remove outliers in the data. In the process of removing abnormal data, it strives to ensure the comprehensiveness and reliability of the data, improve the credibility and authenticity of the data, and reduce the errors introduced in the subsequent prediction;

[0023] 2) The grey-LSTM model is a multivariate prediction model. The present invention combines the respective advantages of these two different models, obtains the predicted value through the grey model, and learns how to map the predicted value of the grey model to the actual observed value through the LSTM model, and finally outputs the corrected predicted value that is closer to the actual situation after considering the deviation, thereby further improving the accuracy of the prediction result. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 A flowchart of a preferred prediction method of the present invention;

[0025] Figure 2 It is a box plot under different data;

[0026] Figure 3 It is a diagram of the data processing process;

[0027] Figure 4 This is the flow chart of the grey GM(1,1) model.

[0028] Figure 5 This is the LSTM network unit structure diagram;

[0029] Figure 6 This is a comparison chart of the loss of the last training batch for different optimizers;

[0030] Figure 7 This is the loss map of the soil moisture prediction model at a depth of 10 cm;

[0031] Figure 8 This is the curve change diagram of the soil moisture prediction model at a depth of 10cm. DETAILED DESCRIPTION

[0032] The present invention is further described in detail below in conjunction with specific embodiments and drawings.

[0033] Figure 1 The flowchart of the prediction method of the preferred embodiment of the present invention is as follows: After collecting the moisture data, evaporation data and precipitation data of the grassland pasture soil at different depths and times, the data needs to be preprocessed.

[0034] The preprocessing step includes outlier removal, because unreasonable data that deviates from the overall range of the data will have a greater interference with the analysis and affect the robustness of the model, resulting in great irrationality and adverse effects on the final prediction results. The present invention first uses the box plot method to identify outliers in the data, which is somewhat different from the conventional method of identifying outliers. Figure 2 As shown in the figure, the box plot method can display the median, tail length, main distribution interval, and outliers of the data, which is better for data processing. In the box plot method, there is an upper limit Q U +k(Q U -Q L ), lower limit Q L -k(Q U -Q L ), where Q U represents the upper quartile, Q L represents the lower quartile, k is the calibration coefficient, which is generally 1.5. When the data exceeds the upper and lower limits, the data is judged to be an outlier.

[0035] The outliers identified by the above box plot method do not necessarily mean that they are abnormal. They may be the result of complex environmental changes. In order to prevent the serious deviation caused by the spline interpolation after the removal of continuous data, the present invention uses an outlier algorithm based on nearest neighbor judgment to analyze and process the data, identify outliers again, and remove commonly recognized outliers. In the case of removing abnormal data, it strives to ensure the comprehensiveness and reliability of the data and reduce the errors brought by the subsequent prediction.

[0036] The outlier mining algorithm is a method of mining smaller pattern data, and its purpose is to mine abnormal data that deviates from the majority of data. The LDOF algorithm stands out from many outlier algorithms due to its superiority in mining data. Its expression is shown in the following formula:

[0037]

[0038] In the formula, Q p is the set of K nearest neighboring points of data P, a is Q p For any data object in Q p Any two different data objects.

[0039] In the LDOF algorithm, for each object p in the data set, k (k = 0, 1, 2...) nearest neighbors are retrieved and the outlier factor of each object p is calculated. The objects are sorted according to their LDOF scores, and the top 10 objects with the highest LDOF scores are output. The outliers identified by the LDOF algorithm are compared multiple times and combined with the box plot method, and the same outliers are removed. Figure 3 The processing of some data is shown, and NaN represents the discarded data.

[0040] After removing abnormal data, the data are interpolated using spline and then standardized.

[0041] The normalization formula is as follows:

[0042]

[0043] Where x, y, μ, and σ represent data input, output, mean, and standard deviation, respectively.

[0044] Because different data have their own range of values, in order to ensure the reliability of subsequent results and use the same comparison standard for algorithm operations, the data needs to be standardized.

[0045] Then, the grey-LSTM prediction model is used to predict soil moisture.

[0046] Input the standardized data sequence, use the grey system theory, and use the GM (1,1) model to process the historical data and generate the forecast values ​​for each future time point. The grey GM (1,1) model is a commonly used model. Its complete process is as follows: Figure 4 As shown, these forecasts are often smoother than the actual observed values.

[0047] In the GM(1,1) grey model, let is the original grassland soil moisture series, To generate a sequence for accumulation, we have:

[0048]

[0049] Accumulate the series to fit and obtain the prediction equation:

[0050]

[0051] In the formula: a is the development gray number; u is the endogenous control gray number.

[0052] Next, we construct an LSTM model, using the predicted values ​​of the grey model as input features (X) and the corresponding actual observed data as target outputs (Y). In this way, the LSTM will learn how to map from the predicted values ​​of the grey model to the actual observed values, thereby capturing the difference or deviation between the two.

[0053] LSTM training adjusts the weights and biases of LSTM through the back-propagation algorithm and the appropriate loss function, so that it can minimize the error between the predicted value and the actual observed value. This step ensures model convergence through a large amount of training data and iterations. When the model training is completed, the new gray model prediction value can be input into LSTM. LSTM will output a revised prediction value based on the pattern it has learned, which is closer to the actual observed value because it takes into account the potential deviation in the gray model prediction. The final output prediction value is the revised prediction value after the LSTM network takes into account the deviation, which is also closer to the actual situation than the direct gray prediction value.

[0054] The LSTM (Long Short Term Memory) model is a neural network that has the ability to memorize long-term and short-term information. It is a type of recurrent neural network (RNN). It is good at processing time-series and nonlinear data, has strong applicability, and can effectively prevent gradient disappearance. Its network is mainly composed of three parts: forget gate, update gate, and output gate. It realizes the judgment of effective information, retains effective information, and forgets useless information, solving the problem that traditional methods cannot effectively obtain long-term information and the gradient disappears when the model is too deep.

[0055] In LSTM cells, such as Figure 5 As shown, C t-1 and C t They are the cell states of the previous LSTM cell and the current LSTM cell, respectively, and are used to carry memory information. t is the input of the current cell, h t-1 and h t are the hidden states of the previous cell and the current cell respectively, and sigmoid and tanh are activation functions. It should be noted that h t-1 and x t The connection position is a matrix concatenation operation, that is, [h t-1 ,x t The circular nodes in the cells represent the operations on the elements in the same position of two matrices of the same dimension. t is the forget gate, which determines the cell state C of the previous layer t-1 The degree of information retention of the corresponding position element, 0 means completely forgotten, 1 means completely retained. tIt is the update gate, which determines that the input needs to retain the updated information after passing through the tanh activation function. After the memory information and the update information are fused, a new memory C is generated through the tanh activation function. t .O t is the output gate, which determines which parts of the neuron state are output, and uses the tanh activation function to process the neuron state to get the final result. The calculation formula involved in LSTM is as follows:

[0056]

[0057] Among them, W and b are the corresponding weight parameter matrix and offset, which are iteratively updated by gradient descent. LSTM realizes the mining of implicit information in data and the memory and update of historical information through memory gate, update gate and output gate. According to the above formula, we can get:

[0058]

[0059] Among them, C t is the state of the neuron, h t is the output.

[0060] The loss function is used to measure the difference between the predicted value and the actual monitored value of soil moisture during the model training process. The present invention uses the mean square error (MSE), which is defined as:

[0061]

[0062] in, is the model prediction value, y i is the actual monitoring value, and N is the total sample size.

[0063] The closer the MSE value is to 0, the smaller the gap between the LSTM model prediction value and the true value, and the better the model fitting effect. Therefore, how to minimize MSE becomes the main goal of LSTM model training.

[0064] The present invention adopts SGD and Adam optimizers to perform model training at the same time, and selects the Adam optimizer with the best fitting model according to the training results.

[0065] Verification Example

[0066] As a representative area of ​​temperate grassland, Xilin Gol Grassland in Inner Mongolia is one of the four major grasslands in China. The grassland is located in the Xilin River Basin of the Inner Mongolia Plateau, with an average annual precipitation of 340 mm. Xilin Gol Grassland in Inner Mongolia is not only one of the typical external climate environments, but also a key area for maintaining ecological balance, effectively reducing the occurrence of sandstorms and bad weather, and becoming an important place for studying the response mechanism of ecosystems to human interference and global climate change. Therefore, the present invention selects Xilin Gol Grassland as a specific test object, and collects its climate data and soil moisture data from 2012 to 2022 as original data by reading literature and consulting statistical yearbooks.

[0067] First, the soil moisture data, soil evaporation data, and precipitation data collected at different times and depths are preprocessed. Specifically, the box plot method and LDOF algorithm are combined to eliminate abnormal data, and then spline interpolation is performed.

[0068] After the data is standardized, it is input into the processed data sequence, and the GM (1,1) grey model is used to process the historical data and generate the predicted values ​​for each future time point. Then, the LSTM model is used to correct the predicted values ​​and output the final predicted values.

[0069] The LSTM model training uses the predicted value of the grey model as the input feature X, and the corresponding actual observation data as the target output Y, so that the LSTM model learns how to map from the predicted value of the grey model to the actual observation value.

[0070] The loss function selected is the minimum mean square error (MSE).

[0071] The learning rate used in the LSTM training process is 0.01.

[0072] Figure 6 The figure shows the loss value of the last training batch when using SGD and Adam optimizers for model training. It can be seen from the figure that the Adam optimizer is better than the SGD optimizer overall, and the subsequent models use the Adam optimizer.

[0073] Figure 7 The loss function value of the soil moisture prediction model at a depth of 10 cm using the Adam optimizer changes with the training batch. It can be seen that the loss function finally tends to stabilize and converge. Therefore, the LSTM model can be used to predict future data, which verifies the rationality of this model.

[0074] Figure 8The figure shows the change of the prediction curve of the prediction model under 10cm soil moisture. It can be seen that the gray-LSTM prediction model has superior performance and is suitable for processing time series data and predicting future data. The model has a good effect and a high degree of fit with the soil moisture data of Xilin Gol grassland. It has a certain reference value for predicting the soil moisture data of Xilin Gol grassland.

Claims

1. A soil moisture prediction method based on external climate environment, characterized in that: The steps include: 1) Collect soil moisture data, evaporation data and precipitation data at different depths and times; 2) Data preprocessing and data standardization 2-1) Removal of outliers The box plot method is used to identify outliers in the data, and then the outlier algorithm based on nearest neighbor judgment is used to analyze and process the data, identify outliers, and then eliminate the commonly recognized outliers; 2-2) Perform spline interpolation on the data after removing the abnormalities; 2-3) Standardize the data; 3) Predict soil moisture using the Grey-LSTM prediction model Specifically, the grey model is used to process historical data and generate prediction values ​​for future time points, and then the LSTM model is used to correct the prediction values ​​and output the final prediction values.

2. The prediction method according to claim 1, characterized in that: The LSTM model training uses the predicted values ​​of the grey model as input features and the corresponding actual observed data as target output, so that the LSTM model learns how to map from the predicted values ​​of the grey model to the actual observed values.

3. The prediction method according to claim 2, characterized in that: The LSTM model training adopts the minimum mean square error as the loss function.

4. The prediction method according to claim 3, characterized in that: The LSTM model training adopts Adam optimizer.

5. The prediction method according to claim 1, characterized in that: In the box plot method, there is an upper limit Q U +k(Q U -Q L ), lower limit Q L -k(Q U -Q L ), where Q U represents the upper quartile, Q L represents the lower quartile, k is the calibration coefficient, k=1.5, when the data exceeds the upper and lower limits, the data are judged to be outliers.

Images