A method, device and medium for predicting nitrogen leaching loss in farmland

Abstract

The present application discloses a method, device and medium for predicting nitrogen leaching loss in farmland, and relates to the field of nitrogen leaching loss prediction. The method comprises: preprocessing farmland nitrogen leaching loss test data; constructing a training set and a test set based on the preprocessed farmland nitrogen leaching loss test data; in the process of training the initial XGBoost regression model with the training set, optimizing the hyperparameters with the Bayesian optimization method and the five-fold cross validation method, and determining the SHAP value of each sample in the training set with the Shapley additive interpretation method; testing the XGBoost regression model with the test set until the set conditions are met to obtain a nitrogen leaching loss prediction model; inputting the preprocessed nitrogen leaching loss data of the farmland to be predicted into the nitrogen leaching loss prediction model to obtain a nitrogen leaching loss prediction result. The present application can solve the problem that the results are difficult to explain due to the "black box effect" of machine learning, so as to achieve accurate and comprehensive prediction of nitrogen leaching loss.

Description

Technical Field

[0001] The present application relates to the field of nitrogen leaching loss prediction, and in particular to a method, device and medium for predicting nitrogen leaching loss in farmland. Background Art

[0002] Due to population growth and the improvement of people's living standards, the demand for food production continues to grow, and large amounts of water and fertilizer input have become the main measures to increase crop yields. Part of the excessive nitrogen fertilizer applied in farmland remains in the soil, while a large part will migrate with water and leached to areas outside the crop root zone and enter the groundwater, causing non-point source pollution and eutrophication of water bodies and many other environmental problems. Therefore, clarifying the characteristics of nitrogen leaching in farmland production and quantitatively predicting nitrogen leaching are of great significance to improving agricultural production efficiency and protecting the environment.

[0003] At present, the prediction models for nitrogen leaching mainly focus on the impact of a single factor on nitrogen leaching, and most of them are exponential and linear models. Due to the diversity and large differences in soil and environmental conditions in different test areas, predicting nitrogen leaching by a single factor cannot fully explain the complex changes in nitrogen leaching. Some mechanism models [denitrification-decomposition model (DNDC), water and nitrogen management model (WNMM), etc.] require more parameters, and the simulation accuracy is affected by multiple conditions.

[0004] The existing methods for quantitatively predicting nitrogen leaching from farmland are mainly single-factor linear, exponential or polynomial regression and multi-factor linear or stepwise regression. However, most of the linear, exponential or polynomial regression and multi-factor linear or stepwise regression are based on the mathematical principle of least squares method (estimating the regression coefficient by minimizing the residual sum of squares, so as to minimize the difference between the model prediction value and the actual observation value) to establish relevant models. Based on this, the existing technology has the following disadvantages:

[0005] (1): The conventional fitting model has a low degree of fit and poor prediction effect.

[0006] (2): It is necessary to gradually screen more indicators in order to finally determine the required relevant indicators. The operation process is relatively cumbersome.

[0007] (3): Conventional regression methods are difficult to operate on text-type variables such as soil type.

[0008] (4): The accuracy of the prediction results of conventional models is greatly affected by soil heterogeneity and other factors. The models are mostly applicable to specific locations and specific experimental conditions, and have poor applicability in larger areas.

[0009] In order to solve the above problems existing in the prior art, machine learning is now often used to predict nitrogen leaching loss (for example, Chinese patent application CN115965121A). However, due to the "black box effect" of machine learning, the prediction results are difficult to explain, and accurate and comprehensive prediction of nitrogen leaching loss cannot be achieved in a true sense. Summary of the invention

[0010] The purpose of this application is to provide a method, device and medium for predicting nitrogen leaching loss in farmland, which can solve the problem of difficult-to-interpret results caused by the "black box effect" of machine learning, and at the same time, can achieve accurate and comprehensive prediction of nitrogen leaching loss.

[0011] To achieve the above objectives, this application provides the following solutions:

[0012] In a first aspect, the present application provides a method for predicting nitrogen leaching loss from farmland, comprising:

[0013] Obtain farmland nitrogen leaching test data; the farmland nitrogen leaching test data include: soil properties, sampling measures, farmland management measures and total nitrogen leaching amount; the soil properties include: soil pH, organic matter content, soil total nitrogen content, clay content, sand content and soil type; sampling measures include: sampling method and sampling depth; farmland management measures include: amount of straw returned to the field, total nitrogen application and water input; total nitrogen application includes: nitrogen application of chemical fertilizers, nitrogen input from straw returned to the field and nitrogen input from organic fertilizers; water input includes: rainfall and irrigation;

[0014] Preprocessing the farmland nitrogen leaching test data;

[0015] Constructing a training set and a test set based on the preprocessed farmland nitrogen leaching test data;

[0016] Build an initial XGBoost regression model based on the determined hyperparameters;

[0017] The initial XGBoost regression model is trained using the training set. During the training process, the hyperparameters are optimized using the Bayesian optimization method and the five-fold cross validation method to obtain the XGBoost regression model, and the SHAP value of each sample in the training set is determined using the Shapley additive interpretation method;

[0018] Use the test set to test whether the XGBoost regression model meets the set conditions;

[0019] If the XGBoost regression model does not meet the set conditions, the hyperparameters are adjusted, and the process returns to the step of constructing an initial XGBoost regression model based on the determined hyperparameters;

[0020] If the XGBoost regression model meets the set conditions, the XGBoost regression model is used as the nitrogen leaching loss prediction model;

[0021] Acquire nitrogen leaching loss data of the farmland to be predicted, and preprocess the nitrogen leaching loss data of the farmland to be predicted to obtain preprocessed data; the nitrogen leaching loss data of the farmland to be predicted includes: soil properties of the farmland to be predicted, sampling measures of the farmland to be predicted, farmland management measures of the farmland to be predicted, and total nitrogen leaching loss of the farmland to be predicted;

[0022] The pre-processed data is input into the nitrogen leaching loss prediction model to obtain a nitrogen leaching loss prediction result.

[0023] Optionally, the preprocessing includes: missing value processing, logarithmic processing, data standardization and one-hot encoding processing.

[0024] Optionally, preprocessing the farmland nitrogen leaching test data includes:

[0025] Processing missing values ​​on the farmland nitrogen leaching test data to obtain first processed data;

[0026] performing logarithmic processing on the first processed data to obtain second processed data;

[0027] performing standardization processing on the second processed data to obtain third processed data;

[0028] The third processed data is subjected to one-hot encoding processing to obtain the pre-processed farmland nitrogen leaching test data.

[0029] Optionally, the missing value processing is performed on the farmland nitrogen leaching test data, including:

[0030] When soil properties, sampling measures or farmland management measures are missing, delete the missing data;

[0031] When the total nitrogen leaching loss is missing, for the data with nitrate nitrogen leaching loss, the missing total nitrogen leaching loss will be converted according to the ratio of 95% of nitrate nitrogen leaching loss; for the data without nitrate nitrogen leaching loss, the missing data of total nitrogen leaching loss will be deleted.

[0032] Optionally, constructing a training set and a test set based on the pre-processed farmland nitrogen leaching test data includes:

[0033] The pandas library is used to divide the farmland nitrogen leaching test data into independent variables and dependent variables; the independent variables include: soil properties, sampling measures and farmland management measures; the dependent variables include: total nitrogen leaching loss;

[0034] The independent variable and the dependent variable are randomly divided by using the sklearn library to obtain the training set and the test set.

[0035] Optionally, Bayesian optimization and five-fold cross validation are used to optimize hyperparameters, including:

[0036] Randomly sample a set of hyperparameters within the set hyperparameter range as the initial hyperparameters and perform Bayesian optimization;

[0037] In each Bayesian optimization iteration, the sampling function values ​​of all candidate points are calculated based on the current proxy model and the existing observations, and the point with the highest sampling function value is selected as the next point for Bayesian optimization;

[0038] A five-fold cross validation method is adopted to divide the training set into n subsets, and n-1 subsets are used in turn to train the next point, and the remaining subset is used to verify the next point, and the mean square error is calculated n times respectively, and the average value of the n mean square errors is determined;

[0039] After the optimization process is completed, the hyperparameters corresponding to the point with the smallest mean square error are selected as the optimized hyperparameters.

[0040] Optionally, the SHAP value of each sample in the training set is determined using the Shapley additive interpretation method, including:

[0041] The Kernel SHAP method is adopted, based on the concept of Shapley value, and a kernel function is used to determine the SHAP value of each sample in the training set.

[0042] Optionally, the SHAP value of each sample in the training set is determined using the Shapley additive interpretation method, including:

[0043] The Tree SHAP method is used to determine the SHAP value of each sample in the training set by iteratively traversing the tree upwards, taking advantage of the properties of the tree structure.

[0044] In a second aspect, the present application provides a computer device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-mentioned method for predicting nitrogen leaching loss in farmland.

[0045] In a third aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-mentioned method for predicting nitrogen leaching loss in farmland.

[0046] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0047] The present application provides a method, device and medium for predicting nitrogen leaching loss in farmland. The present application constructs a model based on farmland nitrogen leaching loss test data including soil properties, sampling measures, farmland management measures and total nitrogen leaching loss, taking into account the influence of multiple factors on total nitrogen leaching loss, and solving the problem that conventional models have poor prediction effects through a single variable. During the training process, the Shapley additive interpretation method is used to determine the SHAP value of each sample in the training set, which can quickly screen data with higher importance and solve the problem that the results are difficult to explain due to the "black box effect" of machine learning. By optimizing hyperparameters using the Bayesian optimization method and the five-fold cross-validation method, the accuracy of the model prediction can be improved, thereby achieving accurate and comprehensive prediction of nitrogen leaching loss. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0049] Figure 1 This is an application environment diagram of a method for predicting nitrogen leaching loss in farmland in one embodiment of the present application;

[0050] Figure 2 A schematic diagram of a flow chart of a method for predicting nitrogen leaching loss in farmland provided in one embodiment of the present application;

[0051] Figure 3 An example diagram of SHAP value visualization provided in an embodiment of the present application;

[0052] Figure 4 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0053] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0054] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0055] The method for predicting farmland nitrogen leaching provided in the embodiment of the present application can be applied to Figure 1In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be set up separately, or it can be integrated on the server 104, or it can be placed on the cloud or other servers. The terminal 102 can send the nitrogen leaching loss data of the farmland to be predicted to the server 104. After the server 104 receives the nitrogen leaching loss data of the farmland to be predicted, the server 104 pre-processes the nitrogen leaching loss data of the farmland to be predicted to obtain pre-processed data. The pre-processed data is input into the constructed nitrogen leaching loss prediction model to obtain the nitrogen leaching loss prediction result. The server 104 can feed back the obtained nitrogen leaching loss prediction result to the terminal 102. In addition, in some embodiments, the method for predicting nitrogen leaching loss in farmland can also be implemented independently by the server 104 or the terminal 102. For example, the terminal 102 can directly perform nitrogen leaching loss prediction processing on the nitrogen leaching loss data of the farmland to be predicted, or the server 104 can obtain the nitrogen leaching loss data of the farmland to be predicted from the data storage system and perform nitrogen leaching loss prediction processing on the nitrogen leaching loss data of the farmland to be predicted.

[0056] The terminal 102 may be, but is not limited to, various desktop computers, laptop computers, smart phones, tablet computers, IoT devices, and portable wearable devices. The IoT devices may be smart speakers, smart TVs, smart air conditioners, smart vehicle-mounted devices, etc. The portable wearable devices may be smart watches, smart bracelets, head-mounted devices, etc. The server 104 may be implemented as an independent server or a server cluster consisting of multiple servers, or may be a cloud server.

[0057] In an exemplary embodiment, Figure 2 As shown, a method for predicting nitrogen leaching loss in farmland is provided. The method is executed by a computer device, and can be executed by a computer device such as a terminal or a server alone, or by a terminal and a server together. In the embodiment of the present application, the method is applied to Figure 1 The server 104 in the example is used as an example to illustrate, including the following steps 200 to 209. Among them:

[0058] Step 200: Obtain farmland nitrogen leaching test data. Farmland nitrogen leaching test data mainly include but are not limited to soil properties, sampling measures, farmland management measures and total nitrogen leaching loss. Soil properties include: soil pH, organic matter content, soil total nitrogen content, clay content, sand content and soil type. Sampling measures include: sampling method and sampling depth. Farmland management measures include: amount of straw returned to the field, total nitrogen application and water input. The total nitrogen application includes: nitrogen application of chemical fertilizers, nitrogen input from straw returned to the field and nitrogen input from organic fertilizers. Water input includes: rainfall and irrigation.

[0059] Step 201: pre-processing the farmland nitrogen leaching test data.

[0060] Step 202: construct a training set and a test set based on the preprocessed farmland nitrogen leaching test data.

[0061] Step 203: constructing an initial XGBoost regression model based on the determined hyperparameters. For example, a set of hyperparameters is randomly sampled within the set hyperparameter range as the determined hyperparameters to construct the initial XGBoost regression model.

[0062] Step 204: The initial XGBoost regression model is trained using the training set. During the training process, the hyperparameters are optimized using the Bayesian optimization method and the five-fold cross validation method to obtain the XGBoost regression model, and the SHAP value of each sample in the training set is determined using the Shapley additive interpretation method. In the set hyperparameter range, the hyperparameters are optimized using the Bayesian optimization method and the five-fold cross validation method.

[0063] Step 205: Use the test set to test whether the XGBoost regression model meets the set conditions.

[0064] Step 206: If the XGBoost regression model does not meet the set conditions, adjust the hyperparameters and return to step 203.

[0065] Step 207: If the XGBoost regression model meets the set conditions, the XGBoost regression model is used as the nitrogen leaching loss prediction model.

[0066] Step 208: Obtain the nitrogen leaching loss data of the farmland to be predicted, and pre-process the nitrogen leaching loss data of the farmland to be predicted to obtain pre-processed data. The nitrogen leaching loss data of the farmland to be predicted includes: soil properties of the farmland to be predicted, sampling measures of the farmland to be predicted, farmland management measures of the farmland to be predicted, and total nitrogen leaching loss of the farmland to be predicted.

[0067] Step 209: input the pre-processed data into the nitrogen leaching loss prediction model to obtain the nitrogen leaching loss prediction result.

[0068] By implementing the above steps 200 to 209, the present application can quickly screen data with a high degree of importance, solve the problem of difficult-to-interpret results due to the "black box effect" of machine learning, and optimize hyperparameters by using the Bayesian optimization method and the five-fold cross-validation method, thereby improving the accuracy of model predictions and achieving accurate and comprehensive predictions of nitrogen leaching losses. In addition, the present application can also solve the problem of poor prediction effects of conventional models using a single variable by using farmland nitrogen leaching loss test data including soil properties, sampling measures, farmland management measures, and total nitrogen leaching losses as the data basis for building a model.

[0069] In another exemplary embodiment of the present application, relevant data in the farmland nitrogen leaching test study are collected from data platforms such as experiments, HowNet and Web of Science. All soil property data, sampling measures, and farmland management measures in the relevant data of the farmland nitrogen leaching test study are obtained as independent variables, and the total nitrogen leaching amount is used as a dependent variable to form a data strip. A data strip includes all independent variables and dependent variables. Afterwards, further sorting is required, that is, checking whether the article has a numerical value or text provided for each data strip to form the farmland nitrogen leaching test data.

[0070] In another exemplary embodiment of the present application, in order to further improve the accuracy of data processing and eliminate the influence of dimension, the preprocessing method in the above step 201 may include but is not limited to four processes of missing value processing, logarithmic processing, data standardization and one-hot encoding processing (using pandas library and numpy library). Based on this, the implementation process of step 201 can be described as:

[0071] (1) The missing values ​​of the farmland nitrogen leaching test data were processed to obtain the first processed data.

[0072] For example, the process of missing value processing is: if there is a missing value in the independent variable, delete this data. For literature with missing soil clay and sand content, find literature data with the same longitude and latitude as the test area of ​​the missing literature. For literature with missing dependent variables (total nitrogen leaching loss), for literature that only provides nitrate nitrogen leaching loss, its total nitrogen leaching loss is converted according to the ratio of nitrate nitrogen leaching loss to 95%. If there is no data related to nitrogen leaching loss, delete this data.

[0073] (2) Perform logarithmic processing on the first processed data to obtain second processed data.

[0074] For example, the process of logarithmic processing is: since the distribution of the dependent variable (total nitrogen leaching loss) deviates from the normal distribution, the data of the dependent variable (total nitrogen leaching loss) is calculated by taking the logarithm (ln or log 10 ). The purpose of logarithmic processing is to make the data closer to normal distribution and reduce the impact of outliers.

[0075] (3) The second processed data is standardized to obtain third processed data.

[0076] For example, the process of standardization is: all independent variables and logarithmically processed dependent variables are standardized using the Z-score standardization formula to standardize the second processed data. The mean of the standardized data is 0 and the standard deviation is 1. Assume that the second processed data is 1, 2, 3, which becomes -1, 0, 1 after standardization.

[0077] The standardization formula is:

[0078] Z=(x-μ) / σ.

[0079] Wherein, μ is the mean of the second processed data, σ is the standard deviation of the second processed data, Z is the standardized result, and x is the data to be standardized.

[0080] The purpose of standardization is to eliminate the dimensional effects between different variables.

[0081] (4) Perform one-hot encoding on the third processed data to obtain the preprocessed farmland nitrogen leaching test data.

[0082] For example, the process of one-hot encoding is to convert the categorical variables such as soil type in the study into the coding form of 0 and 1 according to the category. As shown in Table 1, the soil types are divided into three types: brown soil, tidal soil, and brown soil. The soil types after one-hot encoding are shown in Table 2.

[0083] Table 1 Soil types before one-hot encoding

[0084]

[0085] Table 2 Soil type table after one-hot encoding

[0086]

[0087] In this embodiment, the purpose of performing one-hot encoding is to solve the problem of changes in the weights of categorical variables during regression caused by conventional encoding, to avoid the model's preference for categorical variables, and to make the interpretation and analysis of variables more accurate and reliable.

[0088] Based on the above description, an example of the nitrogen leaching test data of farmland after pretreatment is shown in Table 3.

[0089] Table 3 Example of data of nitrogen leaching test on farmland after pretreatment

[0090]

[0091] In another exemplary embodiment of the present application, the training set is the object of the XGBoost regression model construction application, and the test set is the data basis for testing the XGBoost regression model. In order to further increase the prediction accuracy of the constructed model, in this embodiment, the pandas library is used to divide the independent variables and the dependent variables (with the total nitrogen leaching loss as the dependent variable and the other variables as independent variables) to form data strips, and the sklearn library is used to randomly divide all the data strips into training sets and test sets. For example, the ratio of the training set to the test set can be 4:1. Based on this, assuming that there are 200 data strips in total, the training set contains 160 data strips after random division, and the test set contains 40 data strips.

[0092] In actual application, predictions can be made by building multiple decision trees based on the data in the training set. Each decision tree outputs a continuous value, and then the prediction results of the decision trees are integrated by taking the average or weighted average to obtain the final result to complete the construction of the XGBoost regression model. After the XGBoost regression model is built, the test set is used to test the XGBoost regression model, that is, the established XGBoost regression model is tested and evaluated on the test set.

[0093] In another exemplary embodiment of the present application, in order to further improve the accuracy of model prediction and thus achieve accurate and comprehensive prediction of nitrogen leaching loss in a real sense, in this embodiment, in the above step 204, the process of optimizing hyperparameters using the Bayesian optimization method and the five-fold cross validation method can be described as follows:

[0094] (1) Establish the hyperparameter set you want to search, that is, determine the hyperparameter range. For example, the established hyperparameter set is shown in Table 4, including but not limited to the maximum depth of each decision tree, learning rate, sample training ratio, feature ratio considered by the decision tree, etc.

[0095]

[0096] (2) Randomly sample a set of hyperparameters within the set hyperparameter range as the initial hyperparameters for Bayesian optimization. In each Bayesian optimization iteration, the sampling function values ​​of all candidate points are calculated based on the current proxy model and the existing observations, and the point with the highest sampling function value is selected as the next point for Bayesian optimization.

[0097] For example, Bayesian search is performed on the established hyperparameter set on the validation set. Bayesian optimization is a method of optimizing the objective function through Bayesian inference. In parameter search, Bayesian optimization can help find the optimal parameter combination to maximize or minimize the selected objective function. The implementation process can be:

[0098] ① Select a surrogate model: Select Bayesian optimization to use a surrogate model (such as a Gaussian process) to estimate the underlying surface of the objective function. In each optimization step, the surrogate model predicts the performance of unknown parameter combinations based on the relationship between the observed parameter-performance pairs.

[0099] ② Initial sampling (initializing the Bayesian search object): Start the optimization process by randomly sampling a set of initial hyperparameters from the hyperparameter set just established (as shown in Table 4).

[0100] ③ Iterative optimization (running Bayesian search): Determining the next point in Bayesian optimization is done through a sampling function, which helps Bayesian optimization choose where to evaluate the objective function in each iteration in order to effectively balance the strategies of exploration and exploitation to find the area with the highest probability of the global optimal solution. For example, the sampling function used is Expected Improvement (EI). EI calculates the probability at each point that it may be better than the current best observation based on the current surrogate model. Specifically, for each candidate point x, EI can be calculated as follows:

[0101] EI = E[max(f(x) - fb(x), 0)].

[0102] Where f(x) is the mean prediction of the proxy model at the candidate point x, fb(x) is the best currently known observation, and E[ ] is the expected function. EI evaluates the possible improvement of evaluating the objective function at the candidate point x. The higher the value, the greater the expected benefit of evaluating at that point, so it is more likely to choose that point for evaluation.

[0103] In each iteration, the sampling function values ​​are calculated for all candidate points based on the current surrogate model and the existing observations, and the point with the highest value is selected as the next point to be evaluated.

[0104] (3) The parameters are evaluated by using the five-fold cross-validation method (the higher the R2 in the five-fold cross-validation, the better the hyperparameters), and the relevant experience in parameter adjustment is used to determine the range of the hyperparameters. Based on this, the five-fold cross-validation method is adopted to divide the training set into five subsets, and four of the subsets are used in turn to train the next point, and the remaining subset is used to verify the next point. The mean square error is calculated five times respectively, and the average of the five mean square errors is determined. After the optimization process is completed, the hyperparameter corresponding to the point with the smallest average mean square error is selected as the optimized hyperparameter.

[0105] For example, use the selected parameter combination to train the model on the training set (the input in the training is the independent variable and the output is the dependent variable), adopt a five-fold cross validation, divide the training set into five subsets, use four of the subsets for training in turn, and then use the remaining subset for validation, calculate the mean square error five times and find the average. The calculation formula of mean square error (MSE) is:

[0106] .

[0107] In the formula, is the mean square error value, is the true value of the test set target, is the target prediction value for the test set, Indicates samples, .

[0108] After a certain number of iterations, the optimization process is stopped and the best parameters are returned (the optimal parameters are obtained). That is, after the optimization process is completed, the model hyperparameter set with the smallest mean square error in the five-fold validation process is selected to determine the hyperparameters.

[0109] In this embodiment, the process of optimizing hyperparameters using the Bayesian optimization method and the five-fold cross validation method can be implemented by calling BayesSearchCV in the skopt library.

[0110] In addition, the parameter search method in this embodiment can also be a grid search and a random search method to achieve a similar model building effect on a small parameter set.

[0111] In another exemplary embodiment of the present application, the searched hyperparameters (i.e., optimized hyperparameters) are used to establish an XGBoost regression model with the independent variables on the training set as input and the corresponding dependent variables as output, and this model is used to input the independent variables on the test set to obtain the predicted value of the dependent variable, and this value is tested with the actual value of the dependent variable to calculate the goodness of fit ( R 2 ), repeat 200 times and take R 2 The establishment of XGBoost regression model can be achieved by calling XGBoost library.

[0112] in, R 2 The calculation formula is:

[0113] .

[0114] In the formula, is the target mean of the test set.

[0115] Further, based on the description of this embodiment, an importance analysis of the output prediction results and features is performed. For example, after the XGBoost regression model is established, the dependent variable can be quantitatively predicted by inputting independent variables, and an importance analysis based on the Shapley Additive explanation method (SHAP) is used. During the training process, for each sample, its corresponding SHAP value is calculated, that is, the Shapley additive explanation method is used to determine the SHAP value of each sample in the training set. It can be achieved through two main methods: one is to use the Kernel SHAP method, based on the concept of Shapley value, and use the kernel function to determine the SHAP value of each sample in the training set. It approximates the Shapley value by sampling the feature space and calculating the SHAP value of each sample. The other is to use the Tree SHAP method, for tree models (such as decision trees, random forests, etc.), using the properties of the tree structure, and determining the SHAP value of each sample in the training set by iteratively traversing the tree upward.

[0116] In this embodiment, the calculation of the SHAP value can be achieved by calling the shap library in Python. The data with a higher SHAP value has a higher degree of importance in the total nitrogen leaching loss. For example, the obtained SHAP value example is as follows Figure 3 As shown, it shows that the amount of nitrogen fertilizer has the greatest impact on the total nitrogen leaching loss. Based on this, the difficulty of feature observation and prediction effect are comprehensively considered, the relevant features are retained and a model is established for prediction. For example, if the clay content and sand content are difficult to measure and the cost is high, the XGBoost regression model with water input and total nitrogen application as independent variables is selected. This XGBoost regression model is used for prediction. The water input and total nitrogen application data recorded in production are used as input into the model to obtain the predicted total nitrogen leaching loss results.

[0117] In another exemplary embodiment of the present application, the nitrogen leaching loss data of the farmland to be predicted is preprocessed according to the same steps and parameters as the training set, and the output data, i.e., the total nitrogen leaching loss, can be reversely restored according to the preprocessing process, that is, after inversely solving the Z-score standardization formula, the exponential operation can be performed to obtain the actual prediction result.

[0118] In summary, compared with the prior art, this application also has the following advantages:

[0119] 1. The SHAP used in this application can quickly screen indicators with higher importance, solving the problem that the results are difficult to explain due to the "black box effect" of machine learning.

[0120] 2. This application uses the one-hot encoding method to encode text type variables such as soil type, which solves the problem of weight changes in categorical variables during regression caused by conventional encoding.

[0121] 3. This application simultaneously considers the impact of multiple factors on total nitrogen leaching loss and constructs a model through multiple prediction variables, thus solving the problem of poor prediction effect of conventional models through a single variable.

[0122] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Figure 4 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store nitrogen leaching loss prediction processing data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for predicting nitrogen leaching loss in farmland is implemented.

[0123] Those skilled in the art will understand that Figure 4 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components. In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0124] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0125] In an exemplary embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0126] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0127] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0128] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.

[0129] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0130] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A method for predicting nitrogen leaching from farmland, characterized in that: The method for predicting farmland nitrogen leaching includes: Obtain farmland nitrogen leaching test data; the farmland nitrogen leaching test data include: soil properties, sampling measures, farmland management measures and total nitrogen leaching amount; the soil properties include: soil pH, organic matter content, soil total nitrogen content, clay content, sand content and soil type; sampling measures include: sampling method and sampling depth; farmland management measures include: amount of straw returned to the field, total nitrogen application and water input; total nitrogen application includes: nitrogen application of chemical fertilizers, nitrogen input from straw returned to the field and nitrogen input from organic fertilizers; water input includes: rainfall and irrigation; Preprocessing the farmland nitrogen leaching test data; Constructing a training set and a test set based on the preprocessed farmland nitrogen leaching test data; Build an initial XGBoost regression model based on the determined hyperparameters; The initial XGBoost regression model is trained using the training set. During the training process, the hyperparameters are optimized using the Bayesian optimization method and the five-fold cross validation method to obtain the XGBoost regression model, and the SHAP value of each sample in the training set is determined using the Shapley additive interpretation method; A test set is used to test whether the XGBoost regression model meets the set conditions; wherein, the independent variable on the test set is used as input to obtain the value of the predicted dependent variable, and this value is tested with the actual dependent variable value to calculate the goodness of fit; If the XGBoost regression model does not meet the set conditions, the hyperparameters are adjusted, and the process returns to the step of constructing an initial XGBoost regression model based on the determined hyperparameters; If the XGBoost regression model meets the set conditions, the XGBoost regression model is used as the nitrogen leaching loss prediction model; Acquire nitrogen leaching loss data of the farmland to be predicted, and preprocess the nitrogen leaching loss data of the farmland to be predicted to obtain preprocessed data; the nitrogen leaching loss data of the farmland to be predicted includes: soil properties of the farmland to be predicted, sampling measures of the farmland to be predicted, farmland management measures of the farmland to be predicted, and total nitrogen leaching loss of the farmland to be predicted; The pre-processed data is input into the nitrogen leaching loss prediction model to obtain a nitrogen leaching loss prediction result.

2. The method for predicting farmland nitrogen leaching according to claim 1, characterized in that: The preprocessing includes: missing value processing, logarithmic processing, data standardization and one-hot encoding processing.

3. The method for predicting farmland nitrogen leaching according to claim 2, characterized in that: The farmland nitrogen leaching test data is preprocessed, including: Processing missing values ​​on the farmland nitrogen leaching test data to obtain first processed data; performing logarithmic processing on the first processed data to obtain second processed data; performing standardization processing on the second processed data to obtain third processed data; The third processed data is subjected to one-hot encoding processing to obtain the pre-processed farmland nitrogen leaching test data.

4. The method for predicting farmland nitrogen leaching according to claim 3, characterized in that: The missing value processing of the farmland nitrogen leaching test data includes: When soil properties, sampling measures or farmland management measures are missing, delete the missing data; When the total nitrogen leaching loss is missing, for the data with nitrate nitrogen leaching loss, the missing total nitrogen leaching loss will be converted according to the ratio of 95% of nitrate nitrogen leaching loss; for the data without nitrate nitrogen leaching loss, the missing data of total nitrogen leaching loss will be deleted.

5. The method for predicting farmland nitrogen leaching according to claim 1, characterized in that: A training set and a test set are constructed based on the pre-processed farmland nitrogen leaching test data, including: The pandas library is used to divide the farmland nitrogen leaching test data into independent variables and dependent variables; the independent variables include: soil properties, sampling measures and farmland management measures; the dependent variables include: total nitrogen leaching loss; The independent variable and the dependent variable are randomly divided by using the sklearn library to obtain the training set and the test set.

6. The method for predicting farmland nitrogen leaching according to claim 1, characterized in that: Bayesian optimization and five-fold cross-validation were used to optimize hyperparameters, including: Randomly sample a set of hyperparameters within the set hyperparameter range as the initial hyperparameters and perform Bayesian optimization; In each Bayesian optimization iteration, the sampling function values ​​of all candidate points are calculated based on the current proxy model and the existing observations, and the point with the highest sampling function value is selected as the next point for Bayesian optimization; A five-fold cross validation method is adopted to divide the training set into n subsets, and n-1 subsets are used in turn to train the next point, and the remaining subset is used to verify the next point, and the mean square error is calculated n times respectively, and the average value of the n mean square errors is determined; After the optimization process is completed, the hyperparameters corresponding to the point with the smallest mean square error are selected as the optimized hyperparameters.

7. The method for predicting farmland nitrogen leaching according to claim 1, characterized in that: The Shapley additive interpretation method is used to determine the SHAP value of each sample in the training set, including: The Kernel SHAP method is adopted, based on the concept of Shapley value, and a kernel function is used to determine the SHAP value of each sample in the training set.

8. The method for predicting farmland nitrogen leaching according to claim 1, characterized in that: The Shapley additive interpretation method is used to determine the SHAP value of each sample in the training set, including: The Tree SHAP method is used to determine the SHAP value of each sample in the training set by iteratively traversing the tree upwards, taking advantage of the properties of the tree structure.

9. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method for predicting nitrogen leaching loss from farmland according to any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for predicting nitrogen leaching loss from farmland according to any one of claims 1 to 8 is implemented.

Images