Soil-water characteristic curve prediction method for optimizing XGBoost model

Abstract

The invention discloses a soil-water characteristic curve prediction method based on an ensemble learning algorithm, aims to solve the problem that soil-water characteristic curve testing is strenuous and time-consuming through an improved XGBoost model, and belongs to the field of geotechnical engineering research. The method comprises the following steps: step 1, collecting soil-water characteristic curve data and constructing a database; 2, dividing database data into a training set (70%) and a test set (30%); and step 3, optimizing unknown parameters in the XGBoost model by using a grid search algorithm. And step 4, monotone weight constraint is carried out on the optimized XGBoost model. And step 5, predicting the data of the test set by using the final XGBoost model. The XGBoost model is optimized based on a grid search algorithm and a monotonic weight constraint means, and an efficient and stable soil-water characteristic curve prediction method is provided for geotechnical engineering construction and maintenance.

Description

Technical Field

[0001] The present invention relates to the field of geotechnical engineering technology, and in particular to a soil-water characteristic curve prediction method based on a grid search algorithm and a monotonic weight constraint method to optimize an XGBoost model. Background Art

[0002] Soil-water characteristic curve (SWCC) plays a vital role in determining the behavior of unsaturated soil in many geotechnical engineering applications, such as resource development in arid and semi-arid areas, slope stability problems, unsaturated shear strength and hydraulic conductivity function. SWCC represents the relationship between volumetric water content and matrix suction, which is a key aspect in understanding the mechanical response of soil under unsaturated conditions. Therefore, a deep understanding of SWCC can improve the reliability of engineering solutions in saturated-unsaturated soil systems. At present, the main methods used to obtain soil-water characteristic curves in this field include empirical model prediction, theoretical model prediction and experimental measurement. The shortcomings of the above methods can be mainly summarized as complex model calculation form, poor applicability, long test cycle and low efficiency. However, machine learning has many advantages such as strong nonlinear fitting ability, self-adaptation and self-learning ability, and has been widely used in many fields such as prediction, classification, function approximation, comprehensive evaluation and pattern recognition. Among them, XGBoost is an efficient and flexible machine learning algorithm based on decision trees. It integrates multiple weak learners (usually decision trees) to form a powerful model, and iteratively optimizes the prediction of the model by minimizing the loss function and using the gradient descent method to improve the overall prediction accuracy. In addition, XGBoost can capture the complex and highly nonlinear relationship between volumetric water content and matrix suction by integrating decision trees, and can process soil water characteristic curve data under the influence of multiple factors through efficient parallel computing and gradient boosting framework, so it has significant advantages in dealing with nonlinear problems like SWCC. However, since XGBoost has many hyperparameters (such as tree depth, learning rate, etc.), these parameters need to be carefully tuned to achieve the best results, and the tuning process is relatively complex and time-consuming. In addition, when developing and building XGBoost models, it is necessary to ensure consistency with known physical mechanisms (such as monotonicity). Summary of the invention

[0003] Technical problem: In order to overcome the defects and shortcomings of the existing technology, a soil-water characteristic curve prediction method based on grid search algorithm and monotonic weight constraint to optimize the XGBoost model is proposed. This method comprehensively considers the influence of matrix suction, dry density, specific gravity, porosity, non-uniformity coefficient, curvature coefficient and the proportion of silt, sand and clay content on soil moisture content, and constructs and optimizes the XGBoost model based on the basic principles of XGBoost, grid search algorithm and monotonic weight constraint. This model will be applied to the prediction of soil-water characteristic curves of different soils, and can provide a new method for the measurement of soil-water characteristic curves under saturated-unsaturated systems and different working conditions.

[0004] In order to achieve the above technical objectives, the present invention provides a prediction method for soil-water characteristic curve based on a grid search algorithm and a monotonic weight constraint method to optimize the XGBoost model, which mainly includes the following steps:

[0005] Step 1: Collect the moisture content data of different soil types under different working conditions such as matrix suction, dry density, specific gravity, porosity, unevenness coefficient, curvature coefficient, and the proportion of silt, sand, and clay content, and build a soil-water characteristic curve database;

[0006] Step 2: Randomly divide the database sample set, and use 70% of the soil-water characteristic curve data in the sample set for the training phase to train the relationship between matrix suction, dry density, specific gravity, porosity, unevenness coefficient, curvature coefficient, and the proportion of silt, sand, and clay content and moisture content. The remaining 30% of the data is used in the testing phase to verify the stability and generalization ability of the model;

[0007] Step 3: Based on the principle and structure of XGBoost, use the grid search algorithm to optimize the XGBoost model and obtain the optimal unknown parameter combination of XGBoost;

[0008] Step 4: Select the optimal XGBoost structure optimized by the grid search algorithm for monotonic weight constraint, set the weight of matrix suction in the input variable to -1, and set other variables to 0, so that the predicted soil-water characteristic curve conforms to the basic physical law, that is, there is a monotonic decreasing relationship between water content and matrix suction, so as to enhance the stability and interpretability of the model;

[0009] Step 5: Use the test set data that has not gone through the training phase as input, and use the XGBoost model optimized in step 4 to predict the soil-water characteristic curve.

[0010] The moisture content data of different soil types collected under different working conditions such as matrix suction, dry density, specific gravity, porosity, uniformity coefficient, curvature coefficient, and the proportion of silt, sand and clay content were used to construct a soil-water characteristic curve database. 70% of the data was used for the development and construction of the XGBoost model, and 30% of the data was used for testing and verification of the model.

[0011] Objective function of the XGBoost model with K trees and T leaf nodes at t iterations The loss function l(y is the difference between the predicted value and the actual value rea ,y pre ) and regularization term to prevent the model from overfitting XGBoost optimizes the actual value y by iterative optimization rea and the predicted value y pre The loss function gradient is used to train the next weak learner and add it to the model To reduce the residual error of the previous round. In order to minimize the objective function, the Taylor series of the loss function is expanded to the second order During the iteration process, since each sample will eventually fall into a leaf node, the objective function is transformed from a function about each sample to a function about each leaf node, thereby obtaining the optimal leaf node weight. and the objective function value Where n represents the number of samples in the data set, and γ and λ are regularization parameters used to control the complexity of the tree. t (x i ) is the newly generated tree for sample x in the tth iteration i The predicted value, g i yes f t (x i ), h i yes f t (x i ), G j and H j are the g of all samples in leaf node j. i and h i The entire XGBoost structure has many unknown parameters such as learning rate, number of trees, maximum tree depth, regularization parameters γ and λ, etc.

[0012] Use the grid search algorithm to search for XGBoost's learning rate, number of trees, maximum tree depth, regularization parameters γ and λ to avoid human errors caused by manual parameter adjustment. The grid search algorithm optimizes the design of XGBoost as follows:

[0013] a. Initialization parameters and definition parameter ranges: the learning rate range is [0.01, 0.05, 0.1, 0.2], the number of trees range is [100, 300, 500], the maximum depth of the tree range is [3, 5, 7, 9], and the regularization parameters γ and λ range are [0, 0.1, 0.5, 1.0].

[0014] b. Generate parameter combinations and train the model: Generate all possible parameter combinations based on the defined parameter space. For each generated parameter combination, use the training data to train the model and evaluate its performance on the test set. The evaluation indicators are:

[0015]

[0016] c. Record results and select optimal parameters: After the grid search is completed, save the performance indicators of each parameter combination and obtain the best performing hyperparameter combination.

[0017] d. Model output: Retrain the model using the best parameter combination and finally output the optimized XGBoost model.

[0018] The optimal XGBoost optimized by the grid search algorithm was subjected to monotonic weight constraints. The weight of the matrix suction in the input variable was set to -1, and the weights of other variables such as dry density, specific gravity, porosity, uniformity coefficient, curvature coefficient and proportion of silt clay content were set to 0. This made the predicted soil-water characteristic curve conform to the basic physical laws, that is, there is a monotonically decreasing relationship between water content and matrix suction, thereby enhancing the stability and interpretability of the model.

[0019] Beneficial effects of the present invention:

[0020] This method collected soil-water characteristic curve data of 12 soil types under different working conditions, including clay, clay loam, loam, loamy sand, sand, sandy clay, sandy clay loam, sandy loam, silt, silt loam, silt clay and silt clay loam, totaling 6655 sets of data (divided into 605 soil-water characteristic curves), which has certain data representativeness. Based on the XGBoost model, the grid search algorithm is used to optimize the unknown parameters such as learning rate, number of trees, maximum depth of trees and regularization parameters, which reduces the human error caused by manual parameter adjustment, and selects the optimal XGBoost structure optimized by the grid search algorithm for monotonic weight constraint, sets the weight of matrix suction in the input variable to -1, and sets other variables to 0, so that the predicted soil-water characteristic curve conforms to the basic physical law, that is, it satisfies the monotonically decreasing relationship between water content and matrix suction, so as to enhance the stability and interpretability of the model. The XGBoost model after grid search algorithm and monotonic weight constraint optimization has excellent performance, and its prediction accuracy and stability are better than the traditional XGBoost model, and even better than traditional measurement and prediction methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 The present invention is a flowchart of the soil-water characteristic curve prediction method of XGBoost optimized by the grid search algorithm and monotonic weight constraint means.

[0022] Figure 2 This is the distribution of the data collected in the present invention in the soil texture triangle.

[0023] Figure 3 This is the structural diagram of XGBoost in the present invention.

[0024] Figure 4 It is an iterative optimization diagram of the grid search algorithm in the present invention.

[0025] Figure 5 The XGBoost model optimized by the grid search algorithm in the present invention predicts the soil-water characteristic curve on the training set and the test set respectively.

[0026] Figure 6 This is a comparison chart of the soil-water characteristic curve prediction effects of the XGBoost model optimized by the grid search algorithm and monotonic weight constraint method in the present invention and the traditional XGBoost model.

[0027] Specific implementation instructions

[0028] Step 1: Collect the moisture content data of different soil types under different working conditions such as matrix suction, dry density, specific gravity, porosity, unevenness coefficient, curvature coefficient, and the proportion of silt, sand, and clay content, and build a soil-water characteristic curve database;

[0029] Step 2: Randomly divide the database sample set, and use 70% of the soil-water characteristic curve data in the sample set for the training phase to train the relationship between matrix suction, dry density, specific gravity, porosity, unevenness coefficient, curvature coefficient, and the proportion of silt, sand, and clay content and moisture content. The remaining 30% of the data is used in the testing phase to verify the stability and generalization ability of the model;

[0030] Step 3: Based on the principle and structure of XGBoost, use the grid search algorithm to optimize the XGBoost model and obtain the optimal parameter combination of XGBoost;

[0031] Step 4: Select the optimal XGBoost structure optimized by the grid search algorithm for monotonic weight constraint, set the weight of matrix suction in the input variable to -1, and set other variables to 0, so that the predicted soil-water characteristic curve conforms to the basic physical law, that is, there is a monotonic decreasing relationship between water content and matrix suction, so as to enhance the stability and interpretability of the model;

[0032] Step 5: Use the test set data that has not gone through the training phase as input, and use the XGBoost model optimized in step 4 to predict the soil-water characteristic curve.

[0033] like Figure 1 As shown, the main flow chart of the soil-water characteristic curve prediction method based on the grid search algorithm and monotonic weight constraint optimization XGBoost model described in the present invention. First, the method collects soil-water characteristic curve data of 12 soil types under different working conditions, including clay, clay loam, loam, loamy sand, sand, sandy clay, sandy clay loam, sandy loam, silt, silt loam, silt clay and silt clay loam, totaling 6655 groups of data (divided into 605 soil-water characteristic curves), which has certain data representativeness. Based on the XGBoost model, the grid search algorithm is used to optimize the unknown parameters such as learning rate, number of trees, maximum depth of trees and regularization parameters, which reduces the human error caused by manual parameter adjustment. The optimal XGBoost structure optimized by the grid search algorithm is selected for monotonic weight constraint, and the weight of matrix suction in the input variable is set to -1, and other variables are set to 0, so that the predicted soil-water characteristic curve conforms to the basic physical laws, that is, the relationship between water content and matrix suction is monotonically decreasing, so as to enhance the stability and interpretability of the model. Finally, the data of the test set that has not been through the training set stage is imported into the constructed model to complete the prediction of the soil-water characteristic curve.

[0034] like Figure 2As shown, the soil types collected by the present invention are diverse and widely distributed, with a total of 12 soil types (clay, clay loam, loam, loamy sand, sand, sandy clay, sandy clay loam, sandy loam, silt, silt loam, silt clay and silt clay loam), covering all soil types in the soil texture classification standards formulated by the United States Department of Agriculture (UNSODA), and have certain data representativeness.

[0035] like Figure 3 As shown in the figure, XGBoost is selected as the basic model. The objective function of the XGBoost model with K trees and T leaf nodes in t iterations is The loss function l(y is the difference between the predicted value and the actual value rea ,y pre ) and regularization term to prevent the model from overfitting XGBoost optimizes the actual value y by iterative optimization rea and the predicted value y pre The loss function gradient is used to train the next weak learner and add it to the model To reduce the residual error of the previous round. In order to minimize the objective function, the Taylor series of the loss function is expanded to the second order During the iteration process, since each sample will eventually fall into a leaf node, the objective function is transformed from a function about each sample to a function about each leaf node, thereby obtaining the optimal leaf node weight. and the objective function value Where n represents the number of samples in the data set, and γ and λ are regularization parameters used to control the complexity of the tree. t (x i ) is the newly generated tree for sample x in the tth iteration i The predicted value, g i yes f t (x i ), h i yes f t (x i ), G j and H j are the g of all samples in leaf node j. i and h i The sum.

[0036] like Figure 4 As shown in the figure, during the iteration process, the grid search algorithm optimizes the learning rate, number of trees, maximum depth of the tree, regularization parameter and other unknown parameters of XGBoost. 2 The optimal parameter combination is found by taking the optimal value of .

[0037] like Figure 5 As shown in the figure, the optimal parameter combination of XGBoost optimized by the grid search algorithm is selected, and the training set data is used to complete the construction of the XGBoost model. Finally, the test set data without training is used to verify the model.

[0038] like Figure 6 As shown in the figure, the XGBoost model optimized by the grid search algorithm and monotonic weight constraint (Model 2) and the XGBoost model optimized only by the grid search algorithm (Model 1) are used to predict the soil-water characteristic curve of 12 different soil types, including clay, clay loam, loam, loamy sand, sand, sandy clay, sandy clay loam, sandy loam, silt, silty loam, silty clay and silty clay loam. It can be seen that the XGBoost model optimized by the grid search algorithm has a higher prediction accuracy, indicating that the grid search algorithm is an effective means to optimize the unknown parameters in the XGBoost model, but the XGBoost model optimized only by the grid search algorithm has a non-monotonic situation in the matrix suction and water content when predicting the soil-water characteristic curve. The XGBoost model optimized by the grid search algorithm and the monotonic weight constraint not only shows higher accuracy and stability when predicting the soil-water characteristic curve, but also ensures the basic physical law of forced monotonic decrease between water content and matrix suction.

[0039] In summary, the present invention utilizes the advantages of the grid search algorithm, such as strong global search capability, simple parameter optimization, and strong robustness, to determine the optimal parameter combination of XGBoost, thereby enhancing the accuracy of the XGBoost model in predicting the soil-water characteristic curve. Moreover, the XGBoost model is optimized by means of monotonic weight constraints, thereby ensuring the basic physical law of forced monotonic decrease between water content and matrix suction, thereby enhancing the stability and interpretability of the XGBoost model in predicting the soil-water characteristic curve. The present invention not only provides an effective soil-water characteristic curve calculation method for the field of geotechnical engineering research, but also further improves the prediction accuracy and obtains better results.

Claims

1. A soil-water characteristic curve prediction method based on grid optimization algorithm and monotonic weight constraint optimization XGBoost, characterized in that: The steps include: Step 1: Collect the moisture content data of different soil types under different working conditions such as matrix suction, dry density, specific gravity, porosity, unevenness coefficient, curvature coefficient, and the proportion of silt, sand, and clay content, and build a soil-water characteristic curve database; Step 2: Randomly divide the database sample set, and use 70% of the soil-water characteristic curve data in the sample set for the training phase to train the relationship between matrix suction, dry density, specific gravity, porosity, unevenness coefficient, curvature coefficient, and the proportion of silt, sand, and clay content and moisture content. The remaining 30% of the data is used in the testing phase to verify the stability and generalization ability of the model; Step 3: Based on the principle and structure of XGBoost, use the grid search algorithm to optimize the XGBoost model and obtain the optimal unknown parameter combination of XGBoost; Step 4: Select the optimal XGBoost structure optimized by the grid search algorithm for monotonic weight constraint, set the weight of matrix suction in the input variable to -1, and set other variables to 0, so that the predicted soil-water characteristic curve conforms to the basic physical law, that is, there is a monotonic decreasing relationship between water content and matrix suction, so as to enhance the stability and interpretability of the model; Step 5: Use the test set data that has not gone through the training phase as input, and use the XGBoost model optimized in step 4 to predict the soil-water characteristic curve.

2. The soil-water characteristic curve prediction method based on grid search algorithm and monotonic weight constraint means to optimize XGBoost according to claim 1, characterized in that: The objective function of the XGBoost model with K trees and T leaf nodes described in step 3 at t iterations is By the predicted value y pre and the actual value y re The difference loss function l(y rea ,y pre ) and regularization term to prevent the model from overfitting XGBoost trains the next weak learner by iteratively optimizing the gradient of the loss function between the actual value and the predicted value and adds it to the model To reduce the residual error of the previous round. In order to minimize the objective function, the Taylor series of the loss function is expanded to the second order During the iteration process, since each sample will eventually fall into a leaf node, the objective function is transformed from a function about each sample to a function about each leaf node, thereby obtaining the optimal leaf node weight. and the objective function value Where n represents the number of samples in the data set, and γ and λ are regularization parameters used to control the complexity of the tree. t (x i ) is the newly generated tree for sample x in the tth iteration i The predicted value, g i yes f t (x i ), h i yes f t (x i ), G j and H j are the g of all samples in leaf node j. i and h i The sum.

3. The soil-water characteristic curve prediction method based on grid search algorithm and monotonic weight constraint means to optimize XGBoost according to claim 1, characterized in that: The method described in step 3 optimizes the parameters of the XGBoost model by using a grid search algorithm, outputs the optimal parameter combination, and assigns the optimal parameter combination to the XGBoost model to obtain an optimized XGboost model, including the following steps: a. Initialization parameters and definition parameter ranges: the learning rate range is [0.01, 0.05, 0.1, 0.2], the number of trees range is [100, 300, 500], the maximum depth of the tree range is [3, 5, 7, 9], and the regularization parameters γ and λ range are [0, 0.1, 0.5, 1.0]. b. Generate parameter combinations and train the model: Generate all possible parameter combinations based on the defined parameter space. For each generated parameter combination, use the training data to train the model and evaluate its performance on the test set. The evaluation indicators are: c. Record the results and select the best parameters: After the grid search is completed, save the performance index R of each parameter combination 2 And get the best performing hyperparameter combination. d. Model output: Retrain the model using the best parameter combination and finally output the optimized XGBoost model.

4. The soil-water characteristic curve prediction method based on grid search algorithm and monotonic weight constraint means to optimize XGBoost according to claim 1, characterized in that: As described in step 4, the optimal XGBoost structure optimized by the grid search algorithm is subjected to monotonic weight constraints. The network weight of the matrix suction in the input variable is set to -1, and the network weights of other variables such as dry density, specific gravity, porosity, non-uniformity coefficient, curvature coefficient and silt clay content are all set to 0, so that the predicted soil-water characteristic curve conforms to the basic physical laws, that is, there is a monotonically decreasing relationship between water content and matrix suction, which enhances the stability and interpretability of the model.

Images