A method for predicting soil permeability based on PCA and CatBoost regression fusion

Abstract

The present invention discloses a method for predicting soil permeability based on the fusion of PCA and CatBoost regression, comprising collecting a number of sample soils with permeability values, using the numerical feature data of these samples as a sample set, and extracting the feature data of the soil to be predicted for permeability; data cleaning; based on the idea of the PCA principal component analysis method, performing dimensionality reduction processing on the high-dimensional data set of the samples, retaining important features and removing irrelevant features and redundant features; constructing a CatBoost regression model; inputting the feature data of the soil to be predicted for permeability into the trained CatBoost regression model to obtain the predicted permeability value of the sample soil. The present invention provides a method for predicting soil permeability that takes factors into consideration more comprehensively, predicts more accurately, and has a better processing effect on categorical features such as soil type. The present invention combines PCA with CatBoost, which can improve the processing effect of categorical features in sample soil, and can also adapt to the high dimensionality of sample soil features, thereby improving prediction accuracy.

Description

Technical Field

[0001] The present invention belongs to the technical field of soil permeability prediction, and in particular relates to a method for predicting soil permeability based on PCA and CatBoost regression fusion. Background Art

[0002] The migration of pollutants in soil is affected by soil permeability. The prediction of soil permeability is of practical significance for shortening construction period, reducing project costs, guiding engineering pollutant treatment, and promoting the development of disciplines such as soft soil mechanics.

[0003] Existing machine learning prediction techniques for soil permeability primarily involve multiple linear regression (LR) models. For example, a five-factor linear regression model is constructed using soil permeability (K) and five characteristics: silt content (SI), clay content (CL), soil organic matter (OM), soil bulk density (BD), and soil moisture content (MC) as variables. While computationally simple, this prediction method is less effective for non-numerical categorical features such as soil type. Furthermore, while numerous factors actually influence soil permeability, the multiple linear regression model considers fewer sample features. Furthermore, some features have a strong hierarchical relationship with the dependent variable, making them unsuitable for linear regression models and making prediction accuracy difficult to guarantee.

[0004] As environmental problems become increasingly severe, it is necessary to propose a new soil permeability prediction method that can effectively process category characteristics, consider factors more comprehensively, and make more accurate predictions. Summary of the Invention

[0005] The purpose of the present invention is to solve the problems of existing prediction methods such as difficulty in processing non-numerical category features, small number of sample features considered, and low prediction accuracy of existing machine learning models, and to propose a soil permeability prediction method that can process non-numerical category features, consider factors more comprehensively, and make predictions more accurately.

[0006] To achieve the above object, the present invention adopts the following technical solutions:

[0007] A method for predicting soil permeability based on PCA and CatBoost regression fusion, the specific steps are as follows:

[0008] Step 1: Data collection: Collect several soil samples with permeability values, use the numerical characteristic data of these samples as a sample set, and extract the characteristic data of the soil whose permeability is to be predicted; the sample set and the characteristic data of the soil whose permeability is to be predicted include: clay content (Clay), silt content (Silt), sand content (Sand), average diameter of soil particles (dg), standard deviation of soil particle diameter (sg), soil organic carbon content (OC), soil bulk density (Db), soil particle density (Dp), saturated soil volumetric water content (WC_s), unsaturated soil volumetric water content (WC_i), wet aggregate stability (WAS), soil electrical conductivity (EC) and soil type (Texture Class).

[0009] Step 2: Data cleaning: Fill in missing data in the sample set described in step 1, remove abnormal data, and perform normalization operations;

[0010] Step 3 PCA principal component analysis: Based on the idea of PCA principal component analysis, the high-dimensional data set of the sample is reduced in dimension, important features are retained, and irrelevant and redundant features are removed;

[0011] Step 4: Construct a CatBoost regression model. Add the sample soil's category features and soil permeability values to the new sample feature dataset obtained in step 3 for model training. During the training process, the K-fold cross-validation method is used to divide the sample new feature dataset with the category features and soil permeability values into K subsets. Each subset data is used as a validation set, and the remaining K-1 groups of subset data are used as training sets. K rounds of training are performed to obtain a trained CatBoost regression model.

[0012] Step 5: Input the characteristic data of the soil whose permeability is to be predicted into the trained CatBoost regression model to obtain the predicted permeability value of the sample soil.

[0013] Furthermore, the specific steps of step 2 include:

[0014] Step 2.1 performs outlier detection on the dataset; the method is as follows: standardize the soil characteristic dataset obtained in step 1, and then perform the KS test. For features that conform to the normal distribution, detect outliers according to the 3σ principle and clear outliers; for features that are not normally distributed, use the quartile method to detect outliers and clear outliers.

[0015] Step 2.2 fills the missing values for the cleared values; the method is as follows: use the mean interpolation method to process the missing values of the feature data of the data set in step 2.1, and fill the values cleared in step 2.1; if the attribute is measured using a numerical level, the mode of the valid values of the attribute is used to interpolate the missing values; if the attribute is measured using a constant, the average of the valid values of the attribute is used to interpolate the missing values.

[0016] Step 2.3 normalizes the padded dataset to obtain the cleaned dataset.

[0017] Furthermore, step 3 is based on the idea of PCA principal component analysis method to reduce the dimensionality of the high-dimensional data set, retaining important features and removing irrelevant and redundant features to improve the model training speed. The specific steps are as follows:

[0018] Step 3.1: Centralize the sample set data;

[0019] Step 3.2 calculates the covariance matrix of the sample soil characteristic data and finds the unit vector ω that maximizes the variance of the sample soil characteristic data after mapping;

[0020] Step 3.3 projects the original features of the sample soil according to the selected feature vector to obtain the k-dimensional new features of the sample soil after dimensionality reduction.

[0021] Specifically, the method to obtain the unit vector ω with the largest variance in step 3.2 is:

[0022] Calculate the eigenvalues and corresponding eigenvectors of the covariance matrix of the sample soil characteristic data, sort the eigenvalues from large to small, select the top k features according to the sorting order and contribution, and extract their corresponding k eigenvectors.

[0023] Specifically, the method for processing the category features in step 4 is: first, randomly permutate the data set, and then estimate the expected value of the target variable for each category. The formula used is:

[0024]

[0025] in Is the feature vector of the kth sample in the sample set, where the i-th dimension feature, i.e. is the categorical feature that needs to be converted; j is the target value corresponding to the training sample, that is, the soil permeability value; for the training sample x k , D k Indicates the sub-dataset that comes before this sample in the CatBoost random permutation; yes The expected value of the target variable obtained after conversion, that is, the numerical feature converted from the categorical feature; Meaning when When , I is 1, and when it is not equal to , it is 0; P is the added prior value, which is set to the average load value in the sample, and α is a weight coefficient greater than 0.

[0026] The CatBoost algorithm is an improved GBDT method based on symmetric decision trees. It has fewer model parameters and is one of the boosting algorithms that supports categorical variables and high accuracy. The reason why the present invention chooses the CatBoost algorithm is because of its many advantages in regression prediction:

[0027] First, CatBoost has excellent performance, with advantages such as high accuracy, short training time, and high robustness. It has fewer hyperparameters, which facilitates parameter tuning and reduces the possibility of overfitting.

[0028] Secondly, CatBoost is highly practical and scalable, and supports categorical processing. It is also applicable when sample features are categorical rather than numerical, and can also handle cases where the soil sample contains information such as soil category.

[0029] CatBoost improves the traditional GBDT model by converting categorical features that cannot be processed by traditional GBDT into numerical features. CatBoost uses target statistics (TS) to process categorical features.

[0030] When predicting soil permeability, different soil types affect the permeability value. Sample soil feature data often includes categorical features such as soil type, which are difficult for traditional models to handle. However, the CatBoost regression model can effectively handle this.

[0031] CatBoost is an emerging decision tree boosting algorithm that incorporates categorical feature processing methods and feature combination processing modules. The classifiers and regressors generated by it have achieved excellent prediction accuracy in areas such as power forecasting and short-term load forecasting. In the soil permeability prediction problem, there are categorical features such as soil type that have great mining value. Traditional models do not accept text category data such as soil type as input, and the CatBoost model provides advantages for processing these categorical features that traditional prediction methods do not have. PCA principal component analysis is a commonly used data dimensionality reduction method that is suitable for reducing the number of features and extracting the main factors affecting the target. The use of PCA allows the model to consider more factors affecting soil permeability while ensuring training results.

[0032] Combining PCA and CatBoost can improve the processing effect of categorical features in sample soil, and can also adapt to the high dimension of sample soil features, thereby improving prediction accuracy and solving the defects of existing technologies.

[0033] The beneficial effects of the present invention are:

[0034] The present invention provides a method for predicting soil permeability that takes into account more comprehensive factors, makes predictions more accurate, and has better processing effects on categorical features such as soil type. The present invention uses PCA to reduce the dimensionality of a large number of factors affecting permeability, so that the model can consider more features; and uses CatBoost regression to use the statistical value of the target variable to improve the processing ability of categorical features. The present invention combines PCA with CatBoost, which can improve the processing effect of categorical features in sample soils, and can also adapt to the high dimensionality of sample soil features, thereby improving prediction accuracy, solving the defects of the existing technology that the accuracy is not high, and the sample features can only contain numerical data, ignoring the effect of soil type on permeability. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 This is a flow chart of the method according to an embodiment of the present invention;

[0036] Figure 2 It is a box plot of soil data under the quartile method;

[0037] Figure 3 Sort the eigenvalues between the covariance matrices of soil permeability samples;

[0038] Figure 4 It is a three-dimensional mapping diagram of the relationship between the first three components of soil permeability after dimensionality reduction and the original 12 features;

[0039] Figure 5 This is the prediction accuracy effect diagram of PCA-CatBoost. DETAILED DESCRIPTION

[0040] This paper combines principal component analysis (PCA) with a CatBoost regression model to form a PCA-CatBoost model for predicting soil permeability. PCA reduces the dimensionality of numerous factors influencing permeability, allowing the model to consider more features. CatBoost regression uses the target variable's statistical value to improve the ability to process categorical features, thereby increasing prediction accuracy.

[0041] like Figure 1 , prediction methods include:

[0042] Step 1. Data collection. Samples with soil permeability (Ksat) values were selected from the SWIG dataset. Sample features included clay content (Clay), silt content (Silt), sand content (Sand), average soil particle diameter (dg), standard deviation of soil particle diameter (sg), soil organic carbon content (OC), soil bulk density (Db), soil particle density (Dp), saturated soil volumetric water content (WC_s), unsaturated soil volumetric water content (WC_i), wet aggregate stability (WAS), soil electrical conductivity (EC), and soil type (Texture Class), totaling 135 samples. Soil type is a categorical feature, and the four possible categories are LOAM (loam), SANDY LOAM (sandy loam), CLAY LOAM (clay loam), and SANDY CLAY LOAM (sandy clay loam). Generate a sample set (F, y) = [f1, f2, f3, ..., f 13 ,y]. F=[f1,f2,f3,……,f 13 ] is the feature vector set, and y is the target vector. Some sample data are shown in Table 1. The last column Ksat is the permeability:

[0043] Table 1

[0044] feature Clay Silt Sand dg Sg OC Db Dp WC_s WC_i WAS EC Ksat Sample 25 6.678 24.897 68.425 0.258 8.685 0.643 1.284 2.525 0.461 0.121 46.154 0.80 9.645 Sample 26 9.027 21.587 69.386 0.248 9.864 0.293 1.310 2.525 0.483 0.120 47.191 0.60 8.457 Sample 27 7.211 23.275 69.514 0.264 8.885 0.368 1.272 2.551 0.486 0.121 46.067 0.80 10.468 Sample 28 16.485 25.376 58.139 0.129 14.206 0.488 1.369 2.538 0.416 0.119 60.123 0.50 7.349

[0045] Step 2. Data cleaning

[0046] Step 2.1 performs outlier detection on the dataset;

[0047] The numerical characteristics of the obtained soil dataset samples were standardized and then KS test was performed. The results of KS test are shown in Table 2:

[0048] Table 2

[0049] feature statistic pvalue feature statistic pvalue Clay 0.052797055 0.826547917 Db 0.082766812 0.296808607 Silt 0.091037770 0.200532110 Dp 0.207364121 0.000014364 Sand 0.120772784 0.035712986 WC_s 0.051048464 0.855267192 dg 0.130284889 0.018556214 WC_i 0.066368595 0.568738963 Sg 0.070620836 0.489199749 WAS 0.106495825 0.086901801 OC 0.175410939 0.000419563 EC 0.128904386 0.020469105

[0050] Features with a pvalue greater than 0.05 satisfy a normal distribution. These include Clay, Silt, Sg, Db, WC_s, WC_i, and WAS. Features that do not satisfy a normal distribution include Sand, dg, OC, Dp, and EC. For features that test for a normal distribution, outliers are detected using the 3σ principle: let the standard deviation of the sample set for feature j be σ and the mean be μ. Since feature j follows a normal distribution, the probability that the value of feature j falls within the range (μ - 3σ, μ + 3σ) is 0.9974. Values outside the range (μ - 3σ, μ + 3σ) are cleared. After verification, all features that meet the 3σ principle and do not contain outliers, so no outliers are required.

[0051] The quartile method is used to detect outliers for non-normal distribution characteristics. The box plot of soil data under the quartile method is as follows: Figure 2 Clear the abnormal values: let the upper quartile of the sample set on feature j be Q1, and the lower quartile be Q2, then the maximum and minimum value boundaries are as follows:

[0052] Max=Q1+k(Q1-Q2)

[0053] Min=Q2-k(Q1-Q2)

[0054] Here, k can be 1.5 or 3, and in this embodiment, 1.5 is used. Based on this, the upper and lower bounds of feature j are obtained, and the values outside the bounds are cleared. The partial results after clearing are shown in Table 3:

[0055] Table 3

[0056] feature Sand dg OC Dp EC Sample 25 1.291 NAN -0.529 0.413 0.39 Sample 26 1.392 1.781 -1.359 0.413 -0.39 Sample 27 1.405 NAN -1.181 0.659 0.39 Sample 28 0.218 -0.044 -0.897 0.536 -0.77

[0057] Among them, NAN is the value that is cleared under the quartile method outlier detection, indicating that the dg feature values of samples 25 and 27 are abnormal and need to be cleared.

[0058] It can be obtained that the overall data after some samples are cleared is shown in Table 4:

[0059] Table 4

[0060] feature Clay Silt Sand dg Sg OC Db Dp WC_s WC_i WAS EC Sample 25 -1.910 -0.294 1.291 NAN -2.052 -0.529 -1.308 0.413 -1.230 -0.450 -0.998 0.39 Sample 26 -1.473 -0.754 1.392 1.781 -1.584 -1.359 -0.817 0.413 -0.613 -0.587 -0.941 -0.39 Sample 27 -1.811 -0.519 1.405 NAN -1.973 -1.181 -1.535 0.659 -0.529 -0.450 -1.003 0.39 Sample 28 -0.085 -0.227 0.218 -0.044 0.139 -0.897 0.298 0.536 -2.490 -0.723 -0.235 -0.77

[0061] Step 2.2: Fill in missing values for the cleared values;

[0062] Then, the cleared values are interpolated by the mean, and the missing values are interpolated with the mean value of the valid value of the attribute, as shown in Table 5:

[0063] Table 5

[0064] feature Clay Silt Sand dg Sg OC Db Dp WC_s WC_i WAS EC Sample 25 -1.910 -0.294 1.291 -0.313 -2.052 -0.529 -1.308 0.413 -1.230 -0.450 -0.998 0.39 Sample 26 -1.473 -0.754 1.392 1.781 -1.584 -1.359 -0.817 0.413 -0.613 -0.587 -0.941 -0.39 Sample 27 -1.811 -0.519 1.405 -0.313 -1.973 -1.181 -1.535 0.659 -0.529 -0.450 -1.003 0.39 Sample 28 -0.085 -0.227 0.218 -0.044 0.139 -0.897 0.298 0.536 -2.490 -0.723 -0.235 -0.77

[0065] Step 2.3 normalizes the padded dataset to obtain the cleaned dataset.

[0066] Normalize the feature vector set data. Temporarily remove the category feature f 13 , scale the data of the remaining 12 features of each sample to the range [0,1] using the formula:

[0067]

[0068] Among them, f i (j) is the eigenvalue of the jth sample in the i-th eigenvector, f i(j)′ is the eigenvalue of the jth sample of the i-th eigenvector after normalization, min(f i ) is the minimum value of the elements in the i-th eigenvector, max(f i ) is the maximum value of the elements in the i-th eigenvector.

[0069] Step 3. PCA principal component analysis

[0070] Perform dimensionality reduction on the 12-dimensional dataset obtained in step 2, retaining important features and removing irrelevant and redundant features. The specific steps are as follows:

[0071] Step 3.1. Center the 12-dimensional sample soil data obtained in Step 2. Calculate the mean of the original data for each dimension of the sample. The new data is the difference between the original data and the mean. The mean of the new data is 0. The formula is as follows:

[0072]

[0073]

[0074] in, represents the mean value of the soil permeability data sample points on feature j, and n is the number of samples, which is 135 in this embodiment. represents the jth eigenvalue of the i-th sample, is the value of the centralized feature j of the i-th soil permeability data sample.

[0075] Step 3.2. Calculate the unit vector ω that maximizes the variance of the soil permeability sample points after mapping. Based on the vector mapping method, map the soil permeability data sample points according to the unit vector ω. The unit vector ω is required to maximize the variance of the sample after mapping. The formula is as follows:

[0076]

[0077] Wherein, Var(X) represents the variance of the soil permeability data sample after being mapped onto the unit vector ω; ω is a unit vector; n is the number of samples, which is 135 in this embodiment; and m is the number of features of the sample, which is 12 in this embodiment.

[0078] Calculate the eigenvalues λ and corresponding eigenvectors between the covariance matrix of soil permeability samples, and sort the eigenvalues λ from large to small, such as Figure 3 When 8 features are selected, the contribution reaches 98%.

[0079] In this embodiment, the first eight features are selected according to the sorting order and contribution, and the corresponding eight feature vectors are extracted to obtain a set of:

[0080] {(λ1,u1),(λ2,u2),(λ3,u3),(λ4,u4),(λ5,u5),(λ6,u6),(λ7,u7),(λ8,u8)}

[0081] Among them, λ i is the eigenvalue, u i is the feature vector. λ1~λ8 are the new 8 features obtained after dimensionality reduction, which are the 8-dimensional mapping of the original 12-dimensional features in the new space. Each new feature in the 8-dimensional new features contains the information of the original 12-dimensional features, but the weights of the original 12-dimensional features in different new features are different. The feature vector u i Defines the weight when mapping the original 12-dimensional features to the new 8-dimensional features.

[0082] Step 3.3 projects the original features onto the selected feature vector to obtain the 8-dimensional soil permeability features after dimensionality reduction, where the first three components are as follows: Figure 4 shown.

[0083] Step 4. Build CatBoost regression model

[0084] Step 4.1: Add the eliminated category features and soil permeability values to the 8-dimensional feature vector set obtained in step 3 to obtain a 10-dimensional sample set. Some of the data are shown in Table 6. 70% of the set is used as the training set and 30% as the validation set.

[0085] Table 6

[0086] sample Feature 1 Feature 2 Feature 3 Feature 4 Feature 5 Feature 6 Feature 7 Feature 8 Texture Class Ksat Sample 25 -1.884 1.101 -0.022 -0.171 -1.164 0.554 0.157 -0.004 SANDY LOAM 9.645 Sample 26 -1.022 2.428 -0.219 0.090 -0.577 -0.096 -0.023 -1.998 SANDY LOAM 8.457 Sample 27 -1.629 1.384 -0.267 0.041 -0.387 0.645 0.174 -1.391 SANDY LOAM 10.468 Sample 28 0.215 0.598 0.067 -1.043 -2.604 -0.722 0.412 -0.922 SANDY LOAM 7.349

[0087] Step 4.2: Input the training set into the CatBoost model for training. CatBoost improves the traditional GBDT model by converting categorical features that cannot be processed by traditional GBDT into numerical features. CatBoost uses target variable statistics (TS) to process categorical features. The specific approach is: first, randomly permutate the dataset, and then estimate the expected value of the target variable for each category. The formula used is:

[0088]

[0089] in is the feature vector of the kth sample in the sample set, where the i-th dimension feature, i.e. is the categorical feature that needs to be converted. j is the target value corresponding to the training sample, that is, the soil permeability value. k , D k Indicates the sub-dataset that comes before this sample in the CatBoost random permutation. yes The expected value of the target variable obtained after the conversion, that is, the numerical feature converted from the categorical feature. Meaning when When , I is 1, and when it is not equal, it is 0. To reduce the noise of low-frequency data, a prior distribution term is added, introducing two values: P and α. P is the added prior value, which is set to the average load value in the sample, and α is a weight coefficient greater than 0. In this embodiment, k = 1 and n = 95.

[0090] The model is trained using the K-fold cross-validation method. The original data is divided into K groups, each subset data is used as a validation set, and the remaining K-1 groups of subset data are used as training sets. K rounds of training are performed in this way, where K=10.

[0091] After completing the training in step 4.3, input the feature data of the validation set into the model to obtain the corresponding soil permeability prediction value. In actual use, input other soil feature values into the model to obtain the soil permeability prediction result. The results obtained by the model prediction for the above samples 16, 17, and 18 are shown in Table 7 below:

[0092] Table 7

[0093] sample Actual permeability value Permeability prediction value Sample 25 9.645 9.550 Sample 26 8.457 8.522 Sample 27 10.468 10.323 Sample 28 7.349 7.237

[0094] Step 5. Model Evaluation

[0095] After completing the model training, it is necessary to evaluate the prediction effect of the model. This embodiment mainly uses the accuracy (R 2 ), root mean square error (RMSE), mean absolute error (MAE), and mean absolute percentage error (MAPE). The formulas are as follows:

[0096]

[0097]

[0098]

[0099]

[0100] Where y is the actual value of soil permeability, is the predicted value, is the mean of the actual value, and m is the number of samples.

[0101] Figure 5 This is a graph showing the prediction accuracy of PCA-CatBoost. To better evaluate the model's performance, the model's prediction results are compared with existing methods. The comparison results are shown in Table 8.

[0102] Table 8

[0103] Prediction Model <![CDATA[R 2 ]]> RMSE MAE MAPE LR 0.7068 2.4077 1.9624 0.4920 Bayesian Ridge 0.6734 2.5410 2.1230 0.5828 PCA-CatBoost 0.7768 2.1007 1.6070 0.3991

[0104] As shown in Table 7, for samples containing categorical features, the accuracy, root mean square error, mean absolute error, and mean absolute percentage error of the PCA-CatBoost model are all superior to traditional linear regression and Bayesian Ridge regression methods. The present invention uses PCA to reduce the dimensionality of a large number of factors affecting permeability, allowing the model to consider more features; and uses CatBoost regression to use the statistical value of the target variable to improve the processing ability of categorical features. Combining PCA with CatBoost can improve the processing effect of categorical features in sample soil, and can also adapt to the high dimensionality of sample soil features, thereby improving prediction accuracy.

Claims

1. A method for predicting soil permeability based on PCA and CatBoost regression fusion, characterized in that: The specific steps are as follows: Step 1: Data collection: Collect several soil samples with permeability values, use the numerical characteristic data of these samples as a sample set, and extract the characteristic data of the soil to be predicted permeability; Step 2: Data cleaning: Fill in missing data in the sample set described in step 1, remove abnormal data, and perform normalization operations; Step 3 PCA principal component analysis: Based on the idea of PCA principal component analysis, the high-dimensional data set of the sample is reduced in dimension, important features are retained, and irrelevant and redundant features are removed; Step 4: Construct a CatBoost regression model. Add the sample soil's category features and soil permeability values to the new sample feature dataset obtained in step 3 for model training. During the training process, the K-fold cross-validation method is used to divide the sample new feature dataset with the category features and soil permeability values into K subsets. Each subset data is used as a validation set, and the remaining K-1 groups of subset data are used as training sets. K rounds of training are performed to obtain a trained CatBoost regression model. Step 5: Input the characteristic data of the soil whose permeability is to be predicted into the CatBoost regression model trained in step 4 to obtain the predicted permeability value of the soil; The characteristic data of the sample set and the soil to be predicted for permeability in step 1 include: clay content, silt content, sand content, average soil particle diameter, standard deviation of soil particle diameter, soil organic carbon content, soil bulk density, soil particle density, saturated soil volumetric water content, unsaturated soil volumetric water content, wet aggregate stability, soil electrical conductivity, and soil type; The specific steps for step 2 include: Step 2.1: Perform outlier detection on the data set; standardize the sample set obtained in step 1, and then perform the KS test. For features that meet the normal distribution, detect outliers according to the 3σ principle and clear out the outliers; for features that are not normally distributed, use the quartile method to detect outliers and clear out the outliers; Step 2.2: Fill in the missing values for the cleared values. Use the mean interpolation method to process the missing values of the feature data in the data set in step 2.1 and fill in the values cleared in step 2.

1. If the feature data is measured using numerical levels, the mode of the feature data's valid values is used to interpolate the missing values. If the feature data is measured using constants, the mean of the feature data's valid values is used to interpolate the missing values. Step 2.3 normalizes the padded dataset to obtain the cleaned dataset. The specific steps for step 3 are as follows: Step 3.1: Centralize the sample set data; Step 3.2: Calculate the covariance matrix of the sample soil characteristic data to find the unit vector ω that maximizes the variance of the sample soil characteristic data after mapping; calculate the eigenvalues and corresponding eigenvectors of the covariance matrix of the sample soil characteristic data, sort the eigenvalues from large to small, select the top k features based on the sorting order and contribution, and extract the corresponding k eigenvectors; Step 3.3 projects the original features of the sample soil according to the selected feature vector to obtain the k-dimensional new features of the sample soil after dimensionality reduction.

2. The method for predicting soil permeability based on PCA and CatBoost regression fusion according to claim 1, characterized in that: When training the model in step 4, the method for processing category features is: first randomly permutate the data set, and then estimate the expected value of the target variable for each category. The formula used is: in is the feature vector of the kth sample in the sample set, where the i-th dimension feature, i.e. is the categorical feature that needs to be converted; For the training sample x k , D k Indicates the sub-dataset that comes before this sample in the random permutation of CatBoost; y j It's D k The target value corresponding to the j-th training sample in , that is, the soil permeability value; yes The expected value of the target variable obtained after conversion, that is, the numerical feature converted from the categorical feature; Meaning when When , I is 1, and when it is not equal to , it is 0; P is the added prior value, which is set to the average load value in the sample, and α is a weight coefficient greater than 0.

3. The method for predicting soil permeability based on PCA and CatBoost regression fusion according to claim 1, characterized in that: The method to obtain the unit vector ω with the largest variance in step 3.2 is: Calculate the eigenvalues and corresponding eigenvectors of the covariance matrix of the sample soil characteristic data, sort the eigenvalues from large to small, select the top k features according to the sorting order and contribution, and extract their corresponding k eigenvectors.

Images