A method for predicting chloride ion diffusion coefficient based on CatBoost model

Abstract

The application discloses a kind of based on CatBoost regression model's chlorine ion diffusion coefficient prediction method.The method first collects measured sample data and carries out data cleaning, ensure the quality of data.Subsequently, by LASSO regression feature selection method, the feature that has significant contribution to target value is screened out, and the optimal feature coefficient is selected using cross validation.Then, define regression model evaluation function.Then, using CatBoost regression model is trained, and the hyperparameter is optimized by grid search, and the best model parameter is obtained.Prediction is carried out on training set and test set, and model evaluation index is calculated and output, finally, fitting effect diagram and residual plot are drawn, show the prediction ability and error distribution of model.By combining LASSO feature selection and CatBoost regression model, the application improves the prediction accuracy and stability of chlorine ion diffusion coefficient, and can effectively handle complex regression problems.

Description

Technical Field

[0001] This invention relates to a regression analysis method based on machine learning, particularly a model applied to predicting the chloride ion diffusion coefficient of concrete. By employing the CatBoost regression model, an efficient and accurate regression prediction method is provided. Background Technology

[0002] Concrete is a heterogeneous, multi-scale composite material composed of cement, coarse aggregate, and fine aggregate. Chloride ions are one of the main causes of steel corrosion in concrete. When chloride ions diffuse into the concrete and reach the surface of the steel reinforcement, they damage the passivation film on the reinforcement, leading to corrosion. Different concrete mix proportions and material compositions significantly affect the chloride ion diffusion behavior. Therefore, studying the chloride ion diffusion coefficient helps optimize the concrete mix proportions and material selection, thereby improving its resistance to chloride ion attack. Concrete structures with poor durability require frequent maintenance and replacement, which not only consumes a large amount of resources but also has a negative impact on the environment. Therefore, accurately predicting the chloride ion diffusion coefficient of concrete is of great significance for its durability design.

[0003] Currently, most research still relies on traditional experimental methods, such as analyzing the specific physical structure of concrete, studying concrete with different material compositions, or focusing solely on the chloride ion diffusion coefficient of concrete at a certain age. While these methods provide valuable perspectives for understanding the chloride ion diffusion coefficient, they often have certain limitations. With the rapid development of computer technology, especially in the field of artificial intelligence, machine learning-based prediction methods are gradually becoming a new research direction.

[0004] Machine learning regression models can predict the chloride ion diffusion coefficient of concrete by collecting various characteristic parameters without relying on complex experimental procedures. Different machine learning algorithms have their own advantages in predicting the chloride ion diffusion coefficient of concrete: some algorithms have high prediction accuracy, while others are more powerful in terms of functionality. However, traditional machine learning algorithms require complex preprocessing of these data before model training, such as one-hot encoding or label encoding. This not only increases the preparation work for experiments but also often leads to overfitting problems during model training, thus affecting the accuracy of prediction results and the model's generalization ability. Summary of the Invention

[0005] Objective: To address the aforementioned technical problems, this invention proposes a method for predicting the chloride ion diffusion coefficient of concrete based on a CatBoost regression model. The regression model is trained using a large amount of experimental data, ultimately achieving high-precision prediction results. Hyperparameter tuning via GridSearchCV further enhances the model's accuracy.

[0006] To achieve the above-mentioned technical objectives, the present invention adopts the following technical solution:

[0007] A method for predicting chloride ion diffusion coefficient based on a CatBoost regression model includes the following steps:

[0008] Step 1: Collect the actual test sample dataset and clean the sample data;

[0009] Step 2: Divide the cleaned sample dataset from Step 1 into a training set and a test set;

[0010] Step 3: Perform LASSO regression feature selection processing on the cleaned sample data in Step 1. Specifically, apply L1 regularization constraints to the latent feature variables in the training set, and select the regularization coefficient with the smallest mean square error through cross-validation. Select the feature subset of the feature variables whose corresponding coefficients are not 0. This feature subset serves as the input basis for subsequent CatBoost regression modeling and prediction.

[0011] Step 4: Optimize the hyperparameters of the CatBoost regression model using a grid search method to select the optimal parameter combination;

[0012] Step 5: Combine the feature subset obtained in Step 3 and the optimal parameter combination selected in Step 4 to train the training set data obtained in Step 2 and construct a chloride ion diffusion coefficient prediction model.

[0013] Step 6: Based on the feature subset determined in Step 3, define the performance evaluation function of the chloride ion diffusion coefficient prediction model, and calculate the mean square error (MSE), root mean square error (RMSE), and coefficient of determination (R²). 2 Three evaluation indicators are used to assess the fitting ability of the chloride ion diffusion coefficient prediction model obtained in step 5, so as to determine whether the chloride ion diffusion coefficient prediction model can be applied to the task of high-precision prediction of chloride ion diffusion coefficient in concrete.

[0014] Beneficial effects: This invention performs LASSO regression feature selection on cleaned sample data: the latent feature variables in the sample data are processed by LASSO regression, and after cross-validation, the feature variable represented by the non-zero feature coefficient with the smallest mean square error is selected. Features that have a significant impact on the target variable are retained, and unimportant features in concrete can be compressed or deleted to ensure the representativeness and quality of the selected features.

[0015] In an optional embodiment, in step 1, the interquartile range method is used to calculate the interquartile range of each parameter column, and values ​​that are below 1.5 times the IQR below the first quartile or above 1.5 times the IQR above the third quartile are considered outliers.

[0016] Beneficial effects: This method has the advantages of identifying extreme outliers, improving data quality and model accuracy, and exhibits stronger robustness and stability, especially when dealing with non-normally distributed data.

[0017] In an optional embodiment, in step 2, the training samples account for 70% of the total sample data, and the test samples account for 30% of the total sample data.

[0018] Beneficial effects: This split ratio achieves a good balance between training and testing, which helps improve the model's generalization ability and prediction stability.

[0019] In an optional embodiment, in step 3, the objective function for LASSO regression feature selection is set as follows:

[0020]

[0021] Among them, y i x represents the true value of the chloride ion diffusion coefficient for each row of data; i Let be a row vector with 1 row and j columns, containing j parameter values; i is the total number of data points in the dataset; j is the total number of parameters; β is a column vector with j rows and 1 column, containing the assignments to the j parameters; λ is the penalty coefficient; and ||β||1 is the L1 norm.

[0022] LASSO regression employs L1 regularization, which reduces the sparse coefficients of redundant variables to 0 by adjusting the penalty coefficient, thereby achieving feature selection.

[0023] Beneficial effects: This method introduces L1 regularization to automatically filter out redundant features, thereby improving the model's generalization ability and prediction stability, and reducing the risk of overfitting.

[0024] In an alternative embodiment, the hyperparameters optimized in step 4 include the number of iterations, the depth of the tree, and the learning rate.

[0025] In an optional embodiment, in step 6, if the following three conditions are met, the fitting effect of the chloride ion diffusion coefficient prediction model is deemed satisfactory:

[0026] Condition 1: Mean Square Error (MSE), with a value between 0 and 1.5;

[0027] Condition 2: Root mean square error (RMSE), with a value between 0 and 2.0;

[0028] Condition 3: Coefficient of determination R 2 The value ranges from 0.8 to 1.

[0029] Beneficial effects: MSE measures the average of the squared errors between predicted and actual values. A smaller value indicates higher overall prediction accuracy and greater sensitivity to larger prediction errors. RMSE reflects the standard deviation of the prediction error and is more sensitive to larger errors; a smaller value indicates a smaller overall model error. R 2 The value is used to measure how well the model fits the data. The closer the value is to 1, the better the model can explain the trend of data changes.

[0030] In an optional embodiment, after the chloride ion diffusion coefficient prediction model is predicted and evaluated in step 6, a fitting effect diagram and a residual diagram are further plotted to visually demonstrate the prediction performance of the model on the training set and the test set. The fitting effect diagram is used to observe the consistency between the predicted value and the true value, and the residual diagram is used to determine the distribution characteristics of the error and the deviation structure.

[0031] Beneficial effects: By visualizing the fitting effect and residual distribution, the accuracy and error characteristics of the model prediction can be intuitively evaluated, thereby assisting in model optimization and result interpretation. Attached Figure Description

[0032] Figure 1 This is a flowchart of the chloride ion diffusion coefficient prediction method based on the CatBoost model of the present invention.

[0033] Figure 2 A graph showing the coefficients of latent feature variables used for parameter selection in the LASSO regression feature selection method.

[0034] Figure 3 This represents the residual distribution of the CatBoost model on the training and test sets.

[0035] Figure 4 This represents the residual distribution of the random forest model on the training and test sets.

[0036] Figure 5 A visualization of regression analysis on the training and test sets of the CatBoost model;

[0037] Figure 6 A visualization of regression analysis on the training and test sets of a random forest model;

[0038] Figure 7 A visualization of regression analysis on the LightGBM model test set;

[0039] Figure 8 A visualization of the regression analysis of the BP neural network model on the test set of instances;

[0040] Figure 9 A visualization of regression analysis of a convolutional neural network (CNN) model on a test set of instances;

[0041] Figure 10 A visualization of the regression analysis of the Temporal Convolutional Network (TCN) model on the test set of instances;

[0042] Figure 11 A visualization of the regression analysis of the Least Squares Support Vector Machine (LSSVR) model on the test set of instances;

[0043] Figure 12 A visualization of the regression analysis of the Long Short-Term Memory (LSTM) network model on the test set of instances;

[0044] Figure 13 A visualization of the regression analysis of the GPR model on the test set of the instance. Detailed Implementation

[0045] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of the application will be further described in detail below with reference to the accompanying drawings. The described embodiments are only a part of the embodiments involved in this invention. All non-innovative embodiments based on these embodiments by other researchers in the art are within the protection scope of this invention. Furthermore, the step numbers in the embodiments of this invention are only set for ease of explanation and do not limit the order of the steps. The execution order of each step in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0046] The purpose of this invention is to predict the chloride ion diffusion coefficient of concrete using a machine learning method, namely the CatBoost regression model. The advantages of using the CatBoost regression model are as follows:

[0047] First, it can automatically and efficiently encode categorical variables, avoiding the hassle of manually performing One-Hot encoding or Label encoding, thereby reducing the complexity of feature engineering.

[0048] 2. CatBoost regression models can automatically handle missing values ​​in the data during model training, without requiring additional steps for missing value imputation or deletion. This makes it very practical when dealing with real-world datasets, especially when missing values ​​are present.

[0049] Meanwhile, compared to directly using some classic machine learning regression algorithms such as BP neural network, convolutional neural network CNN, temporal convolutional network TCN, Gaussian process regression GPR, least squares support vector machine LSSVR, distributed gradient boosting LightGBM based on decision tree algorithm, long short time memory network LSTM algorithm, and random forest, the CatBoost model still has high accuracy and the ability to provide distribution prediction.

[0050] In one embodiment of the present invention, a method for predicting the chloride ion diffusion coefficient based on a hybrid machine learning model, such as... Figure 1 As shown, it includes the following steps:

[0051] Step 1: Collect and summarize the measured data, and perform simple data cleaning on the raw data.

[0052] Step 2: To ensure balanced training samples and improve model performance, random sampling is used for training, with training samples accounting for 70% of the total data and test samples accounting for 30% of the total data.

[0053] Step 3: Perform LASSO regression feature selection on the pre-processed data. Put the potential feature variables into LASSO regression for processing. After cross-validation, select the coefficient corresponding to the minimum mean square error and select the feature parameters that contribute to the target value. Screen out the feature subset of the feature variables whose corresponding coefficients are not 0.

[0054] Step 4: Train the CatBoost regression model and optimize the hyperparameters (iterations, depth, and learning_rate) through grid search to select the optimal parameter combination.

[0055] Step 5: Combine the feature subset obtained in Step 3 and the optimal parameter combination selected in Step 4 to train the training sample data obtained in Step 2 and construct a chloride ion diffusion coefficient prediction model.

[0056] Step 6: Define the performance evaluation function of the chloride ion diffusion coefficient prediction model based on the CatBoost regression model, calculate and output the evaluation index of the CatBoost model on the test set and training set, and comprehensively evaluate the fitting ability of the chloride ion diffusion coefficient prediction model based on the CatBoost regression model.

[0057] Step 7: Plot the CatBoost model's fit and residuals on the training and test sets to visually demonstrate the consistency between the model's predicted and actual values ​​and the error distribution pattern. The fit plot shows the linearity of the relationship between the true and predicted values, while the residual plot is used to observe whether the error is randomly distributed, thus helping to verify the reasonableness of the model's fit and its generalization ability.

[0058] In this embodiment, a total of 625 pieces of raw data were collected in step 1. There were no missing items in the raw data and no obvious outliers were detected, so all data were retained.

[0059] Specifically, in step 2, in order to balance the samples and improve model performance, the training samples account for 70% of the total sample data and the test samples account for 30% of the total sample data.

[0060] Furthermore, in step 3, the objective function for LASSO regression feature selection is set as follows:

[0061]

[0062] Among them, y i x represents the true value of the chloride ion diffusion coefficient for each row of data. i Let i be a row vector with 1 row and j columns, containing j parameter values, where i is the total number of data points in the dataset, j is the total number of parameters, β is a column vector with 1 column and j rows, containing the assignments to the j parameters, λ is the penalty coefficient, and |β||1 is the L1 norm.

[0063] It is particularly important to note that when building a predictive model, selecting too many variables can easily lead to overfitting, meaning the model performs well on the training set but poorly on the test set. Regularization can effectively solve the problem of overfitting. Regularization is a form of regression that constrains, adjusts, or reduces coefficient estimates towards zero, thereby reducing model complexity and instability during the learning process and avoiding the danger of overfitting. Therefore, LASSO regression uses L1 regularization, compressing the sparse coefficients of redundant variables to zero by adjusting the penalty coefficient, thus achieving feature selection.

[0064] In this embodiment, each data point contains 12 parameters related to the chloride ion diffusion coefficient of concrete, which are: cement OPC, kg / m³ 2 Fly ash FA, kg / m³ 3 Blast furnace slag GGBS, kg / m³ 3 Silica fume SF, kg / m³ 3 Superplasticiser, water-reducing agent, kg / m 3 , Water Water, kg / m 3 Fine aggregate (kg / m³) 3 Coarse aggregate (kg / m³) 3 Water-cement ratio (w / b), exposure time (t / a), annual mean temperature (T / ℃), and chloride ion concentration in seawater [Cl]. — [in seawater, g / L, and chloride ion diffusion coefficient Dc, 10] -12 m 2 Therefore, each data point contains 13 data items per second, with n = 625 and j = 12. Here, n represents the number of samples and j represents the number of features.

[0065] like Figure 2The 12 latent feature variables were processed by LASSO regression. After cross-validation, the coefficients corresponding to the minimum mean squared error were selected. It was found that the coefficients of each latent feature variable were not compressed to 0, indicating that all 12 features contributed to the target value. Therefore, all features were retained.

[0066] Furthermore, step 4 utilizes a CatBoost regression model for training and optimizes the hyperparameters using GridSearchCV to select the optimal hyperparameter configuration, thereby improving the model's predictive power and accuracy. Grid search exhaustively explores all possible hyperparameter combinations and uses cross-validation to select the optimal parameters, thus determining the best number of iterations, tree depth, learning rate, and other hyperparameters.

[0067] Furthermore, in step 5, this invention quantifies the model's performance using a series of regression model performance metrics, thereby evaluating its accuracy and stability and providing data support for model optimization and selection. The fitting effect is considered satisfactory if the following three conditions are met:

[0068] Condition 1: Mean Square Error (MSE), with a value between 0 and 1.5;

[0069] Condition 2: Root mean square error (RMSE), with a value between 0 and 2.0;

[0070] Condition 3: Coefficient of determination R 2 The value ranges from 0.8 to 1;

[0071] MSE (Mean Squared Error) measures the average of the squared errors between predicted and actual values. A smaller value indicates higher overall prediction accuracy and greater sensitivity to larger prediction errors. RMSE (Relative Tolerance Error) reflects the standard deviation of the prediction error and is more sensitive to larger errors; a smaller value indicates a smaller overall model error. The coefficient of determination (R²) is also relevant. 2 The value is used to measure how well the model fits the data. The closer the value is to 1, the better the model can explain the trend of data changes.

[0072] If the above three conditions cannot be met simultaneously, the parameters of the CatBoost regression model should be adjusted to improve the accuracy and reliability of concrete durability prediction.

[0073] In step 5, the performance of the trained CatBoost regression model is evaluated on the training and test sets, and common regression evaluation metrics are calculated, such as mean squared error (MSE), root mean squared error (RMSE), and coefficient of determination (R²). 2 This step helps to quantitatively analyze model performance, compare results on different datasets, and identify potential overfitting or underfitting problems.

[0074] In step 6, the experiment uses visualization to show the model's prediction performance and error distribution, helping to more intuitively understand and evaluate the regression model's performance. Specifically, the fit plot shows the relationship between the actual and predicted values; ideally, the predicted values ​​should be close to the actual values, and the points should be concentrated on the diagonal. The residual plot, on the other hand, shows the deviation between the predicted and actual values; ideally, the residual points should be randomly distributed without a clear pattern or trend. Specifically, such as... Figure 3 As shown in the figure, this graph illustrates the residual distribution of the CatBoost regression model on the training and test sets.

[0075] like Figure 5 As shown in the figure, this graph compares the predictions of the regression model with the actual values ​​on the training and test sets. The points on both sides are closely distributed near the ideal fit line, indicating that the model has a good fit on both the training and test sets, and the predicted values ​​are close to the actual values.

[0076] Furthermore, on Figures 6 to 13 The evaluation parameters for predicting the chloride ion diffusion coefficient of concrete using the machine learning regression algorithm model and the CatBoost regression model of this invention are compared, as shown in Table 1. These parameters specifically include: mean square error (MSE), root mean square error (RMSE), and coefficient of determination (R²). 2 MSE and RMSE describe the overall prediction accuracy of the model on the dataset, and the coefficient of determination R is... 2 It describes how well the model fits the real situation.

[0077] MSE RMSE <![CDATA[R 2 ]]> GPR 0.643 0.7786 0.8321 LSSVR 1.0512 1.0253 0.7089 LSTM 1.073 1.0359 0.7028 TCN 0.7319 0.8555 0.7973 BP 0.8178 0.9043 0.7735 LIGHTGBM 0.4888 0.6991 0.8648 CNN 2.2336 1.4945 0.3815 RandomForest 1.8470 1.3590 0.8251 CatBoost 1.8834 1.3724 0.8216

[0078] Table 1

[0079] As can be seen from Table 1, although the CatBoost regression model has lower mean squared error (MSE) and root mean squared error (RMSE) than some other models, such as LIGHTGBM, the CatBoost regression model has a higher coefficient of determination (R²). 2 The relatively high value indicates that it can capture data patterns well. Meanwhile, the CatBoost regression model has strong adaptability; it is an efficient gradient boosting algorithm, particularly adept at handling categorical features, and can provide powerful performance with relatively few adjustments. Furthermore, the CatBoost regression model can suppress overfitting: it has built-in mechanisms for handling data imbalance and avoiding overfitting, and compared to other algorithms, it may exhibit better stability and generalization ability in practical applications.

[0080] Compared with existing technologies, the advantages of this invention are:

[0081] 1. CatBoost regression models have a significant advantage in handling categorical features. Unlike traditional machine learning algorithms such as XGBoost and LightGBM, which require manual one-hot encoding or label encoding, CatBoost regression models natively support categorical feature processing, using the statistically based OrderedTarget Statistics method for efficient transformation. This approach not only avoids target leakage and overfitting risks but also significantly reduces preprocessing workload, allowing the model to adapt more naturally to raw data in real-world scenarios. It is particularly suitable for complex datasets with a large number of categorical variables, greatly improving modeling efficiency and effectiveness.

[0082] 2. The CatBoost regression model is a highly optimized gradient boosting algorithm with excellent execution efficiency in its overall architecture.

[0083] It employs an improved gradient estimation method, OrderedBoosting, and a unique symmetric tree structure, achieving faster training speeds and fewer iterations while maintaining accuracy. Compared to XGBoost and LightGBM, CatBoost regression models often achieve comparable or better predictive performance without tedious parameter tuning. Its default parameters have been extensively validated experimentally, making it very user-friendly for non-professional users and truly out-of-the-box.

[0084] 3. The CatBoost regression model integrates multiple mechanisms to prevent overfitting, greatly enhancing its robustness during training. For example, its unique Symmetric Tree structure not only speeds up model inference but also naturally provides regularization, preventing overfitting on complex data. Furthermore, the CatBoost regression model's automatic hyperparameter tuning mechanism further reduces the model's dependence on the training set, improving generalization ability.

[0085] 4. Although the CatBoost regression model may show only slight differences from other algorithms on individual evaluation metrics, its overall higher coefficient of determination R0 is a significant advantage. 2 The stable error distribution and high accuracy indicate that it can more effectively capture the true patterns in the data. When faced with practical problems such as data fluctuations, uneven feature dimensions, and imbalanced sample class distributions, the CatBoost regression model demonstrates superior robustness and stability compared to other algorithms, making it particularly suitable for industrial applications where high model stability is required.

[0086] In summary, the CatBoost regression model not only boasts superior performance but also offers a machine learning solution better suited to real-world business needs through its unique architectural design and mechanism optimizations. It demonstrates irreplaceable advantages in applications handling rich categorical features, requiring rapid model deployment, and demanding high stability.

Claims

1. A method for predicting chloride ion diffusion coefficient based on a CatBoost regression model, characterized in that, Includes the following steps: Step 1: Collect the actual test sample dataset and clean the sample data; Step 2: Divide the cleaned sample dataset from Step 1 into a training set and a test set; Step 3: Perform LASSO regression feature selection processing on the cleaned sample data in Step 1. Specifically, apply L1 regularization constraints to the latent feature variables in the training set, and select the regularization coefficient with the smallest mean square error through cross-validation. Select the feature subset of the feature variables whose corresponding coefficients are not 0. This feature subset serves as the input basis for subsequent CatBoost regression modeling and prediction. Step 4: Optimize the hyperparameters of the CatBoost regression model using a grid search method to select the optimal parameter combination; Step 5: Combine the feature subset obtained in Step 3 and the optimal parameter combination selected in Step 4 to train the training set data obtained in Step 2 and construct a chloride ion diffusion coefficient prediction model. Step 6: Based on the feature subset determined in Step 3, define the performance evaluation function of the chloride ion diffusion coefficient prediction model. Calculate the mean square error (MSE), root mean square error (RMSE), and coefficient of determination (R²) to evaluate the fitting ability of the chloride ion diffusion coefficient prediction model obtained in Step 5, so as to determine whether the chloride ion diffusion coefficient prediction model can be applied to the high-precision prediction task of the chloride ion diffusion coefficient in concrete. In step 3, the objective function for LASSO regression feature selection is set as follows: ； in, This represents the true value of the chloride ion diffusion coefficient for each row of data. for OK The row vector of the column, which contains Each parameter value; This represents the total number of data points in the dataset. This represents the total number of parameters. for A column vector in row 1 and column 1, which contains the values ​​of... Assigning values ​​to various parameters; This is the penalty coefficient; for Norm; LASSO regression was used Regularization, by adjusting the penalty coefficient, compresses the sparse coefficients of redundant variables to 0, thereby achieving feature selection; In step 4, the hyperparameters include the number of iterations, the depth of the tree, and the learning rate. After completing the prediction and evaluation of the chloride ion diffusion coefficient prediction model in step 6, a fitting effect diagram and a residual diagram are further plotted to visually demonstrate the prediction performance of the model on the training set and the test set. The fitting effect diagram is used to observe the consistency between the predicted value and the true value, and the residual diagram is used to determine the distribution characteristics of the error and the deviation structure.

2. The method for predicting chloride ion diffusion coefficient based on the CatBoost regression model according to claim 1, characterized in that, In step 1, the interquartile range method is used to calculate the interquartile range of each parameter column and consider values ​​that are 1.5 times the IQR below the first quartile or 1.5 times the IQR above the third quartile as outliers.

3. The method for predicting chloride ion diffusion coefficient based on the CatBoost regression model according to claim 1, characterized in that, In step 2, training samples account for 70% of the total sample data, and test samples account for 30% of the total sample data.

4. The method for predicting chloride ion diffusion coefficient based on the CatBoost regression model according to claim 1, characterized in that, In step 6, if the following three conditions are met, the fitting effect of the chloride ion diffusion coefficient prediction model is deemed satisfactory: Condition 1: Mean Square Error (MSE), with a value between 0 and 1.5; Condition 2: Root mean square error (RMSE), with a value between 0 and 2.0; Condition 3: The coefficient of determination R², with a value between 0.8 and 1.

Images