Method for predicting heavy metal concentration in soil colloid solution based on machine learning model

Abstract

The present application belongs to the technical field of environmental data analysis, and particularly relates to a method for predicting heavy metal concentration in soil colloid solution based on a machine learning model. Colloid extraction is performed on heavy metal contaminated soil samples, and characteristic variables of the soil colloid solution are determined. The characteristic variables are standardized and coded, and the preprocessed characteristic variables are respectively input into pre-trained machine learning models for predicting the concentrations of As, Cd, Pb and Zn in the soil colloid solution to obtain predicted values corresponding to the heavy metal concentrations. The pre-trained machine learning models for As and Pb are two-stage models, the first stage being a release probability classification model and the second stage being a release amount regression model. The pre-trained machine learning models for Cd and Zn are single-stage regression models. The present application provides a method for predicting heavy metal concentration in soil colloid solution based on a machine learning model, which provides a reliable technical method for accurately predicting heavy metal release characteristics.

Description

Technical Field

[0001] This invention belongs to the field of environmental data analysis technology, specifically relating to a method for predicting the concentration of heavy metals in soil colloidal solutions based on a machine learning model. Background Technology

[0002] The release / migration of heavy metals mediated by soil colloids is influenced by a combination of factors, including colloid concentration, particle composition, surface charge characteristics, and solution chemistry. These factors exhibit complex nonlinear relationships and interactions, but traditional statistical analysis methods struggle to fully capture these intricate relationships. For example, correlation and principal component analysis are insufficient for identifying the sources of heavy metal pollution in soil. Therefore, considering the potential multi-factor influence on soil colloid-mediated heavy metal release, traditional statistical analysis methods have significant limitations.

[0003] Machine learning methods, especially ensemble learning algorithms such as random forest models, are highly efficient in processing complex data, revealing hidden nonlinear relationships, and developing predictive models, and have been widely applied in the field of environmental data analysis. Random forest is an ensemble learning method based on decision trees. By constructing multiple decision trees and combining their prediction results, it can effectively improve the stability and prediction accuracy of the model. In pollutant source tracing research, random forests have been successfully applied to analyze the sources of pollutants in different environmental media such as water, soil, and air, and effectively predict the spatial distribution and pollution risk of heavy metals. However, it is currently not possible to use machine learning methods to predict the concentration of heavy metals in soil colloidal solutions. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides a method for predicting the concentration of heavy metals in soil colloidal solutions based on a machine learning model. This invention integrates experimental parameters from soil colloid extraction and the property parameters of the colloids to construct a machine learning model for predicting the concentration of heavy metals in soil colloidal solutions, with accurate prediction results.

[0005] This invention is specifically achieved through the following technical solutions: This invention provides a method for predicting the concentration of heavy metals in soil colloidal solutions based on a machine learning model, comprising the following steps: (1) Colloidal extraction of the soil sample to be tested: The soil sample to be tested was subjected to soil colloid extraction according to the designed experimental protocol. A dispersant was added, followed by shaking dispersion, ultrasonic dispersion and centrifugation to obtain a soil colloidal solution. (2) Determining the characteristic variables of soil colloidal solutions: The colloid concentration, particle size distribution parameters, and Zeta potential in the soil solution were determined, and a set of characteristic variables was constructed in conjunction with the colloid extraction parameters involved in step (1). The set of characteristic variables includes: oscillation time, liquid-solid ratio, ultrasonic time, dispersant type, soil type, average particle size, D10, D50, D90, polydispersity index (PDI), particle concentration, Zeta potential, D90 / D10, D90 / D50, D50 / D10, absolute value of Zeta potential, product of colloid concentration and particle size, and product of potential and particle size. (3) Preprocessing of feature variables: The feature variables obtained in step (2) are subjected to Z-score standardization and one-hot encoding to make them meet the model input requirements; (4) Model prediction: The preprocessed feature variables were input into the As, Cd, Pb or Zn concentration prediction models to obtain the predicted values ​​of As, Cd, Pb or Zn concentrations in soil colloidal solutions. The As or Pb concentration prediction model is a two-stage model. The first stage is a classification model to determine whether heavy metals are released from soil colloids, and the second stage is a regression model to predict the concentration of heavy metals after release. The Cd or Zn concentration prediction model is a single-stage regression model.

[0006] Preferably, in step (4), the As or Pb concentration prediction model is a random forest classifier in the first stage and a gradient boosting regression tree (GBRT) model in the second stage; the Cd or Zn concentration prediction model is an XGBoost model.

[0007] Preferably, for Cd concentration prediction, the XGBoost model incorporates logarithmic transformation; for Zn prediction, the XGBoost model incorporates Winsorize tail reduction and logarithmic transformation.

[0008] Preferably, the As, Cd, Pb, or Zn concentration prediction model in step (4) is obtained through the following method: Considering the effects of oscillation time, liquid-to-solid ratio, dispersant type, and ultrasonic time on soil colloid extraction efficiency, design an orthogonal experimental scheme or a single-factor experimental scheme. Soil samples contaminated with heavy metals from different regions were collected. The soil samples were mixed with a dispersant according to an orthogonal experimental design or a single-factor experimental design. The mixture was then shaken, sonicated and centrifuged. The supernatant was collected to obtain a soil colloidal solution. The concentration of colloids, particle size distribution parameters, and Zeta potential in the soil solution were measured. Combined with the soil colloid extraction parameters, a set of characteristic variables was constructed. This set of characteristic variables included: oscillation time, liquid-to-solid ratio, ultrasonic time, dispersant type, soil type, average particle size, D10, D50, D90, polydispersity index (PDI), particle concentration, Zeta potential, D90 / D10, D90 / D50, D50 / D10, absolute value of Zeta potential, the product of colloid concentration and particle size, and the product of potential and particle size. The characteristic variables were then standardized and coded. The concentrations of As, Cd, Pb, and Zn in the soil colloidal solution after filtration and acidification were measured, and the results, along with the processed feature variables, constituted a training dataset. The modeling strategy is selected based on the data distribution characteristics of heavy metal elements: a two-stage classification-regression framework is used for the concentration prediction of As and Pb, and a single-stage regression framework is used for the concentration prediction of Cd and Zn. The model is trained and optimized using the training dataset to obtain a prediction model for As, Cd, Pb, or Zn concentrations.

[0009] Building a dataset involves the following steps: S1. Experimental Procedure for Soil Colloid Extraction Considering the effects of oscillation time, liquid-to-solid ratio, dispersant type, and ultrasonic time on the extraction efficiency of soil colloids, design an orthogonal experimental scheme or a single-factor experimental scheme for soil colloid extraction using Design Expert 13.0 software.

[0010] Soil colloid extraction process: Weigh approximately 5.00 g of soil sample that has passed through a 60-mesh sieve and place it in a 500 ml plastic bottle. According to the designed experimental protocol, add a certain volume of deionized water or dispersant solution to the plastic bottle according to the set solid-liquid ratio, and mix thoroughly. Seal the plastic bottle with sealing film to prevent leakage or spraying. Place the plastic bottle in a fully automatic rotary shaker, set the rotation speed to 65 rpm / min, and shake for the set shaking time. After shaking, use an ultrasonic cell disruptor to ultrasonically disperse the sample for a certain time (450 W, 40 kHz), keeping the temperature below 25 ℃ during the ultrasonic process. Finally, centrifuge at 6000 rpm for 13 min using a refrigerated centrifuge, and collect the supernatant as the soil colloidal solution, storing it at 4 ℃.

[0011] S2. Determine the characteristic variables of soil colloidal solutions: The concentration, particle size distribution parameters, and zeta potential of the soil colloids were determined using nanoparticle tracking analysis (NTA) and a zeta potential meter, respectively. Combined with the soil colloid extraction parameters in S1, a set of characteristic variables was constructed. The set of characteristic variables included: oscillation time, liquid-solid ratio, ultrasonic time, dispersant type, soil type, average particle size, D10, D50, D90, polydispersity index (PDI), particle concentration, zeta potential, D90 / D10, D90 / D50, D50 / D10, absolute value of zeta potential, product of colloid concentration and particle size, and product of potential and particle size.

[0012] S3. Determine the target variable for soil colloidal solution: The concentrations of As, Cd, Pb, and Zn in the filtered and acidified soil colloidal solution were determined using inductively coupled plasma mass spectrometry and flame atomic absorption spectrophotometry.

[0013] S4. Feature variable preprocessing: The feature variables obtained from S2 are standardized and encoded to meet the model input requirements.

[0014] The target variable and the processed feature variables constitute the dataset.

[0015] Preferably, the feature importance analysis method is used to output the contribution of each feature variable to the prediction of heavy metal concentration, which is used to identify the key factors affecting the release of heavy metals in the soil sample to be tested.

[0016] Preferably, the colloidal extraction parameters in step (1) are as follows: during oscillation, the rotation speed is 65 rpm / min, the oscillation time is 6-96 h, and the liquid-to-solid ratio is 5-100 g / mL; during sonication, the power is 450 W, the frequency is 40 kHz, the sonication time is 0.5-15 min, and the dispersant type is sodium phosphate, sodium carbonate, water, sodium sulfate, or sodium nitrate.

[0017] Preferably, the colloidal concentration and particle size distribution parameters in step (2) are determined by NTA, and the Zeta potential is determined by a Zeta potential meter.

[0018] This invention also provides a model for predicting heavy metal concentrations in soil colloidal solutions, used in the method comprising: Soil colloid extraction module: used to shake, ultrasonically disperse and centrifuge the mixture of soil sample and deionized water or dispersant solution to obtain soil colloidal solution; Parameter measurement module: used to measure the concentration of colloids, particle size distribution parameters and Zeta potential in the soil solution; Feature processing module: used to combine soil colloid extraction parameters with measured colloidal property parameters to construct a set of feature variables, and perform standardization and coding preprocessing; Model prediction module: It has a built-in As, Cd, Pb or Zn concentration prediction model, which is used to receive preprocessed feature variables and output heavy metal concentration prediction values; the As or Pb concentration prediction model is a two-stage classification-regression model, and the Cd or Zn concentration prediction model is a single-stage regression model. Results output module: Used to display the predicted heavy metal concentration and prediction reliability parameters.

[0019] Preferably, the prediction model for As, Cd, Pb, or Zn concentrations is obtained by machine learning training using the colloidal property parameters, colloidal extraction parameters, and measured heavy metal concentrations of the soil sample to be tested.

[0020] Preferably, the system further includes a model update module for iteratively optimizing the pre-trained machine learning model based on new measured data.

[0021] Compared with the prior art, the present invention has the following beneficial effects: This invention proposes a method for predicting heavy metal concentrations in soil colloidal solutions based on a machine learning model. The method involves: extracting colloids from the soil sample; determining the characteristic variables of the soil colloidal solution; standardizing and encoding the characteristic variables to meet model input requirements; and inputting the preprocessed characteristic variables into As, Cd, Pb, or Zn concentration prediction models to obtain predicted values ​​for the As, Cd, Pb, or Zn concentrations in the soil colloidal solution. The As or Pb concentration prediction model is a two-stage model: the first stage is a release probability classification model to determine whether heavy metals are released from the soil colloids, and the second stage is a release amount regression model to predict the concentration of heavy metals after release. The Cd or Zn concentration prediction model is a single-stage regression model. This invention integrates experimental parameters such as oscillation time, liquid-to-solid ratio, dispersant type, and ultrasonic time, as well as colloidal property parameters such as colloid concentration, particle size distribution, and Zeta potential, to construct a machine learning model for predicting heavy metal concentrations in soil colloidal solutions. The prediction results are accurate, providing a reliable technical method for accurately predicting heavy metal release characteristics. Attached Figure Description

[0022] Figure 1 Box plots showing the distribution of soil colloid concentration in different regions.

[0023] Figure 2 Box plots of colloidal zeta potentials in soils from different regions.

[0024] Figure 3 D10 box plots of soil colloids in different regions.

[0025] Figure 4 D50 box plots of soil colloids in different regions.

[0026] Figure 5 D90 box plots of soil colloids in different regions.

[0027] Figure 6 These are PDI line diagrams of soil colloids in different regions.

[0028] Figure 7 This is a graph showing the fitting of actual data and predicted values ​​for the heavy metal As test set.

[0029] Figure 8 This is a graph showing the fitting of actual data and predicted values ​​for the heavy metal Cd test set.

[0030] Figure 9 This is a graph showing the fitting of actual data and predicted values ​​for the heavy metal Pb test set.

[0031] Figure 10 This is a graph showing the fitting of actual data and predicted values ​​for the heavy metal Zn test set.

[0032] Figure 11 This is an important feature that influences the release of heavy metal As based on the output of the random forest model.

[0033] Figure 12 This is an important feature that influences the release of heavy metal Cd based on the output of the random forest model.

[0034] Figure 13 This is an important feature that influences the release of heavy metal Pb based on the output of the random forest model.

[0035] Figure 14 This is an important feature that influences the release of heavy metal Zn based on the output of the random forest model.

[0036] Figure 15 This is a graph showing the fit between the actual data and predicted values ​​for the heavy metal test set. Detailed Implementation

[0037] To enable those skilled in the art to better understand and implement the technical solutions of the present invention, the present invention will be further described below with reference to specific embodiments and accompanying drawings. However, the embodiments described are not intended to limit the present invention. Unless otherwise specified, the experimental methods and detection methods described in the following embodiments are conventional methods; unless otherwise specified, the reagents and materials described are commercially available.

[0038] This invention provides a method for predicting the concentration of heavy metals in soil colloidal solutions based on a machine learning model, comprising the following steps: (1) Colloid extraction of soil samples to be tested: The soil samples to be tested are subjected to colloid extraction according to the designed soil colloid extraction experimental scheme. Dispersant is added, followed by shaking dispersion, ultrasonic dispersion and centrifugation to obtain soil colloid solution; (2) Determine the characteristic variables of the soil colloidal solution: Determine the concentration of colloids, particle size distribution parameters and Zeta potential in the soil solution, and construct a set of characteristic variables in combination with the soil colloid extraction parameters involved in step (1). The set of characteristic variables includes: oscillation time, liquid-solid ratio, ultrasonic time, dispersant type, soil type, average particle size, D10, D50, D90, polydispersity index (PDI), particle concentration, Zeta potential, D90 / D10, D90 / D50, D50 / D10, absolute value of Zeta potential, product of colloid concentration and particle size and product of potential and particle size. (3) Feature variable preprocessing: Perform Z-score standardization and one-hot encoding on the feature variables obtained in step (2); (4) Model prediction: Input the feature variables processed in step (3) into the As, Cd, Pb or Zn concentration prediction model respectively to obtain the predicted values ​​of As, Cd, Pb or Zn concentration in soil colloidal solution; among them, the As or Pb concentration prediction model is a two-stage model, the first stage is a classification model, used to determine whether heavy metals are released from soil colloids, and the second stage is a regression model, used to predict the concentration of heavy metals after release; the Cd or Zn concentration prediction model is a single-stage regression model.

[0039] In the As or Pb concentration prediction models, the first-stage classification model is a random forest classifier, and the second-stage regression model is a gradient boosting regression tree (GBRT) model; the Cd or Zn concentration prediction models are XGBoost models. For Cd concentration prediction, the XGBoost model incorporates logarithmic transformation; for Zn concentration prediction, the XGBoost model incorporates Winsorize tail reduction and logarithmic transformation.

[0040] This invention also provides a model for predicting heavy metal concentrations in soil colloidal solutions, used in the above method, comprising: Soil colloid extraction module: used to oscillate, ultrasonically disperse and centrifuge the mixture of soil sample and dispersant to obtain soil colloidal solution; Parameter measurement module: used to measure the concentration of colloids, particle size distribution parameters and Zeta potential in the soil solution; Feature processing module: used to combine colloid extraction parameters with measured colloid property parameters to construct a set of feature variables, and perform standardization and coding preprocessing; Model prediction module: It has a built-in As, Cd, Pb or Zn concentration prediction model, which is used to receive preprocessed feature variables and output heavy metal concentration prediction values; the As or Pb concentration prediction model is a two-stage classification-regression model, and the Cd or Zn concentration prediction model is a single-stage regression model. Results output module: Used to display the predicted heavy metal concentration and prediction reliability parameters.

[0041] The As, Cd, Pb, or Zn concentration prediction models are obtained through machine learning training on colloidal property parameters, colloidal extraction parameters, and measured heavy metal concentrations of heavy metal-contaminated soil samples. The system also includes a model update module for iteratively optimizing the pre-trained machine learning model based on new measured data.

[0042] The specific operating procedures are provided below: (1) Soil colloid extraction test protocol The effects of four factors—oscillation time, liquid-to-solid ratio, dispersant type, and ultrasonic time—on the extraction efficiency of soil colloids were investigated. Based on the factor level design table (Table 1), an orthogonal experimental design for soil colloid extraction was designed using Design Expert 13.0 software. Each experiment was repeated three times. Thirty groups of soil colloid extraction experiments were conducted using heavy metal-contaminated soil samples from different regions, including Baiyin in Gansu, Bijie in Guizhou, Shenyang in Liaoning, and Zhuzhou in Hunan.

[0043] Table 1 Factor Level Design Table (2) Soil colloid extraction process 1) Weighing: Weigh approximately 5.00 g of soil sample that has passed through a 60-mesh sieve and place it into a 500 ml plastic bottle.

[0044] 2) Shaking: According to the orthogonal experimental design for soil colloid extraction, add a certain volume of deionized water or dispersant solution to the plastic bottle according to the set solid-liquid ratio, mix thoroughly, and then seal the plastic bottle with sealing film to prevent solution leakage or spraying. Place the plastic bottle in a fully automatic rotary shaker, set the rotation speed to 65 rpm / min, and shake for the set shaking time.

[0045] 3) Ultrasonic dispersion: After oscillation, immediately transfer the plastic bottle to an ultrasonic cell disruptor for ultrasonic dispersion. Set the ultrasonic power to 450W and the frequency to 40kHz, and sonicate for the set time. To ensure constant temperature and avoid localized high temperatures caused by cavitation that could damage the colloidal properties, place the plastic bottle in a constant-temperature ultrasonic environment to keep the sample temperature below 25℃.

[0046] 4) Centrifugation. After ultrasonic dispersion, the suspension was rapidly and evenly aliquoted into six pre-cooled 50mL plastic centrifuge tubes, maintaining mass equilibrium. The tubes were then centrifuged at 6000 rpm for 13 minutes in a refrigerated centrifuge to obtain the soil colloidal solution. After centrifugation, in a clean bench, the supernatant was slowly and steadily aspirated along the wall of the centrifuge tube using a pipette, ensuring that the sediment at the bottom was not disturbed. The collected supernatant is the target soil colloidal suspension, which was then stored at 4 °C.

[0047] (3) Determination of soil colloidal property parameters The concentration and particle size distribution parameters of soil colloids were determined using nanoparticle tracking analysis (NTA). The zeta potential of the soil colloidal solution was measured using a zeta potential meter.

[0048] (4) Concentration of heavy metals in soil colloidal solution The concentrations of As, Cd, Pb and Zn in soil colloidal solution were determined using inductively coupled plasma mass spectrometry (ICPMS) and flame atomic absorption spectrophotometry (AAS).

[0049] (5) Data and Models 1) Data sources and characteristic variables The data used in this invention comes from experimental research data. The original dataset contains 120 samples, corresponding to soil colloid concentration, particle size distribution parameters, zeta potential, and heavy metal concentration in lead-zinc contaminated soils from four different regions under 30 different extraction parameter conditions.

[0050] Table 2. Characteristic Variable System and Its Environmental Significance 2) Feature engineering and data preprocessing Before building the model, the raw data underwent systematic preprocessing: Feature Construction: Based on environmental science principles, several derived features were constructed to more comprehensively describe the interaction between colloids and heavy metals: Particle size ratio characteristics: D90 / D10, D90 / D50, D50 / D10 Stability index: Absolute value of Zeta potential Interactive characteristics: the product of colloidal concentration × particle size and potential × particle size. Data standardization: Z-score standardization is applied to continuous numerical features to eliminate the influence of dimensions. .

[0051] Categorical variable coding: Unithermal coding of dispersant type and soil type is performed and converted into numerical form.

[0052] Target variable processing: Due to the wide distribution range of heavy metal concentrations, a logarithmic transformation was performed on the target variable to improve the data distribution. .

[0053] 3) Model building and evaluation framework Data partitioning strategy: Stratified sampling was used to ensure that the proportions of the four soil types remained consistent between the training and test sets. Specifically, the partitioning ratio was 80% (96 samples) for the training set and 20% (24 samples) for the test set.

[0054] Model architecture: Single-objective random forest regression models were established for As, Cd, Pb and Zn respectively.

[0055] Hyperparameter tuning: To simplify calculations and avoid multi-process compatibility issues, default parameters (n_estimators=100, other parameters remain at their default values) are used, and a fixed random seed (random_state=42) is used to ensure that the results are repeatable.

[0056] Evaluation metrics: Multiple metrics are used to comprehensively evaluate model performance, including the coefficient of determination (R²). 2 The variance / standard deviation of the 5-fold cross-validation results reflects the model's ability to explain variation; the root mean square error (RMSE) reflects the magnitude of the prediction error; the mean absolute error (MAE) reflects the average prediction error; and the variance / standard deviation of the 5-fold cross-validation results reflects the model's stability.

[0057] 5) Feature Importance Analysis: Feature importance is calculated using a method based on impurity reduction (Gini importance) to identify the factors with the greatest impact on heavy metal release. In decision trees and random forests, "impurity" measures the uncertainty or impurity of the data; the higher the "impurity," the more uneven the mixing.

[0058] Gini impurity formula: Gini = 1 - Σ(pi²) In the formula, pi is the proportion of samples of the i-th class. When completely pure (only one class), Gini = 0; when completely disordered (uniform distribution of classes), Gini is maximized. Therefore, the contribution of each feature in reducing data uncertainty can be determined by the following process: For each split node in each tree: calculate the Gini impurity (Ii) before splitting. before ). Calculate the weighted average Gini impurity (Ii) of the left and right child nodes after the split. after ). Calculate the reduction in impurity:

[0059] ΔGini = I before - I after .

[0060] This reduction is attributed to the features used for splitting.

[0061] result: (1) Characteristics of soil colloid properties Figure 1 This is a statistical distribution map of soil colloid concentration in different regions. From... Figure 1 As can be seen, there are significant differences in soil colloid concentration in different regions. The average colloid concentration of the four soil types is around 10. 10 The concentration of colloids was on the order of magnitude per mL, with the highest average colloid concentration in Bijie soil (3.06 × 10⁻⁶). 10 The highest concentration of [number] cells / mL was found in soil from Zhuzhou (2.33 × 10⁻⁶). 10 The average colloid concentrations in soils from Baiyin and Shenyang were relatively low (number of particles / mL). The soil colloid concentration data showed high dispersion. Analysis of the coefficient of variation (CV) showed the highest dispersion in Bijie soil colloid concentration data (CV = 153.88%), indicating a large variation in colloid concentration under different extraction parameters. Zhuzhou soil showed the second highest dispersion (CV = 141.45%), while the data from Baiyin and Shenyang soils were relatively concentrated, with CVs of 66.40% and 57.79%, respectively. Although the colloid concentration distribution varied among different soils, the median and mean were generally close, indicating a relatively symmetrical distribution of colloid concentration data. However, the maximum values ​​in Bijie and Zhuzhou soils deviated significantly from the mean, indicating the existence of high-concentration outliers, which may be related to extreme responses under specific extraction conditions.

[0062] The zeta potential is an important indicator for characterizing the stability of colloids. Figure 2 This is a statistical distribution map of the zeta potential of soil colloids in different regions. For horizontal differences, the zeta potential of all soil colloids is negative, indicating that the colloid surface carries a negative charge, which is consistent with the electrical characteristics of most soil colloids. The absolute values ​​of the zeta potential of soil colloids in Bijie and Shenyang are relatively large (mean values ​​are -27.04 mV and -28.26 mV, respectively), indicating a higher enrichment of negative charge on the surface; the absolute value of the zeta potential of soil colloids in Baiyin is the smallest (-10.48 mV), indicating a relatively low surface charge density. Regarding stability, the standard deviation of the zeta potential of soil colloids in Baiyin is the smallest (3.67 mV), indicating that its potential value is relatively stable under different extraction parameters; the standard deviation of the zeta potential of soil colloids in Bijie and Shenyang is relatively large (>10 mV), indicating that the zeta potential of soils in Bijie and Shenyang is more sensitive to changes in extraction parameters. From the perspective of the range of zeta potential values, the zeta potential distribution range of soil colloids in Bijie and Shenyang is relatively wide, with minimum values ​​reaching -48.68 mV and -49.46 mV respectively, and maximum values ​​reaching -14.01 mV and -16.06 mV respectively, indicating that the extraction parameter conditions have a more significant impact on the zeta potential of these two types of soil colloids.

[0063] Figures 3-6These represent the particle size distribution parameters (D10, D50, D90) and polydispersity index (PDI) of soil colloids in different regions. Figures 3-6 The results show that the average D50 values ​​of soil colloids from different regions range from 189.2 to 199.8 nm, with little overall difference. However, D10 and D90 exhibit significant regional differentiation. Baiyin soil colloids have the highest average D90 value (338.6 nm), significantly higher than the other three soil types (316.3 ~ 334.8 nm), while its average D10 value (113.8 nm) is similar to that of Bijie (112.3 nm) and Zhuzhou (117.0 nm). This combination of "high D90 and medium D10" indicates that Baiyin soil colloids contain a relatively large amount of coarse particles, while the fine particles are comparable to those of other soils. Therefore, it has the widest particle size distribution and the most complex particle composition. Shenyang soils exhibit the opposite characteristics, with the lowest average D90 value (333.4 nm) and the highest average D10 value (121.1 nm), indicating that the overall particle size of Shenyang soil colloids is relatively small and concentrated, with fewer coarse particles. The average PDI of soil colloids in Baiyin reached 1.228, significantly higher than that in Bijie (1.060), Shenyang (1.060), and Zhuzhou (1.128). This is consistent with the particle size distribution characteristics, further confirming that Baiyin soil colloids have the widest particle size distribution and the most heterogeneous particle composition. Bijie and Shenyang soil colloids had the lowest average PDI, both at 1.060, indicating relatively uniform particle size and lower system complexity. Zhuzhou soil colloids had an average PDI of 1.128, falling between Baiyin, Bijie, and Shenyang, exhibiting moderate polydispersity.

[0064] (2) Concentration of heavy metals in soil colloidal solution Table 3 shows the statistical analysis results of heavy metal concentrations in soil colloidal solutions. Overall, the average concentration of Zn in soil colloidal solutions was the highest (68.323 mg / L), followed by Cd (2.979 mg / L), while the average concentrations of As and Pb were both below 1 mg / L. This indicates that soil colloids have a much stronger ability to enrich Zn than other elements, which is consistent with the high activity and mobility of Zn in soil.

[0065] Table 3. Descriptive statistics of heavy metal concentrations in soil colloidal solutions (3) Model prediction results and analysis 1) Model prediction performance Random forest regression models were established to predict the concentrations of As, Cd, Pb, and Zn, respectively. The training / test set was divided into 96 / 24 sets using stratified sampling while maintaining the proportion of soil types. The specific model prediction performance parameters are shown in Table 4. The Zn model performed relatively best, with R0 on the test set.2 The error reached 0.5066, with an RMSE of 50.70 mg / L. Considering the extremely wide Zn concentration range (0.026–1925 mg / L), this error is within an acceptable range, indicating that the model has some predictive ability for Zn release patterns. The Cd model performed second best, with an R value of 0.5066 on the test set. 2 The R² value was 0.4318, and the RMSE was 2.00 mg / L, similar to the Zn model. The As model performed poorly, with a negative R² value (-0.0224) on the test set, indicating that its predictive ability was even lower than that of the simple mean prediction. The proportion of zero values ​​for As reached 20%, and the concentration distribution was extremely right-skewed, making it difficult for the model to learn effective patterns. The Pb model completely failed, with a test set R² value of -0.0224. 2 The value was as low as -1.2566, which may be related to the high proportion of zero values ​​(30.8%) and the extremely skewed distribution of Pb, reflecting that the threshold effect of Pb release exceeded the applicable premise of conventional regression models.

[0066] Table 4. Prediction Performance Parameters of Random Forest Model 2) Visualization of prediction results Figures 7-10 The actual data and predicted values ​​of the heavy metal test set are fitted separately (scatter plot: the horizontal axis is the actual measured value of heavy metal concentration, and the vertical axis is the predicted value of heavy metal concentration).

[0067] 3) Feature Importance Analysis Feature importance analysis revealed the relative contributions of different factors to the release of As, Cd, Pb and Zn. Figures 11-14 These are important characteristics of heavy metal release based on the output of the random forest model.

[0068] Among the key factors influencing As release, the interaction term between colloid concentration and potential is the most important, indicating that As release is significantly influenced by the synergistic effect of colloid quantity and surface charge. At high concentrations, overlap of the electrical double layer between colloids may affect the adsorption-desorption equilibrium of As. Soil type and colloid concentration also play important roles, reflecting that As release is soil-specific and closely related to the number of colloidal particles. The combined contribution of zeta potential-related characteristics (zeta potential, absolute value of zeta potential, and the interaction term between potential and particle size) is approximately 0.195, indicating that the surface charge of colloids has a significant impact on As release. Particle size parameters (NTAD10, average particle size, NTAD90, NTAD50) contribute approximately 0.14, indicating that As release is related to the specific surface area of ​​the colloids and may be adsorbed onto the colloid surface in a specific adsorption form.

[0069] Among the key factors influencing Cd release, soil type is the most important, reflecting the influence of soil specificity on Cd release. Surface charge characteristics (zeta potential, absolute value of zeta potential) and the interaction term between colloid concentration and potential contribute a combined 0.58, indicating that colloidal electrochemical properties dominate Cd release; the larger the absolute value of the zeta potential, the more stable the colloid and the stronger its ability to enrich Cd. Colloid concentration contributes 0.029, and the liquid-to-solid ratio contributes 0.028, indicating that the amount of colloid and the liquid-to-solid ratio have some influence on Cd release, but far less than soil type and charge factors.

[0070] Among the key factors influencing Pb release, the absolute value of the zeta potential (zeta potential abs) is the most important, indicating that colloidal stability is the most critical factor affecting Pb release. The interaction term between colloidal concentration and potential is the second most important, reflecting the synergistic effect of concentration and potential. The dispersant type (sodium phosphate) contributes 0.072, possibly related to its influence on solution pH and ionic strength, thus affecting Pb release. The particle size parameter (NTAD90) contributes 0.053, indicating that coarse colloidal particles have a certain enrichment capacity for Pb, but overall, Pb release is more easily regulated by electrochemical properties.

[0071] Among the key factors influencing Zn release, soil type is of extremely high importance, indicating that different soil properties have a significant impact on Zn release. The combined contribution of surface charge characteristics (zeta potential and absolute zeta potential) and the concentration-potential interaction term is approximately 0.34, suggesting that colloidal stability and the electrochemical environment affect Zn release to some extent. Colloidal concentration contributes 0.026, and particle size parameters (NTAD50, NTAD90, etc.) contribute approximately 0.08, indicating that overall, Zn release is far less affected by colloidal physical properties than by chemical factors.

[0072] Comparing the key factors influencing the release of As, Cd, Pb, and Zn, the release of As and Pb is primarily dominated by colloidal electrochemical properties (zeta potential, the interaction between colloidal concentration and potential), while particle size parameters also have some influence. The release of Cd and Zn, however, is strongly influenced by soil type and is extremely sensitive to changes in the surface charge of the colloidal particles, showing significant effects from solution chemical conditions.

[0073] (4) Prediction results and analysis of the optimized model 1) Model optimization approach The prediction results of the random forest model indicate that the data distribution characteristics of different heavy metals necessitate differentiated modeling requirements. As and Pb exhibit significant zero inflation (zero value proportions of 20.0% and 30.8%, respectively), and their release process demonstrates a two-stage mechanism of activation and transport. While Cd and Zn do not have non-zero value issues, their concentration distributions are strongly right-skewed (the maximum and minimum values ​​of Zn differ by four orders of magnitude), making conventional regression models susceptible to extreme values. Therefore, a targeted optimization modeling strategy is adopted, and the results are compared with the original random forest model.

[0074] 1. Two-stage classification-regression framework for As and Pb Both As (zero value ratio of 20.0%) and Pb (zero value ratio of 30.8%) exhibit significant zero-value expansion, and their release follows a threshold effect. Only when the interaction between colloids and heavy metals reaches a certain strength will heavy metals desorb from the soil solid phase into the colloidal phase, subsequently adsorb onto the colloidal surface, and be released. This process corresponds to two consecutive physicochemical stages: an adsorption stage and a release stage. The adsorption stage corresponds to the competitive adsorption of heavy metals between soil and colloids under different solution chemical conditions in this invention. When the repulsive force of the colloidal double electron layer dominates, heavy metals desorb from the soil solid phase and adsorb onto the colloidal surface, subsequently entering the release stage. Traditional single-stage regression models struggle to describe both processes simultaneously, leading to prediction failure.

[0075] Therefore, an improved two-stage modeling approach is adopted for a single random forest. The first stage establishes a release probability classification model to predict whether heavy metals are released from soil colloids (i.e., whether the concentration exceeds a threshold). The threshold is set based on the 25th percentile of non-zero samples. A random forest classifier, combined with feature selection (SelectKBest), is used to screen key variables, achieving high accuracy in identifying probabilistic events and key factors. The second stage establishes a release amount regression model, predicting specific concentrations only for the samples (As: 96 non-zero samples; Pb: 83 non-zero samples). Gradient boosting regression trees (GBRT) are used to logarithmically transform the concentration to stabilize the variance, and mutual information is used for feature selection, ultimately providing a more accurate description of the impact of soil colloidal properties (concentration, particle size, zeta potential) on heavy metal release.

[0076] The first stage corresponds to the adsorption process of As and Pb desorbing from the soil solid phase and adsorbing onto the surface of soil colloids, controlled by factors such as colloid surface charge, solution chemistry, and competing ions. The second stage corresponds to the release process of heavy metals onto soil colloids, influenced by the specific surface area, quantity, and stability of the colloids. This two-stage framework is highly consistent with this physicochemical mechanism and is expected to significantly improve prediction performance.

[0077] 2. Cd single-stage XGBoost regression Cd has no zero value, but its concentration distribution is right-skewed (coefficient of variation 244%). XGBoost regression was used, and the target variable was transformed by the natural logarithm to reduce heteroscedasticity. XGBoost's gradient boosting mechanism effectively captures nonlinear relationships, and its built-in regularization prevents overfitting.

[0078] 3. Zn's single-stage XGBoost regression + Winsorize tail reduction Zn concentrations range extremely wide (0.026–1925 mg / L), with several extreme high values. To reduce the dominance of extreme values ​​on the model, the Zn concentrations in the training set were first Winsorized (replacing values ​​above the 99th percentile with that quantile), then logarithmically transformed, and finally XGBoost regression was used. The test set was not Winsorized, only logarithmically transformed.

[0079] 2) Optimize model performance The optimized model's prediction performance parameters are detailed in Table 5. The comparison results with the original random forest model's prediction performance are shown in Table 6. The optimized model significantly improves the local prediction performance of As, Cd, Pb, and Zn concentrations: For the two-stage model of As and Pb, it effectively solves the zero inflation problem, achieving a classification accuracy of 100% and an AUC of 1.0, indicating that the model can completely and correctly identify the probability events of As and Pb release; in the regression stage, R² exceeds 0.85, and RMSE is as low as 0.147 mg / L and 0.391 mg / L, respectively, completely reversing the complete failure of the random forest model. These results further confirm the scientific validity of modeling release separately from release amount, i.e., the adsorption and release processes are controlled by different factors. For Cd, XGBoost combined with logarithmic transformation increases R² from 0.43 to 0.97 and reduces RMSE from 2.05 mg / L to 0.58 mg / L, indicating that logarithmic transformation significantly improves the data distribution, and XGBoost's fitting ability for nonlinear relationships far surpasses that of the random forest model. For Zn, XGBoost, combined with Winsorize tail reduction and logarithmic transformation, improved R² from 0.51 to 0.97 and reduced RMSE from 60.77 mg / L to 19.76 mg / L. The Winsorize treatment eliminated the excessive influence of extreme values ​​(such as 1925 mg / L), making the model more focused on the mainstream distribution and significantly improving the stability of Zn concentration prediction.

[0080] Table 5 Optimized Model Prediction Performance Parameters Table 6 Performance Comparison between Optimized Model and Random Forest Model 3) Visualization of prediction results Figure 15This is a fitting graph of the measured and predicted values ​​of heavy metal concentrations. As data points are closely distributed around the y=x line, showing good fit in both the low concentration region (<0.2 mg / L) and the high concentration region (>1 mg / L), with no significant deviation. The predicted values ​​of Pb data points are highly consistent with the measured values, with only a few points showing slight deviations; the R² reaches 0.85, indicating that the model has high accuracy in predicting Pb concentration. The predicted Cd values ​​are linearly correlated with the measured values, with an R² as high as 0.97, indicating that the model captures almost all variations, and the residuals are evenly distributed on both sides of the zero line. Despite the extremely wide concentration range of Zn, the predicted points are still distributed along the y=x line, with no systematic deviations in either the low concentration region (<100 mg / L) or the high concentration region (>500 mg / L); the Winsorize processing effectively suppressed the influence of extreme values.

[0081] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its purpose and scope. Therefore, it is intended to include any modifications and variations that fall within the scope of the claims and their equivalents.

Claims

1. A method for predicting heavy metal concentrations in soil colloidal solutions based on a machine learning model, characterized in that, Includes the following steps: (1) Colloid extraction of soil samples to be tested: The soil samples to be tested are subjected to colloid extraction according to the designed soil colloid extraction experimental scheme. Dispersant is added, followed by shaking dispersion, ultrasonic dispersion and centrifugation to obtain soil colloid solution; (2) Determine the characteristic variables of the soil colloidal solution: Determine the concentration of colloids, particle size distribution parameters and Zeta potential in the soil solution, and construct a set of characteristic variables in combination with the soil colloid extraction parameters involved in step (1). The set of characteristic variables includes: oscillation time, liquid-solid ratio, ultrasonic time, dispersant type, soil type, average particle size, D10, D50, D90, polydispersity index (PDI), particle concentration, Zeta potential, D90 / D10, D90 / D50, D50 / D10, absolute value of Zeta potential, product of colloid concentration and particle size and product of potential and particle size. (3) Feature variable preprocessing: Perform Z-score standardization and one-hot encoding on the feature variables obtained in step (2); (4) Model prediction: Input the preprocessed feature variables from step (3) into the As, Cd, Pb or Zn concentration prediction models respectively to obtain the predicted values ​​of As, Cd, Pb or Zn concentrations in soil colloidal solution; among them, the As or Pb concentration prediction model is a two-stage model, the first stage is a classification model, used to determine whether heavy metals are released from soil colloids, and the second stage is a regression model, used to predict the concentration of heavy metals after release; the Cd or Zn concentration prediction model is a single-stage regression model.

2. The method according to claim 1, characterized in that, In step (4), in the As or Pb concentration prediction model, the classification model in the first stage is a random forest classifier, and the regression model in the second stage is a gradient boosting regression tree GBRT model. The Cd or Zn concentration prediction model is the XGBoost model.

3. The method according to claim 2, characterized in that, For predicting Cd concentration, the XGBoost model incorporates logarithmic transformation; for predicting Zn concentration, the XGBoost model incorporates Winsorize tail reduction and logarithmic transformation.

4. The method according to claim 1, characterized in that, In step (4), the prediction model for As, Cd, Pb, or Zn concentrations is obtained in the following way: Considering the effects of oscillation time, liquid-to-solid ratio, dispersant type, and ultrasonic time on soil colloid extraction, design an orthogonal experimental scheme or a single-factor experimental scheme. Soil samples contaminated with heavy metals from different regions were collected. The soil samples were mixed with a dispersant according to an orthogonal experimental design or a single-factor experimental design. The mixture was then shaken, sonicated and centrifuged. The supernatant was collected to obtain a soil colloidal solution. The concentration of colloids, particle size distribution parameters, and Zeta potential in the soil solution were determined. Combined with the soil colloid extraction parameters involved, a set of characteristic variables was constructed. The set of characteristic variables included: oscillation time, liquid-solid ratio, ultrasonic time, dispersant type, soil type, average particle size, D10, D50, D90, polydispersity index (PDI), particle concentration, Zeta potential, D90 / D10, D90 / D50, D50 / D10, absolute value of Zeta potential, product of colloid concentration and particle size, and product of potential and particle size. The concentrations of As, Cd, Pb, and Zn in the soil colloidal solution after filtration and acidification were measured, and the results, along with the processed feature variables, constituted a training dataset. The modeling strategy is selected based on the data distribution characteristics of heavy metal concentrations: a two-stage classification-regression framework is used for As or Pb concentration prediction, and a single-stage regression framework is used for Cd or Zn concentration prediction. The model is trained and optimized using the training dataset to obtain a prediction model for As, Cd, Pb, or Zn concentrations.

5. The method according to claim 4, characterized in that, The feature importance analysis method is used to output the contribution of each feature variable to the prediction of heavy metal concentration, which is used to identify the key factors affecting the release of heavy metals in the soil sample to be tested.

6. The method according to claim 1, characterized in that, The parameters for colloidal extraction in step (1) are: shaking time of 6-96 h, liquid-to-solid ratio of 5-100 g / mL, ultrasonic time of 0.5-15 min, and dispersant type of sodium phosphate, sodium carbonate, water, sodium sulfate or sodium nitrate.

7. The method according to claim 1, characterized in that, In step (2), the colloidal concentration and particle size distribution parameters were determined using NTA, and the Zeta potential was determined using a Zeta potential meter.

8. A model for predicting heavy metal concentration in soil colloidal solutions, characterized in that, Used in the method of claim 1, comprising: Soil colloid extraction module: used to oscillate, ultrasonically disperse and centrifuge the mixture of soil sample and dispersant to obtain soil colloidal solution; Parameter measurement module: used to measure the concentration of colloids, particle size distribution parameters and Zeta potential in the soil solution; Feature processing module: used to combine colloid extraction parameters with measured colloid property parameters to construct a set of feature variables, and perform standardization and coding preprocessing; Model prediction module: It has a built-in As, Cd, Pb or Zn concentration prediction model, which is used to receive preprocessed feature variables and output heavy metal concentration prediction values; the As or Pb concentration prediction model is a two-stage classification-regression model, and the Cd or Zn concentration prediction model is a single-stage regression model. Results output module: Used to display the predicted heavy metal concentration and prediction reliability parameters.

9. The heavy metal concentration prediction model in soil colloidal solution according to claim 8, characterized in that, The prediction models for As, Cd, Pb, or Zn concentrations are obtained by machine learning training on the colloidal property parameters, colloidal extraction parameters, and measured heavy metal concentrations of heavy metal contaminated soil samples.

10. The heavy metal concentration prediction model in soil colloidal solution according to claim 8, characterized in that, The system also includes a model update module, which is used to iteratively optimize the pre-trained machine learning model based on new measured data.

Images