Soil arsenic pollution risk assessment method, device and equipment based on optimization model
By employing an optimization model-based method for assessing soil arsenic pollution risk, multiple assessment models are trained by preprocessing data and dividing it into different types of training subsets based on multiple random states. By combining the main prediction and sub-prediction data, a rapid and accurate assessment of soil arsenic pollution risk is achieved, solving the problems of high operational difficulty, long time, and low safety in traditional methods.
Patent Information
- Application Number
- CN202410968121.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-18
- Publication Date
- 2024-11-19
AI Technical Summary
Existing technologies struggle to accurately assess the risk of soil arsenic pollution, especially due to data imbalances caused by the diversity of environmental factors, which affect the accuracy of model predictions.
An optimization model-based approach was adopted. Soil sample data was preprocessed and divided into multiple sets of random state training sets, namely a core training subset, a strong adsorption training subset, and a weak adsorption training subset. Multiple random seeds were used to train the main model and sub-models for arsenic pollution risk assessment. Risk assessment was then conducted by combining the main prediction data and the sub-prediction data.
It enables rapid and accurate risk assessment of soil arsenic pollution, improves the robustness and generalization ability of the model, and solves the problems of high operational difficulty, long time and low safety in traditional methods.
Smart Images

Figure CN118982226A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of machine learning model training, and in particular to a soil arsenic contamination risk assessment method, device and equipment based on an optimization model. Background Art
[0002] Arsenic pollution in soil is extremely harmful, and the factors affecting pollution characteristics are complex, which makes it difficult to implement accurate large-scale predictions. Machine learning can combine empirical values and existing models to predict arsenic pollution levels, which can greatly save costs and improve efficiency. However, the adsorption-desorption process of arsenic in soil is affected by many environmental factors, which not only determines the occurrence form of arsenic in the soil and the risk of pollution diffusion, but also restricts the accuracy of existing models in predicting arsenic pollution risks due to problems such as linear redundancy of environmental factors. Therefore, reasonable pretreatment and processing of environmental factors that affect the adsorption and desorption behavior of arsenic in soil will be conducive to accurately predicting the level of bioavailable arsenic under different soil characteristics. It will be a new strategy for accurately predicting the risk of soil arsenic pollution and a necessary prerequisite for effectively solving the spread of arsenic pollution. Summary of the invention
[0003] Based on this, it is necessary to provide a soil arsenic pollution risk assessment method, device and equipment based on an optimization model that can accurately assess the soil pollution risk in response to the above technical problems.
[0004] A soil arsenic pollution risk assessment method based on an optimization model, the method comprising:
[0005] Acquire a sample data set, wherein the sample data set includes a plurality of sample sequences, each of the sample sequences corresponding to soil characteristic indicators of different soil samples in different experimental environments and corresponding experimental environment parameters;
[0006] Preprocessing all sample sequences in the sample data to obtain a standardized sample data set;
[0007] The standardized sample data set is processed using a plurality of different random seeds to obtain a plurality of corresponding random state training sets, and the sample sequences in the standardized sample data set are divided into a core training subset, a strong adsorption training subset, and a weak adsorption training subset according to the correlation between soil physical and chemical properties and adsorption capacity;
[0008] The main model for arsenic pollution risk assessment is optimized and trained using multiple groups of random state training sets to obtain the trained main model for arsenic pollution risk assessment;
[0009] The corresponding arsenic pollution risk assessment sub-models are trained using the core training subset, the strong adsorption training subset and the weak adsorption training subset, respectively, to obtain arsenic pollution risk assessment sub-models that predict the core category, the strong adsorption category and the weak adsorption category respectively;
[0010] Acquiring a plurality of soil characteristic indicators, wherein the plurality of soil characteristic indicators are extracted from a soil sample to be risk assessed;
[0011] A plurality of soil characteristic indicators are respectively input into the main model for arsenic pollution risk assessment and the sub-model for arsenic pollution risk assessment corresponding to the soil sample category for prediction, to obtain corresponding main prediction data and sub-prediction data, and the soil sample is subjected to risk assessment in combination with the main prediction data and the sub-prediction data.
[0012] In one embodiment, the soil characteristic indicators include: soil pH, organic matter content, cation exchange capacity, clay content, free iron oxide content, and total arsenic and phosphorus;
[0013] The experimental environment parameters include: soil-liquid ratio, solution pH, background ion strength, experimental temperature, experimental time, solution initial concentration and adsorption amount.
[0014] In one embodiment, the method of training the arsenic pollution risk assessment main model using multiple groups of random state training sets includes:
[0015] Using multiple groups of random state training sets to train the arsenic pollution risk assessment main model respectively, multiple trained arsenic pollution risk assessment main models are obtained;
[0016] Multiple trained arsenic pollution risk assessment models are fused in a weighted averaging manner to obtain an optimized arsenic pollution risk assessment main model.
[0017] In one embodiment, dividing the sample sequences in the standardized sample data set into a core training subset, a strong adsorption training subset and a weak adsorption training subset according to the correlation between soil physicochemical properties and adsorption capacity comprises:
[0018] The standardized sample data set is processed by using Pearson correlation analysis and PCA principal component analysis, so as to divide the sequence data in the standardized sample data set into different groups according to the correlation between soil physical and chemical properties and adsorption capacity;
[0019] Different groups were divided into core training subsets, strong adsorption training subsets and weak adsorption training subsets according to the correlation between the PCA principal component weights and the Pearson correlation coefficient of the adsorption amount.
[0020] In one embodiment, before using the arsenic pollution risk assessment main model and the arsenic pollution risk assessment sub-model to make predictions based on multiple soil characteristic indicators:
[0021] After analyzing the plurality of soil characteristic indicators by using principal component analysis, the soil samples are classified to obtain the soil sample categories, wherein the soil sample categories include core categories, strong adsorption categories, and weak adsorption categories;
[0022] Predicting the plurality of soil characteristic indicators using an arsenic pollution risk assessment sub-model corresponding to the soil sample category;
[0023] The arsenic pollution risk assessment submodel includes a core arsenic pollution risk assessment submodel, a strong adsorption arsenic pollution risk assessment submodel and a weak adsorption arsenic pollution risk assessment submodel for predicting the core category, the strong adsorption category and the weak adsorption category respectively.
[0024] In one embodiment, the risk assessment of the soil sample by combining the main prediction data and the sub-prediction data includes:
[0025] Extracting corresponding sub-prediction data weights according to the soil sample category;
[0026] The comprehensive prediction result is calculated based on the main prediction data, sub-prediction data and sub-prediction data weights.
[0027] A risk assessment is performed on the soil sample according to the comprehensive prediction result.
[0028] In one embodiment, when using the arsenic pollution risk assessment main model and the arsenic pollution risk assessment sub-model to predict soil samples based on multiple soil characteristic indicators, parameters related to the preset experimental environment are also input into the arsenic pollution risk assessment main model and the arsenic pollution risk assessment sub-model to obtain the prediction results under the corresponding preset experimental environment.
[0029] The present application also provides a soil arsenic pollution risk assessment device based on an optimization model, the device comprising:
[0030] A sample data acquisition module is used to acquire a sample data set, wherein the sample data set includes a plurality of sample sequences, each of which corresponds to soil characteristic indicators of different soil samples in different experimental environments and corresponding experimental environment parameters;
[0031] A sample standardization module, used to preprocess all sample sequences in the sample data to obtain a standardized sample data set;
[0032] A training set acquisition module is used to process the standardized sample data set using multiple different random seeds to obtain corresponding multiple groups of random state training sets, and further divide the sample sequences in the standardized sample data set into a core training subset, a strong adsorption training subset and a weak adsorption training subset according to the correlation between soil physical and chemical properties and adsorption capacity;
[0033] A main model training module is used to optimize and train the arsenic pollution risk assessment main model using multiple groups of random state training sets to obtain a trained arsenic pollution risk assessment main model;
[0034] A sub-model training module is used to train the corresponding arsenic pollution risk assessment sub-models using the core training subset, the strong adsorption training subset and the weak adsorption training subset, respectively, to obtain arsenic pollution risk assessment sub-models that predict the core category, the strong adsorption category and the weak adsorption category respectively;
[0035] A sample acquisition module to be evaluated is used to obtain a plurality of soil characteristic indicators, wherein the plurality of soil characteristic indicators are extracted from a soil sample to be risk evaluated;
[0036] The risk assessment module inputs multiple soil characteristic indicators into the arsenic pollution risk assessment main model and the arsenic pollution risk assessment sub-model corresponding to the soil sample category for prediction, obtains corresponding main prediction data and sub-prediction data, and conducts risk assessment on the soil sample in combination with the main prediction data and the sub-prediction data.
[0037] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:
[0038] Acquire a sample data set, wherein the sample data set includes a plurality of sample sequences, each of the sample sequences corresponding to soil characteristic indicators of different soil samples in different experimental environments and corresponding experimental environment parameters;
[0039] Preprocessing all sample sequences in the sample data to obtain a standardized sample data set;
[0040] The standardized sample data set is processed using a plurality of different random seeds to obtain a plurality of corresponding random state training sets, and the sample sequences in the standardized sample data set are divided into a core training subset, a strong adsorption training subset, and a weak adsorption training subset according to the correlation between soil physical and chemical properties and adsorption capacity;
[0041] The main model for arsenic pollution risk assessment is optimized and trained using multiple groups of random state training sets to obtain the trained main model for arsenic pollution risk assessment;
[0042] The corresponding arsenic pollution risk assessment sub-models are trained using the core training subset, the strong adsorption training subset and the weak adsorption training subset, respectively, to obtain arsenic pollution risk assessment sub-models that predict the core category, the strong adsorption category and the weak adsorption category respectively;
[0043] Acquiring a plurality of soil characteristic indicators, wherein the plurality of soil characteristic indicators are extracted from a soil sample to be risk assessed;
[0044] A plurality of soil characteristic indicators are respectively input into the main model for arsenic pollution risk assessment and the sub-model for arsenic pollution risk assessment corresponding to the soil sample category for prediction, to obtain corresponding main prediction data and sub-prediction data, and the soil sample is subjected to risk assessment in combination with the main prediction data and the sub-prediction data.
[0045] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:
[0046] Acquire a sample data set, wherein the sample data set includes a plurality of sample sequences, each of the sample sequences corresponding to soil characteristic indicators of different soil samples in different experimental environments and corresponding experimental environment parameters;
[0047] Preprocessing all sample sequences in the sample data to obtain a standardized sample data set;
[0048] The standardized sample data set is processed using a plurality of different random seeds to obtain a plurality of corresponding random state training sets, and the sample sequences in the standardized sample data set are divided into a core training subset, a strong adsorption training subset, and a weak adsorption training subset according to the correlation between soil physical and chemical properties and adsorption capacity;
[0049] The main model for arsenic pollution risk assessment is optimized and trained using multiple groups of random state training sets to obtain the trained main model for arsenic pollution risk assessment;
[0050] The corresponding arsenic pollution risk assessment sub-models are trained using the core training subset, the strong adsorption training subset and the weak adsorption training subset, respectively, to obtain arsenic pollution risk assessment sub-models that predict the core category, the strong adsorption category and the weak adsorption category respectively;
[0051] Acquiring a plurality of soil characteristic indicators, wherein the plurality of soil characteristic indicators are extracted from a soil sample to be risk assessed;
[0052] A plurality of soil characteristic indicators are respectively input into the main model for arsenic pollution risk assessment and the sub-model for arsenic pollution risk assessment corresponding to the soil sample category for prediction, to obtain corresponding main prediction data and sub-prediction data, and the soil sample is subjected to risk assessment in combination with the main prediction data and the sub-prediction data.
[0053] The above-mentioned soil arsenic pollution risk assessment method, device and equipment based on the optimization model divides all sample sequences in the standardized soil characteristic index sample data set twice to obtain multiple groups of random state training sets for training the main evaluation model and training subsets for training multiple evaluation sub-models. After the model is trained, the trained arsenic pollution risk assessment main model and the corresponding type of arsenic pollution risk assessment sub-model are used to perform targeted joint predictions, and the soil arsenic pollution risk assessment results are obtained according to the prediction results. The method can quickly obtain the ability of soil to adsorb arsenic, solve the problems of traditional adsorption experimental methods in terms of operational difficulty, time requirements and safety, and improve the accuracy and efficiency of scientific research and environmental protection work. BRIEF DESCRIPTION OF THE DRAWINGS
[0054] Figure 1 It is a schematic diagram of a process of a soil arsenic pollution risk assessment method based on an optimization model in one embodiment;
[0055] Figure 2 A schematic diagram of a PCA principal component analysis result in one embodiment;
[0056] Figure 3 is a schematic diagram of Pearson correlation coefficient analysis in one embodiment;
[0057] Figure 4 A schematic diagram of average data per group of K-Means clustering in one embodiment;
[0058] Figure 5 This is a schematic diagram of the importance ranking of factors affecting soil adsorption of As(V) in one embodiment;
[0059] Figure 6 A schematic diagram of the single factor partial dependence of soil adsorption of As(V) in one embodiment;
[0060] Figure 7 A schematic diagram of a two-factor partial dependence diagram of soil adsorption of As(V) in one embodiment;
[0061] Figure 8 The figure is a schematic diagram of the regression scattering points of the predicted value and the true value for data prediction based on the models established by different machine learning algorithms in one embodiment, wherein: Figure 8(a)(b)(c)(d) are the extreme random tree machine learning algorithm, the gradient boosting machine learning algorithm, the random forest machine learning algorithm, and the extreme gradient boosting machine learning algorithm respectively.
[0062] Fig. 9 is a structural block diagram of a soil arsenic pollution risk assessment device based on an optimization model in one embodiment;
[0063] Fig.10 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION
[0064] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0065] The risk assessment of arsenic pollution in soil is usually achieved through the soil's adsorption rate. In the existing technology, a series of experiments are required on the soil to be evaluated to finally obtain the results, and this process is relatively long and unsafe. To address this problem, a machine learning method is proposed to conduct risk assessment of soil arsenic pollution. However, due to the diversity of parameters used in soil arsenic pollution assessment, the training data is unbalanced when training the machine learning model, making it impossible to accurately evaluate the model influencing factors.
[0066] In response to the above two questions, Figure 1 As shown, a soil arsenic contamination risk assessment method based on an optimization model is provided, and the specific steps include:
[0067] Step S100, obtaining a sample data set, the sample data set including a plurality of sample sequences, each sample sequence corresponding to soil characteristic indicators of different soil samples in different experimental environments and corresponding experimental environment parameters.
[0068] Step S110, preprocessing all sample sequences in the sample data to obtain a standardized sample data set.
[0069] Step S120, using multiple different random seeds to process the standardized sample data set to obtain multiple corresponding random state training sets, and also dividing the sample sequences in the standardized sample data set into a core training subset, a strong adsorption training subset and a weak adsorption training subset according to the correlation between soil physical and chemical properties and adsorption capacity.
[0070] Step S130, optimizing and training the arsenic pollution risk assessment main model using multiple groups of random state training sets to obtain a trained arsenic pollution risk assessment main model.
[0071] Step S140, using the core training subset, the strong adsorption training subset and the weak adsorption training subset to respectively train the corresponding arsenic pollution risk assessment sub-models to obtain arsenic pollution risk assessment sub-models that predict the core category, the strong adsorption category and the weak adsorption category respectively.
[0072] Step S150, obtaining a plurality of soil characteristic indicators, wherein the plurality of soil characteristic indicators are extracted from a soil sample to be risk assessed.
[0073] In step S160, a plurality of soil characteristic indicators are respectively input into the main model for arsenic pollution risk assessment and the sub-model for arsenic pollution risk assessment corresponding to the soil sample category for prediction, and the corresponding main prediction data and sub-prediction data are obtained, and the risk assessment of the soil sample is performed in combination with the main prediction data and the sub-prediction data.
[0074] In this embodiment, first of all, considering the problems of the traditional soil adsorption method for arsenic in terms of operational difficulty, time requirements and safety, a machine model is used to predict the arsenic adsorption capacity according to soil characteristic indicators, so as to achieve a fast, efficient and safe soil arsenic pollution risk assessment. At the same time, considering the differences in the adsorption capacity of arsenic by different soils, the corresponding sub-models are trained in a targeted manner according to different types of soils, and the data predicted by the main model are used to correct the data predicted by the main model, so that a more accurate prediction result can be obtained. Further, in this method, the diversity of soil characteristic indicator parameters, that is, the large number, which leads to data imbalance when training the main model, thereby affecting the robustness and generalization ability of the main model, when training the main model, multiple random seeds are used to divide the training set into random states, train the corresponding main model, and finally fuse the main models to obtain a robust and generalized main model.
[0075] In this embodiment, the above-mentioned multiple method steps actually include two parts, steps S100 to S140, namely the model training part, and steps S150 to S160, namely the model application part.
[0076] First, when training the model:
[0077] In step S100, the physical and chemical properties of different soils and adsorption experimental data are first obtained from a large amount of literature to construct a sample data set.
[0078] In this embodiment, through prior knowledge, multiple soil characteristic indicators related to soil samples include: soil pH, organic matter content, cation exchange capacity, clay content, free iron oxide content, and total arsenic and phosphorus. Although these six indicators are conventional general indicators of arsenic adsorption rate of soil in this field, there are actually multiple conventional general indicators, and selecting these six indicators is the first time proposed in this method.
[0079] In step S110, preprocessing all sample sequences in the initial training data set includes: cleaning, interpolating, converting and standardizing each sample sequence in the initial training data set in turn.
[0080] Specifically, the soil characteristic indicators of interest in this method are soil pH, organic matter content, cation exchange capacity, clay content, free iron oxide content, and the total arsenic and phosphorus. Since different literatures do not conduct exactly the same tests on arsenic adsorption indicators, some indicators are missing in the retrieved data. In this case, the data with two or more missing indicators in the sample sequence corresponding to the same soil sample are first removed, that is, the sample sequence in the initial training data set is first cleaned.
[0081] Furthermore, in the cleaned initial training data set, for the missing sample sequences, the mean interpolation method, KNN interpolation method and conversion method were used in combination with the Chinese soil database to interpolate the missing data and obtain the interpolated initial training data set.
[0082] In one embodiment, when the missing characteristic index in the sample sequence is the free iron oxide content, the iron freeness (66%) and activation (25%) are calculated according to the T-Fe and DCB-Fe and DCB-Fe and Fe-OX in the search literature for data filling. The freeness and activation are calculated using the existing data in the data set and according to the formula: freeness = (DCB-Fe / T-Fe), activation = (OX-Fe / DCB-Fe), and the remaining missing parts are filled in again using the filled mean.
[0083] In one embodiment, when the missing characteristic indicator in the sample sequence is cation exchange capacity or clay content, the KNN interpolation method is used in combination with the Chinese soil type database, which is a national soil data set based on the second soil census, for interpolation and filling.
[0084] Finally, the Z-score standardization method is used to standardize the initial training data set after interpolation to obtain a standardized sample data set.
[0085] In step S120, the standardized sample data sets are processed differently to obtain multiple groups of random state training sets for training the arsenic pollution risk assessment main model and training subsets for training each arsenic pollution risk assessment sub-model.
[0086] In this embodiment, by using Pearson correlation analysis and PCA principal component analysis results on the standardized sample data set, and combining the correlation between soil physical and chemical properties and adsorption capacity, three sub-data sets, namely the core subset, the strong adsorption subset and the weak adsorption subset, were established, and their proportions in the data set were calculated.
[0087] Specifically, the sample sequences in the standardized sample data set are divided into a core training subset, a strong adsorption training subset and a weak adsorption training subset according to the correlation between the soil physicochemical properties and the adsorption capacity, including: firstly processing the standardized sample data set by using Pearson correlation analysis and PCA principal component analysis, so as to obtain the sequence data in the standardized sample data set divided into different groups according to the correlation between the soil physicochemical properties and the adsorption capacity, and then dividing the different groups into the core training subset, the strong adsorption training subset and the weak adsorption training subset according to the correlation between the PCA principal component weight and the Pearson correlation coefficient of the adsorption amount.
[0088] First, a Pearson correlation analysis was performed on the input features and output feature adsorption capacity in the standardized sample data set, including soil physical and chemical properties and experimental parameter conditions, to preliminarily predict which factors in the input features have a certain correlation with the adsorption capacity, and mark them as potential influencing factor features. At the same time, check whether there is a relatively obvious linear correlation between each input feature to avoid information redundancy in the process of machine learning model construction, which will affect the stability of the model. After deduplication of all soil type data in the data set, the physical and chemical properties of all soil types were obtained, and PCA principal component analysis was performed on this. The association between the two groups with the highest cumulative difference explanation rate and the pre-set potential factor characteristics was examined, and the soil samples were divided into: core subset, strong adsorption subset and weak adsorption subset, and their proportion in the data set was calculated.
[0089] Specifically, all these multidimensional data are simplified by principal component analysis (PCA) to construct an unsupervised learning pre-classification model, and the results are represented on a two-dimensional plane. The soil physical and chemical factor characteristics are marked on the graph in the form of vectors, such as Figure 2 Then the K-means algorithm is used to cluster them. The potential influencing factors are found and marked through Pearson correlation analysis, such as Figure 3As shown, the vector of potential influencing factors is defined as the adsorption correlation direction. Under K-Means clustering, each group can be preliminarily pre-classified into a core subset, a strong adsorption subset, and a weak adsorption subset. Among them, the three clusters at the intersection of the PC1 and PC2 axes are the core subsets, which mainly reflect the subtle differences in the effects of different physical and chemical factors on the adsorption performance in most samples. The strong adsorption subset is the union of the sample clusters in the positive direction of the PC1 axis and the core subset. After the sample dimensionality reduction, the important features of the principal component have a high positive correlation with the adsorption performance. The weak adsorption subset is the union of the sample clusters in the positive direction of the PC2 axis and the core subset. After the sample dimensionality reduction, the important features of the principal component have a weak correlation with the adsorption performance. The average data of each subset are shown in Figure 4 As shown, at the same time, three sub-datasets, namely the core subset, the strong adsorption subset and the weak adsorption subset, are established, and their proportions in the data set are calculated.
[0090] Specifically, the Pearson correlation coefficient (PCC) is selected to preliminarily measure the correlation between input and output features, such as Figure 3 As shown in the figure, the input features include six soil characteristic indicators and six experimental parameters, soil pH (hereinafter referred to as pH (Soil), organic matter content (hereinafter referred to as OM), cation exchange capacity (hereinafter referred to as CEC), clay content (hereinafter referred to as Clay), free iron oxide content (hereinafter referred to as DCB-Fe), total arsenic and phosphorus (hereinafter referred to as T-As+TP) and adsorption experimental data, that is, the experimental environment parameters include: soil-liquid ratio, solution pH, background ion strength, experimental temperature, experimental time, solution initial concentration and adsorption amount.
[0091] Specifically, the experimental environment parameters include: solution pH (hereinafter referred to as pH(S)), background ion concentration (Ionic Strength, hereinafter referred to as IS), soil / liquid ratio (Soil / Liquid Ratio), experimental temperature (Temperature), experimental time (Time), initial solution concentration (hereinafter referred to as C0) as input features, and soil adsorption capacity (hereinafter referred to as Qe) as output features.
[0092] Furthermore, the mathematical expression of PCC is:
[0093]
[0094] In the above formula, Rxy is the PCC value between x (input variable) and y (target variable). The PCC value range is -1 to 1. Rxy = 1 means that there is a strong linear correlation between the variables, and the symbol indicates positive or negative correlation. However, Rxy = 0 means that there is no correlation between the variables.
[0095] In this embodiment, the standardized sample data set is processed by Pearson correlation analysis and PCA principal component analysis, and the sample sequences in the standardized sample data set are divided into core training subsets, strong adsorption training subsets and weak adsorption training subsets according to the correlation between soil physicochemical properties and adsorption capacity, including: the standardized sample data set is processed by Pearson correlation analysis and PCA principal component analysis, and the sequence data in the standardized sample data set is divided into different groups according to the correlation between soil physicochemical properties and adsorption capacity, and different groups are divided into core training subsets, strong adsorption training subsets and weak adsorption training subsets according to the correlation between the principal component weights of PCA and the Pearson correlation coefficient of the adsorption amount.
[0096] As shown in Table 1, the groups in each sub-dataset and the proportion of its component groups in the total are given.
[0097] Table 1
[0098]
[0099] Next, in step S140, the learning model is trained using the divided core training subset, strong adsorption training subset, and weak adsorption training subset, respectively, to obtain the corresponding arsenic pollution risk assessment sub-model for predicting a certain category of soil, namely, the core arsenic pollution risk assessment sub-model, the strong adsorption arsenic pollution risk assessment sub-model, and the weak adsorption arsenic pollution risk assessment sub-model.
[0100] Furthermore, in step S120, when the main model for arsenic pollution risk assessment is trained, if the entire standardized sample data set is directly used for training, a data imbalance problem will occur, resulting in an inability to accurately assess the model influencing factors.
[0101] The data imbalance problem mentioned above refers to that when the standardized sample data set is divided into training set and test set according to the ratio of 80%:20%, if some strongly adsorbed subsets and core subsets are all (or most) divided into training sets, and all (or most) of the weakly adsorbed subsets are divided into test sets, the model will score low when the test set is verified, thereby affecting the accuracy of the model establishment, and further affecting the analysis of the factors affecting the model.
[0102] In order to address the above-mentioned problem of imbalanced training data, multiple different random seeds are used to process the standardized sample data set, that is, multiple random states are used to divide the sample sequences in the standardized sample data set into training sets and validation sets. Under multiple sets of random seed conditions, the sample sequences contained in the training sets and validation sets of these multiple sets of data sets are different and completely random. In a cross-validation manner, the results of evaluating the influence of model feature factors are more objective and reliable.
[0103] Preferably, 10 different random seeds are used to process the standardized sample data set. In other embodiments, other numbers of random seeds may be used to process the standardized sample data set according to actual conditions.
[0104] Specifically, 10 different random seeds are used to process the standardized sample data set, and the sample sequences in the standardized sample data set are randomly divided into a training set and a validation set. Under the 10 sets of random seed conditions, the sample sequences contained in the training set and the validation set of these 10 sets of data sets are different and completely random. In a cross-validation manner, the results of evaluating the influence of model feature factors are more objective and reliable.
[0105] Ideally, there should be enough sample sequences as training sets and test sets for training models and evaluating model performance. That is, the data of the training set and the test set are large and diverse enough to make the data set representative. Therefore, the larger the data set, the higher the prediction accuracy of the trained model; on the contrary, the greater the variability of the model performance. However, in step S100, the physical and chemical properties of different soils and adsorption experimental data obtained by detecting from a large number of literatures are limited, and the division result of a single random seed may be too one-sided (for example, there is an extreme situation where the weak adsorption subset and the core subset are divided into the training set, and the strong adsorption subset is divided into the test set, which will result in a low test set verification score). Therefore, by introducing 10 random seeds, extreme situations are avoided.
[0106] In step S130, the arsenic pollution risk assessment main model is trained using multiple groups of random state training sets, including: first, the arsenic pollution risk assessment main model is trained using multiple groups of random state training sets respectively to obtain multiple trained arsenic pollution risk assessment main models, and then the multiple trained arsenic pollution risk assessment models are merged in a weighted averaging manner to obtain an optimized arsenic pollution risk assessment main model.
[0107] In this embodiment, different random seeds (0-9) were used in the training data division and model building stages of the arsenic pollution risk assessment main model to model under 10 different data division states, and a fusion model was formed by weighted averaging the prediction results to obtain the optimized fusion model prediction value.
[0108] For example: For example, the actual value is: 10
[0109] Random seed 0 model prediction results: 8
[0110] Random seed 1 model prediction result: 8.6
[0111] Random seed 2 model prediction results: 9
[0112] Random seed 3 model prediction result: 9.5
[0113] Random seed 4 model prediction results: 13
[0114] Random seed 5 model prediction results: 11
[0115] Random seed 6 model prediction result: 12.3
[0116] Random seed 7 model prediction results: 16
[0117] Random seed 8 model prediction results: 12
[0118] Random seed 9 model prediction result: 9.3
[0119] Calculation method: (8+8.6+9+9.5+13+11+12.3+16+12+9.3) / 10=10.87
[0120] Because it is a nonlinear model, when the true value is 10, the deviation of model 7 is large. When the true value reaches 100, the deviation of model 7 is relatively small, and the deviations of other individual models are large. This method can globally reduce the deviation of model prediction results, that is, increase R2 (the larger the better) and reduce RMSE (the smaller the better).
[0121] Compared with the model before optimization, the model uses different random seeds (0-9) to perform 10 different data partitions and model them separately, and then the prediction results of these independent models are weighted averaged to form a fusion model. This method fully considers various data distributions and partition states. This optimization strategy can effectively reduce the deviation of the model caused by data partitioning and increase the robustness and generalization ability of the model. Therefore, the optimized fusion model, i.e. the main model of arsenic pollution risk assessment, is modeled by the training set obtained by different random state partitioning. It can effectively predict the deviation of the numerical values at different stages due to the imbalance of the data set and the division of different random states during prediction, and reduce the deviation of the model prediction results from a global perspective to make the prediction more accurate. Compared with the single model before optimization, its prediction results will be more stable and accurate, and reduce the R2 result (coefficient of determination) affected by the deviation of different data partition states.
[0122] At the same time, based on the optimization method of the above-mentioned main model for arsenic pollution risk assessment, the analytical performance of the model was evaluated.
[0123] Specifically, after training the arsenic pollution risk assessment model using 10 standardized sample data sets after random seed processing and obtaining 10 trained arsenic pollution risk assessment models, the SHAP analysis method was used to calculate the SHAP values of the 10 models. After that, the SHAP values of the same features (influencing factors) were accumulated and averaged to preliminarily obtain the feature importance ranking of arsenic pollution (arsenic adsorption) influencing factors, such as Figure 5 shown.
[0124] Furthermore, the single factor partial dependence diagram analysis method was used to divide the value range of the 10 input features into 100 points, and the single factor PDP values of the 10 models were calculated and the single factor partial dependence diagram was formed. The single factor PDP diagram was used to deeply analyze the influence of the values of each soil physicochemical property on the adsorption results, so as to intuitively understand the interaction between soil physicochemical properties and As(V) adsorption behavior, for example, the influence, trend, and change of changes in soil physicochemical properties on the soil adsorption behavior of As(V), e.g. Figure 6 shown.
[0125] Furthermore, since the adsorption of As in soil is jointly affected by the chemical and electrostatic interactions on the soil surface, its adsorption process usually depends on the pH value of the solution. Therefore, pH is selected as a fixed factor and the other factor is selected as a variable. The two-factor partial dependence diagram analysis method is used to divide the value range of 9 input features and pH value into 100 points, calculate the two-factor PDP values of 10 models and form a two-factor partial dependence diagram. The two-factor PDP diagram is used to deeply analyze the influence of the values of various soil physical and chemical properties on the adsorption results under the influence of different pH values, thereby revealing the reasons for the influence of soil physical and chemical properties on the adsorption behavior of As(V) from a mechanistic level. Figure 7 As shown, factor 1 selects pH value, factor 2 selects other soil physical and chemical properties, and the effects, trends, and changes of factors 1 and 2 on the soil adsorption behavior of As(V) are analyzed. The results of the two-factor PDP diagram are explained by the principles of chemical and electrostatic interactions on the soil surface, further revealing the reasons for the importance ranking of soil physical and chemical properties.
[0126] Specifically, since the training sets divided from all sample sequences using 10 random seeds are different, the value ranges of the training sets are not exactly the same, that is, the values of the 100 points obtained by division are not exactly the same. Therefore, simply using weighted average to calculate the single-factor PDP value and the two-factor PDP value cannot accurately reflect the PDP values of the 10 models. Therefore, the nearest neighbor weighted average data processing method is introduced to solve the method of calculating the single-factor PDP value and the two-factor PDP value of the overall model under various conditions of all sample sequence division, as follows:
[0127] (1) First, define a new data set pdp_data_all to store a certain input feature value and PDP value. Calculate the single-factor PDP value and two-factor PDP value of each model in the 10 models generated by random seeds. After calculating each model, the PDP value data pdp_data of the model will be obtained. Each row of the data will be traversed and checked whether the row of data is in pdp_data_all. If not, add the data. If so, add 1 to the number of occurrences.
[0128] (2) Calculate a certain input feature of all sample sequences as another data set, and rearrange them from small to large in value, and divide its value range into 100 points with equal spacing as index values, denoted as n. For a given index value n, find the index value closest to it in the pdp_data_all data set, denoted as x. Determine the adjacent index values closest to the index value (x) in the two data sets, which are the index values before and after the current index value (n), respectively. Calculate the difference between the current index value (n) and the above adjacent index values, and determine which adjacent index value is closer to the current index value (n). Add the closest index value (x) and possible other closest adjacent index values to a list, which will be used for subsequent calculations.
[0129] (3) Its mathematical expression is:
[0130] ① Let the index value of 100 equally spaced points of a certain input feature of the sample sequence be n, and the index value in the pdp_data_all data set be x.
[0131] ② Find the x closest to n: x = argmin(|xn|), where argmin represents the parameter that makes the function reach the minimum value.
[0132] ③Determine the adjacent pdp_data_all data index value:
[0133] x up1 =argmin(|xn up1 ∣),
[0134] x up2 =argmin(|xn up2 ∣),
[0135] x down1 =argmin(|xn down1 ∣),
[0136] x down2 =argmin(|xn down2 ∣),
[0137] Among them, nup1, nup2, ndown1, and ndown2 respectively represent the previous and next index values of n.
[0138] ④ Calculate the differences:
[0139] diff1 = ∣x - x up1 ∣,
[0140] diff2 = ∣x - x up2 ∣,
[0141] diff3 = ∣n - n up1 ∣,
[0142] diff4 = ∣n - n up2 ∣,
[0143] diff5 = ∣x - xdown1∣,
[0144] diff6 = ∣x - xdown2∣,
[0145] diff7 = ∣n - n down1 ∣,
[0146] diff8 = ∣n - n down2 ∣,
[0147] ⑤ Determine the closest index value based on the comparison of the differences, and add the qualified values to the Values dataset. Values = [x], that is:
[0148] If diff1 < diff3 and diff2 < diff4, then add x up1 and x up2 to Values;
[0149] If diff5 < diff7 and diff6 < diff8, then add x down1 and x down2 to Values.
[0150] ⑥ If n is the first value (the first of 100), then only calculate x up1 , x up2 , n up1 , n up2 ; if n is the last value (the last of 100), then only calculate x down1 , x down2 , n down1 , n down2 .
[0151] ⑦ Calculate the PDP data of the nearest neighbor weighted average of Values: Take n as the index, each index corresponds to multiple x close to n and the number of times x appears. Use the weighted average method to calculate the PDP value corresponding to n:
[0152]
[0153] Get 100 equally spaced points of a certain input feature of all sample sequences and the corresponding PDP values, and use this data to generate single-factor PDP graphs and two-factor PDP graphs.
[0154] ⑧ The value range of the nearest neighbor is the number of random seeds divided by 5. For example, this study uses 10 random seeds to establish different models, so the value is 2, that is, to determine x up1 , x up2 , xdown1, xdown2. If the random seed increases, the value range of the nearest neighbor should be adjusted as required.
[0155] In this embodiment, the learning model for training can be a random forest (RandomForest, hereinafter referred to as RF), an extremely randomized tree (Extremely Randomized Trees, hereinafter referred to as ET), a gradient boosting decision tree (Gradient Boosting Decision Tree, GBDT), and an extreme gradient boosting (eXtremeGradient Boosting, XGB) in the ML model.
[0156] In this embodiment, the grid search cross validation (GridSearchCV) method is used to tune the hyperparameters of the machine learning model. The specific implementation method is as follows:
[0157] (1) Define the parameter range, select various parameter combinations for the model, and set a series of values to be tested for each parameter. Including: the number of estimators (n_estimators), the maximum tree depth (max_depth), and the maximum number of features used (max_features). For the XGB model, there are only n_estimators and max_depth;
[0158] (2) Data segmentation: The entire data set is divided into a training set and a test set according to a ratio of 80%:20%;
[0159] (3) 5-Kfold cross validation: The dataset is divided into 5 parts, ensuring that one subset is used as the validation set each time and the rest are used as the training set;
[0160] (4) Perform grid search: For each combination in the parameter range, calculate the average performance of the model using 5-Kfold cross validation;
[0161] (5) Select the best parameters: Load the test set data into the cross-validation result parameters and select the parameter combination with the highest score as the final parameters of the model.
[0162] After training the above four models, their prediction effects were also verified, and it was found that the performance of the ET model significantly surpassed the other three algorithms (RF, GBDT, and XGB). Based on the training data set and the test data set, a joint scatter plot of the actual and model predicted responses was constructed, and a 95% confidence interval was defined. The confidence interval of ET on the test set was much narrower than that of the other three, indicating that its prediction accuracy and stability were higher. Further evaluation was performed by comparing the coefficient of determination (R 2 ), root mean square error (RMSE) and average iteration time (Iterations Average Time, hereinafter referred to as IAT) indicators, once again confirmed the significant advantages of the ET model in multiple evaluation criteria.
[0163] like Figure 8 As shown in the figure, the scatter plot shows all the predicted values and true values after data partitioning under 10 random seeds and modeling using four different model algorithms. Figure 8 (b) The ET model algorithm, namely the gradient boosting decision tree algorithm, can effectively reduce the variance of the model by introducing randomness, thereby increasing the diversity of the model, resulting in better performance of the model on unknown data. Among its performance indicators on the test set, R2 and RMSE can reach 0.882 and 0.321 respectively, with high accuracy. This method uses machine learning to empower the method of predicting the ability of soil to adsorb arsenic. The precise model established by machine learning can quickly and accurately obtain the ability of soil to adsorb arsenic when predicting different types of soil. This solves the technical bottleneck that traditional adsorption experimental methods are time-consuming and cannot accurately predict the arsenic adsorption levels of different types of soils.
[0164] Therefore, based on the above findings, preferably, using the ET algorithm to predict the adsorption capacity of soil for As(V) can achieve better results.
[0165] In this embodiment, when training each learning model, in addition to inputting the six soil characteristic indicators into the model, the relevant experimental parameters are also included, and the corresponding true value, i.e., the adsorption value true value, is given. In this way, the obtained model will give the corresponding adsorption rate true value according to the soil characteristic indicators and a changeable experimental environment, which makes it possible to flexibly obtain the prediction results under various experimental environments through the model. At the same time, it can be known that according to the actual application requirements, the prediction results under a fixed experimental environment can also be obtained by training.
[0166] When applying the trained model:
[0167] In step S150, the characteristic indexes may be extracted from the soil sample by experimental analysis, and the specific experimental analysis method is not limited to one.
[0168] In step S160, before using the arsenic pollution risk assessment main model and the arsenic pollution risk assessment sub-model to predict according to multiple soil characteristic indicators: after analyzing multiple soil characteristic indicators using the principal component analysis method, the soil samples are classified to obtain soil sample categories, wherein the soil sample categories include core categories, strong adsorption categories, and weak adsorption categories, and the arsenic pollution risk assessment sub-model corresponding to the soil sample category is used to predict multiple soil characteristic indicators, wherein the arsenic pollution risk assessment sub-model includes a core arsenic pollution risk assessment sub-model, a strong adsorption arsenic pollution risk assessment sub-model, and a weak adsorption arsenic pollution risk assessment sub-model for respectively predicting the core category, the strong adsorption category, and the weak adsorption category. The specific content of analyzing multiple soil characteristic indicators using the principal component analysis method has been described in the aforementioned model training part, and will not be repeated here.
[0169] Furthermore, risk assessment of soil samples is performed in combination with the main prediction data and the sub-prediction data, including: extracting corresponding sub-prediction data weights according to the soil sample category, calculating a comprehensive prediction result based on the main prediction data, the sub-prediction data and the sub-prediction data weights, and performing risk assessment on the soil samples based on the comprehensive prediction result.
[0170] Specifically, referring to Table 1 above, the weight of the sub-prediction data is its corresponding overall proportion. For example, when predicting the adsorption capacity of a certain soil, it is first input into the unsupervised learning classification model generated by the principal component analysis method to evaluate the adsorption capacity type of the soil. Assuming that the soil category currently being evaluated is a strong adsorption type, the prediction result obtained by the main model is A, and the prediction result obtained by the corresponding sub-model is B, then the final prediction result data is (1-88.71%)A+88.71%B.
[0171] That is to say, in step S160, the final prediction result is calculated according to the prediction results of the main model and the sub-model in combination with the corresponding overall proportions.
[0172] In the above-mentioned soil arsenic contamination risk assessment method based on the optimization model, by combining the physical and chemical properties of the soil and the experimental conditions as features, the trained model can effectively distinguish the adsorption capacity of different types of soils for As(V). At the same time, in order to solve the problem of uneven training data when training the main model, the training set is divided by multiple random seeds to obtain multiple training sets with different proportions of soil characteristics, which further enhances the accuracy and generalization ability of the prediction model. The prediction model can adapt to the addition of new data, perform self-adjustment and optimization, and is more adaptable than traditional mathematical modeling. This method uses machine learning technology, and the present invention can provide higher prediction accuracy for a large number of different types of soils and different experimental environmental conditions, while also maintaining good interpretability.
[0173] It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.
[0174] In one embodiment, Fig. 9 As shown, a soil arsenic pollution risk assessment device based on an optimization model is provided, comprising: a sample data acquisition module 200, a sample standardization module 210, a training set acquisition module 220, a main model training module 230, a sub-model training module 240, a sample acquisition module 250 to be evaluated, and a risk assessment module 260, wherein:
[0175] The sample data acquisition module 200 is used to acquire a sample data set, wherein the sample data set includes a plurality of sample sequences, each of which corresponds to soil characteristic indicators of different soil samples in different experimental environments and corresponding experimental environment parameters;
[0176] A sample standardization module 210, configured to preprocess all sample sequences in the sample data to obtain a standardized sample data set;
[0177] The training set acquisition module 220 is used to process the standardized sample data set using multiple different random seeds to obtain corresponding multiple groups of random state training sets, and further divide the sample sequences in the standardized sample data set into a core training subset, a strong adsorption training subset and a weak adsorption training subset according to the correlation between soil physical and chemical properties and adsorption capacity;
[0178] The main model training module 230 is used to optimize and train the arsenic pollution risk assessment main model using multiple groups of random state training sets to obtain a trained arsenic pollution risk assessment main model;
[0179] The sub-model training module 240 is used to train the corresponding arsenic pollution risk assessment sub-models using the core training subset, the strong adsorption training subset and the weak adsorption training subset, respectively, to obtain the arsenic pollution risk assessment sub-models that predict the core category, the strong adsorption category and the weak adsorption category respectively;
[0180] The sample acquisition module 250 is used to acquire a plurality of soil characteristic indicators extracted from the soil sample to be risk assessed;
[0181] The risk assessment module 260 inputs a plurality of soil characteristic indicators into the main model of arsenic pollution risk assessment and the sub-model of arsenic pollution risk assessment corresponding to the soil sample category for prediction, obtains corresponding main prediction data and sub-prediction data, and performs risk assessment on the soil sample in combination with the main prediction data and the sub-prediction data.
[0182] For the specific definition of the soil arsenic contamination risk assessment device based on the optimization model, please refer to the definition of the soil arsenic contamination risk assessment method based on the optimization model above, which will not be repeated here. Each module in the above-mentioned soil arsenic contamination risk assessment device based on the optimization model can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.
[0183] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Fig.10As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a soil arsenic pollution risk assessment method based on an optimization model is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a key, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.
[0184] Those skilled in the art will understand that Fig.10 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.
[0185] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:
[0186] Acquire a sample data set, wherein the sample data set includes a plurality of sample sequences, each of the sample sequences corresponding to soil characteristic indicators of different soil samples in different experimental environments and corresponding experimental environment parameters;
[0187] Preprocessing all sample sequences in the sample data to obtain a standardized sample data set;
[0188] The standardized sample data set is processed using a plurality of different random seeds to obtain a plurality of corresponding random state training sets, and the sample sequences in the standardized sample data set are divided into a core training subset, a strong adsorption training subset, and a weak adsorption training subset according to the correlation between soil physical and chemical properties and adsorption capacity;
[0189] The main model for arsenic pollution risk assessment is optimized and trained using multiple groups of random state training sets to obtain the trained main model for arsenic pollution risk assessment;
[0190] The corresponding arsenic pollution risk assessment sub-models are trained using the core training subset, the strong adsorption training subset and the weak adsorption training subset, respectively, to obtain arsenic pollution risk assessment sub-models that predict the core category, the strong adsorption category and the weak adsorption category respectively;
[0191] Acquiring a plurality of soil characteristic indicators, wherein the plurality of soil characteristic indicators are extracted from a soil sample to be risk assessed;
[0192] A plurality of soil characteristic indicators are respectively input into the main model for arsenic pollution risk assessment and the sub-model for arsenic pollution risk assessment corresponding to the soil sample category for prediction, to obtain corresponding main prediction data and sub-prediction data, and the soil sample is subjected to risk assessment in combination with the main prediction data and the sub-prediction data.
[0193] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:
[0194] Acquire a sample data set, wherein the sample data set includes a plurality of sample sequences, each of the sample sequences corresponding to soil characteristic indicators of different soil samples in different experimental environments and corresponding experimental environment parameters;
[0195] Preprocessing all sample sequences in the sample data to obtain a standardized sample data set;
[0196] The standardized sample data set is processed using a plurality of different random seeds to obtain a plurality of corresponding random state training sets, and the sample sequences in the standardized sample data set are divided into a core training subset, a strong adsorption training subset, and a weak adsorption training subset according to the correlation between soil physical and chemical properties and adsorption capacity;
[0197] The main model for arsenic pollution risk assessment is optimized and trained using multiple groups of random state training sets to obtain the trained main model for arsenic pollution risk assessment;
[0198] The corresponding arsenic pollution risk assessment sub-models are trained using the core training subset, the strong adsorption training subset and the weak adsorption training subset, respectively, to obtain arsenic pollution risk assessment sub-models that predict the core category, the strong adsorption category and the weak adsorption category respectively;
[0199] Acquiring a plurality of soil characteristic indicators, wherein the plurality of soil characteristic indicators are extracted from a soil sample to be risk assessed;
[0200] A plurality of soil characteristic indicators are respectively input into the main model for arsenic pollution risk assessment and the sub-model for arsenic pollution risk assessment corresponding to the soil sample category for prediction, to obtain corresponding main prediction data and sub-prediction data, and the soil sample is subjected to risk assessment in combination with the main prediction data and the sub-prediction data.
[0201] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
[0202] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0203] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.
Claims
1. A soil arsenic pollution risk assessment method based on an optimization model, characterized in that: The method comprises: Acquire a sample data set, wherein the sample data set includes a plurality of sample sequences, each of the sample sequences corresponding to soil characteristic indicators of different soil samples in different experimental environments and corresponding experimental environment parameters; Preprocessing all sample sequences in the sample data to obtain a standardized sample data set; The standardized sample data set is processed using a plurality of different random seeds to obtain a plurality of corresponding random state training sets, and the sample sequences in the standardized sample data set are divided into a core training subset, a strong adsorption training subset, and a weak adsorption training subset according to the correlation between soil physical and chemical properties and adsorption capacity; The main model for arsenic pollution risk assessment is optimized and trained using multiple groups of random state training sets to obtain the trained main model for arsenic pollution risk assessment; The corresponding arsenic pollution risk assessment sub-models are trained using the core training subset, the strong adsorption training subset and the weak adsorption training subset, respectively, to obtain arsenic pollution risk assessment sub-models that predict the core category, the strong adsorption category and the weak adsorption category respectively; Acquiring a plurality of soil characteristic indicators, wherein the plurality of soil characteristic indicators are extracted from a soil sample to be risk assessed; A plurality of soil characteristic indicators are respectively input into the main model of arsenic pollution risk assessment and the sub-model of arsenic pollution risk assessment corresponding to the soil sample category for prediction, and the corresponding main prediction data and sub-prediction data are obtained, and the risk assessment of the soil sample is performed in combination with the main prediction data and the sub-prediction data.
2. The soil arsenic contamination risk assessment method according to claim 1, characterized in that: The soil characteristic indicators include: soil pH, organic matter content, cation exchange capacity, clay content, free iron oxide content, and total arsenic and phosphorus; The experimental environment parameters include: soil-liquid ratio, solution pH, background ion strength, experimental temperature, experimental time, solution initial concentration and adsorption amount.
3. The soil arsenic pollution risk assessment method according to claim 2, characterized in that: The method of training the main model for arsenic pollution risk assessment using multiple groups of random state training sets includes: Using multiple groups of random state training sets to train the arsenic pollution risk assessment main model respectively, multiple trained arsenic pollution risk assessment main models are obtained; Multiple trained arsenic pollution risk assessment models are fused in a weighted averaging manner to obtain an optimized arsenic pollution risk assessment main model.
4. The soil arsenic pollution risk assessment method according to claim 2, characterized in that: The method of dividing the sample sequences in the standardized sample data set into a core training subset, a strong adsorption training subset and a weak adsorption training subset according to the correlation between soil physicochemical properties and adsorption capacity includes: The standardized sample data set is processed by using Pearson correlation analysis and PCA principal component analysis, so as to divide the sequence data in the standardized sample data set into different groups according to the correlation between soil physical and chemical properties and adsorption capacity; Different groups were divided into core training subsets, strong adsorption training subsets and weak adsorption training subsets according to the correlation between the PCA principal component weights and the Pearson correlation coefficient of the adsorption amount.
5. The soil arsenic contamination risk assessment method according to any one of claims 1 to 4, characterized in that: Before using the arsenic pollution risk assessment main model and arsenic pollution risk assessment sub-model to make predictions based on multiple soil characteristic indicators: After analyzing the plurality of soil characteristic indicators by using principal component analysis, the soil samples are classified to obtain the soil sample categories, wherein the soil sample categories include core categories, strong adsorption categories, and weak adsorption categories; Predicting the plurality of soil characteristic indicators using an arsenic pollution risk assessment sub-model corresponding to the soil sample category; The arsenic pollution risk assessment submodel includes a core arsenic pollution risk assessment submodel, a strong adsorption arsenic pollution risk assessment submodel and a weak adsorption arsenic pollution risk assessment submodel for predicting the core category, the strong adsorption category and the weak adsorption category respectively.
6. The soil arsenic contamination risk assessment method according to claim 5, characterized in that: The risk assessment of the soil sample by combining the main prediction data and the sub-prediction data includes: Extracting corresponding sub-prediction data weights according to the soil sample category; The comprehensive prediction result is calculated based on the main prediction data, sub-prediction data and sub-prediction data weights. A risk assessment is performed on the soil sample according to the comprehensive prediction result.
7. The soil arsenic contamination risk assessment method according to claim 5, characterized in that: When using the arsenic pollution risk assessment main model and the arsenic pollution risk assessment sub-model to predict soil samples based on multiple soil characteristic indicators, parameters related to the preset experimental environment are also input into the arsenic pollution risk assessment main model and the arsenic pollution risk assessment sub-model to obtain the prediction results under the corresponding preset experimental environment.
8. A soil arsenic pollution risk assessment device based on an optimization model, characterized in that: The device comprises: A sample data acquisition module is used to acquire a sample data set, wherein the sample data set includes a plurality of sample sequences, each of which corresponds to soil characteristic indicators of different soil samples in different experimental environments and corresponding experimental environment parameters; A sample standardization module, used to preprocess all sample sequences in the sample data to obtain a standardized sample data set; A training set acquisition module is used to process the standardized sample data set using multiple different random seeds to obtain corresponding multiple groups of random state training sets, and further divide the sample sequences in the standardized sample data set into a core training subset, a strong adsorption training subset and a weak adsorption training subset according to the correlation between soil physical and chemical properties and adsorption capacity; A main model training module is used to optimize and train the arsenic pollution risk assessment main model using multiple groups of random state training sets to obtain a trained arsenic pollution risk assessment main model; A sub-model training module is used to train the corresponding arsenic pollution risk assessment sub-models using the core training subset, the strong adsorption training subset and the weak adsorption training subset, respectively, to obtain arsenic pollution risk assessment sub-models that predict the core category, the strong adsorption category and the weak adsorption category respectively; A sample acquisition module to be evaluated is used to obtain a plurality of soil characteristic indicators, wherein the plurality of soil characteristic indicators are extracted from a soil sample to be risk evaluated; The risk assessment module inputs multiple soil characteristic indicators into the arsenic pollution risk assessment main model and the arsenic pollution risk assessment sub-model corresponding to the soil sample category for prediction, obtains corresponding main prediction data and sub-prediction data, and conducts risk assessment on the soil sample in combination with the main prediction data and the sub-prediction data.
9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.
Citation Information
Cited By
Industrial gathering area pollution risk management method and system based on multi-source data
CN119990769A
Industrial agglomeration area pollution risk management method and system based on multi-source data
CN119990769B
Method and system for evaluating sewage treatment quality of mixing station
CN121639045A
Soil geology catalysis capability prediction method and related equipment
CN121830832A
Regulation and control method and system for As-polluted paddy field soil
CN121936859A
