Municipal sewage biotoxicity dose effect prediction method based on machine learning

Abstract

The invention provides a municipal sewage biotoxicity dose effect prediction method based on machine learning, and the method comprises the following steps: building a municipal sewage database, and carrying out the preprocessing of the data in the municipal sewage database; adopting a feature selection method to select data with the importance ranked in the first three as input features; setting missing values for the toxicity effect values under different doses in the screened input features to obtain final input features; constructing a regression model, and training the regression model by taking the final input features as input; and through the trained regression model, predicting a dose effect curve, and completing the single-point prediction municipal sewage biotoxicity dose effect curve based on machine learning, which is used for monitoring the municipal sewage toxicity level. Compared with the traditional method, the method provided by the invention has the advantages that at least 75% of experimental workload is saved, and the toxicity change rule of the municipal sewage can be accurately predicted for subsequent research.

Description

Technical Field

[0001] The invention relates to a sewage data analysis method, in particular to a municipal sewage biological toxicity dose effect prediction method based on machine learning. Background Art

[0002] This section merely provides background information related to the present disclosure and is not necessarily prior art.

[0003] It is impossible to accurately assess the complex impact of pollutants in municipal sewage by relying solely on existing basic chemical indicators.

[0004] Certain model organisms are widely used to assess the comprehensive toxicity of chemical substances in water quality due to their fast reproduction, sensitivity, small footprint, simple cultivation, and short life cycle. The dose-effect curves of the above-mentioned model organisms that are fitted to municipal sewage are used to describe the comprehensive toxic effects of municipal sewage on organisms at different doses. The dose-effect curve is an S-shaped curve that satisfies the law of the logistic equation. It is a regular formula formed by calculating the biological toxicity effect values ​​at multiple different doses. It can reflect the sensitivity of organisms to changes in municipal sewage doses, provide a scientific basis for formulating safe exposure limits for municipal sewage, and is of great significance for environmental protection and human health risk assessment.

[0005] However, the calculation of the dose-effect curve of model organisms in municipal sewage is a costly and time-consuming experimental process. The first reason is that the calculation of the dose-effect curve requires at least 5 to 8 experimental data points of the biological toxicity effect of model organisms at different doses, which requires at least hundreds of model organisms and continuous and regular observation of the test results. This process is time-consuming and labor-intensive. The second reason is that municipal sewage is complex and changeable, and it is difficult to grasp the law of toxicity changes. If the dose concentration is set incorrectly, the model organism will not have obvious effects or the effects will be too large in the samples exposed to all doses, and the workload cost will double if the experiment is repeated.

[0006] The common model organism in the prior art is the zebrafish embryo. It has been recognized by the International Organization for Standardization (ISO) and other organizations, and has been included in many international and national standards and guidelines. It has the characteristics of transparency, easy observation, and similarity with human genes.

[0007] In recent years, machine learning technology has been widely used in water environment monitoring and management due to its powerful data mining and analysis capabilities. Data-driven machine learning models can find complex nonlinear relationships in water environment big data, effectively process high-dimensional data, and find potential laws and patterns. At present, there are regression models that predict the values ​​of conventional chemical indicators in actual water environments, but the prediction of toxicity can only use classification models to simply classify the toxicity levels of single chemicals. Therefore, there is an urgent need for a regression model that can predict the toxicity values ​​of complex actual water bodies such as municipal sewage.

[0008] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0009] Purpose of the invention: The technical problem to be solved by the present invention is to provide a method for predicting the biological toxicity dose effect of municipal sewage based on machine learning in response to the shortcomings of the prior art.

[0010] In order to solve the above technical problems, the present invention discloses a method for predicting the biological toxicity dose effect of municipal sewage based on machine learning, comprising the following steps:

[0011] Step 1, establishing a municipal sewage database and preprocessing the data in the municipal sewage database; wherein the data includes: conventional chemical indicators of municipal sewage, toxic effect values ​​at different doses and dose-effect curve information;

[0012] Step 2, screening input features, using a feature selection method, selecting the top three indicators ranked in importance among the conventional chemical indicators and toxic effect values ​​at different doses as input features;

[0013] Step 3, preprocessing the input features, setting missing values ​​for the toxic effect values ​​at different doses in the screened input features, and obtaining the final input features;

[0014] Step 4, construct a regression model, take the final input features as input, and train the regression model; predict the dose-effect curve through the trained regression model, and complete the single-point prediction of municipal sewage biological toxicity dose-effect curve based on machine learning, which is used for municipal sewage toxicity level monitoring.

[0015] Furthermore, the establishment of a municipal sewage database described in step 1 specifically includes:

[0016] Step 1-1, collect influent and effluent samples from different municipal sewage treatment plants, measure and record the conventional chemical indicators and toxic effect values ​​at different doses;

[0017] Step 1-2, calculating a dose-effect curve fitting formula according to the toxic effect values ​​at different doses, and recording the constants in the formula as dose-effect curve information;

[0018] Step 1-3, using the average values ​​of the above conventional chemical indicators, toxic effect values ​​at different doses and dose-effect curve information collected from the sewage treatment plant to be predicted to replace the missing values, abnormal values ​​or repeated values ​​in the data recorded in step 1-1 and step 1-2;

[0019] Step 1-4, storing the data processed by step 1-3 into the municipal sewage database, and standardizing the data in the database.

[0020] Furthermore, the dose-effect curve fitting formula described in step 1-2, that is, the logistic equation satisfied by the dose-effect curve, is as follows:

[0021]

[0022] Wherein, (X, Y) represents a point on the curve, Max represents the maximum toxic effect, which is 100%, and Min represents the minimum toxic effect, which is 0;

[0023] The first constant E(L,I)C 50 represents the half toxic effect concentration, and the second constant Hill slope represents the slope. The above two constants are the predicted dose-effect curve information.

[0024] Furthermore, in the logistic equation described in step 1-2,

[0025] The first constant E(L,I)C 50 , according to different toxic effects, they are respectively the half effective concentration EC 50 , median lethal concentration LC 50 or half inhibitory concentration IC 50 .

[0026] Furthermore, the screening input features described in step 2 specifically include:

[0027] Step 2-1, ranking the characteristic importance of the conventional chemical indicators and the toxic effect values ​​at different doses by using the maximum relevance minimum redundancy method mRMR;

[0028] Step 2-2: According to the ranking result, select the top three indicators as input features.

[0029] Furthermore, the top three indicators of importance described in step 2-2 include:

[0030] 1 conventional chemical indicator, 1 toxic effect value with the maximum relevant dose for influent samples and 1 toxic effect value with the maximum relevant dose for effluent samples.

[0031] Furthermore, the preprocessing of the input features described in step 3 specifically includes:

[0032] Step 3-1, integrating the top three important indicators obtained in step 2-2 into a new database;

[0033] Step 3-2, adding all the toxic effect indicators of different doses obtained in step 1 into a new database;

[0034] Step 3-3, setting the data in the new database processed by step 3-2 to obtain the final input features.

[0035] Further, the setting described in step 3-3 specifically includes:

[0036] 50% of the data in the new database are randomly kept as they are, and in the other 50% of the data, except for the top three indicators of importance, the rest of the data are all set to missing values ​​NaN to obtain the final input features.

[0037] Furthermore, the predicted dose-effect curve described in step 4 specifically includes:

[0038] Step 4-1: Take the final input features obtained in step 3 as input, and the first constant E(L,I)C 50 As output, substitute into the regression model for training;

[0039] Step 4-2, using the trained regression model, inputting the toxic effect value and chemical index of the sewage treatment plant to be predicted at a single dose, and predicting the first constant E(L,I)C 50 , and substitute it into the logistic equation to calculate the second constant Hill slope;

[0040] Step 4-3, combine the first constant E(L,I)C 50 and the second constant Hill slope to obtain the predicted dose-effect curve.

[0041] Furthermore, the regression model described in step 4 includes:

[0042] Decision Tree model, Random Forest model, or Gradient Boosted Tree model.

[0043] Beneficial effects:

[0044] The method proposed in the present invention can use single-point toxic effect value data to accurately predict the dose-effect curve, reducing at least 75% of experimental costs and time, and improving work efficiency. It is suitable for large-scale municipal sewage toxicity level monitoring, and can also be extended to other types of environmental pollutant toxicity prediction, such as industrial wastewater, agricultural runoff, and lake and river pollution. It has broad application prospects and market potential, and can provide strong support for environmental protection. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments, and the above and / or other advantages of the present invention will become more clear.

[0046] Figure 1 It is a schematic diagram of the overall process of the method of the present invention.

[0047] Figure 2 It is a schematic diagram of the results of the mRMR feature selection method.

[0048] Figure 3 It is a schematic diagram of the prediction performance improvement results. DETAILED DESCRIPTION

[0049] Starting from the perspective that there is a close connection between the chemical indicators of municipal sewage and the toxicity level, the present invention uses a machine learning regression algorithm to predict the dose-effect curve of municipal sewage and to discover the key indicators that affect the toxicity level of municipal sewage, providing a reference and standard for the formulation of safe exposure limits for municipal sewage.

[0050] The specific technical solution of the present invention comprises the following steps:

[0051] Step 1: Establish a database of conventional chemical indicators of municipal sewage, toxic effect values ​​at different doses, and dose-effect curves, and pre-process the data;

[0052] Step 1-1: Collect influent and effluent samples from hundreds of representative and universal municipal sewage treatment plants in various provinces and cities across the country, and measure their conventional chemical indicators and toxic effect values ​​at different doses;

[0053] Step 1-2, calculating the dose-effect curve fitting formula according to the toxic effect values ​​at different doses, recording the constants in the formula, and obtaining the database including conventional chemical indicators, toxic effect values ​​at different doses, and dose-effect curve constants;

[0054] In step 1-2, the dose-effect curve is essentially a logistic equation, and its formula is as follows:

[0055]

[0056] Wherein, Max represents the maximum toxic effect, which is 100% in the biological toxicity test of the present invention, and Min represents the minimum toxic effect, which is 0 in the biological toxicity test of the present invention;

[0057] Therefore, the constant in this formula is the half toxic effect concentration E(L,I)C 50 Record the two as the predicted object, i.e. the output of the model, and then you can get the predicted dose-effect curve.

[0058] Different toxic effects are represented by different abbreviations. This formula is universal for various toxic effects. In this formula, the half-effective concentration EC 50 The median lethal concentration (LC) 50 , half inhibitory concentration IC 50 When the model organism shows a lethal effect, the LC 50 When a model organism shows some biological effect inhibition, such as hatching inhibition, IC 50 When other effects similar to malformations occur in model organisms, EC 50 express.

[0059] In steps 1-3, the average values ​​of the indicators measured by the local sewage treatment plants were used to replace the missing values, abnormal values ​​or duplicate values ​​in the collected data;

[0060] Steps 1-4: standardize the data in the database.

[0061] Step 2: Use input feature selection algorithm to obtain the top 3 input features in terms of importance.

[0062] Step 2-1, sort the input features by their importance using the maximum relevance minimum redundancy (mRMR) algorithm. The mRMR algorithm is a commonly used feature selection method for processing high-dimensional data. It calculates the correlation between the input features and the output, and the redundancy between the features, and comprehensively considers the two to obtain the comprehensive score of each feature, that is, the importance, to select the input features. (Reference: Peng, H.; Long, F.; Ding, C. Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy. IEEE Trans. Pattern Anal. Mach. Intell. 2005, 27 (8), 1226-1238.)

[0063] In step 2-1, the input features are the inputs required for the subsequent construction of the machine learning model, which in this database are conventional chemical indicators and toxic effect values ​​at different doses;

[0064] Step 2-2, the mRMR result will show the importance of all indicators, and the top three indicators will be selected;

[0065] In step 2-2, according to the mRMR algorithm to obtain the maximum correlation, the algorithm principle of discarding redundancy and the dose-effect curve calculation principle, it can be known that the top two rankings must be the toxic effect values ​​of different doses used to calculate the dose-effect curve in the actual experiment, one maximum relevant dose for water inlet and one maximum relevant dose for water outlet. The toxic effect values ​​at other doses are ranked lower due to their large redundancy with the two. This step can screen the single-point dose toxic effect values ​​suitable for predicting the dose-effect curve, and one maximum relevant single-point dose for water inlet and one maximum relevant single-point dose for water outlet, which are used to predict the dose-effect curve;

[0066] The third most important feature was the general chemical indicator with the greatest correlation with the dose-effect curve and the least redundancy with other input features.

[0067] Step 3, intentionally set missing values ​​for the toxic effect values ​​at different doses of the input features;

[0068] Step 3-1, extract the top three input features ranked by importance of the three features obtained in step 2, one conventional chemical index, one toxic effect value of the maximum relevant dose of influent, and one toxic effect value of the maximum relevant dose of effluent, and integrate them into a new Excel database;

[0069] Step 3-2, adding the toxic effect values ​​of different doses of all municipal sewage samples obtained in the experiment into the new database to integrate the logistic equation between each dose-effect. The toxic effect values ​​of different doses and the three input features in step 3-1 jointly construct a complete input feature;

[0070] In step 3-3, all input features of 50% of the random municipal sewage samples in the newly generated database in step 3-2 are kept as they are, and for the remaining 50% of the municipal sewage samples, except for the top three input features in terms of feature importance, all other input feature data are set to missing values ​​NaN.

[0071] Step 4: Build the model, using the input features in step 3 as the input of the model, EC 50 (LC 50 or IC 50 ) as the output of the model, and the dose-effect curve was predicted by regression model;

[0072] Step 4-1: Take all input features in the database obtained in step 3 as input, EC 50 (LC 50 or IC 50 ) as output;

[0073] Step 4-2, the regression model can use decision trees, random forests, gradient boosting trees and other models, which can train and predict models for databases with a large number of missing values ​​and capture effective information (reference: Tierney, NJ; Harden, FA; Harden, MJ; Mengersen, KL Using decision trees to understand structure in missing data. BMJ open 2015, 5 (6), e007450.);

[0074] Step 4-3: After the trained model is trained, the toxic effect value at the maximum relevant dose and conventional chemical indicators at a single point can be input to predict the EC 50 (LC 50 or IC 50 ), substitute it into the formula in step 1-2, and use it to calculate the only unknown constant Hillslope;

[0075] Step 4-4, Combine EC 50 (LC 50 or IC 50 ) and Hill slope, that is, predicting the dose-effect curve.

[0076] Step 5: The prediction results are obtained using the commonly used R 2 , root mean square error RMSE, and mean absolute percentage error MAPE are used to evaluate model performance, which are three widely used machine learning model evaluation parameters (reference: Chicco, D.; Warrens, MJ; Jurman, G. The coefficient of determination R-squared is more informative than SMAPE, MAE, MAPE, MSE and RMSE in regression analysis evaluation. Peerj computerscience 2021, 7, e623.).

[0077] Embodiment 1:

[0078] like Figure 1 As shown, a specific embodiment of the method proposed by the present invention comprises the following steps:

[0079] S1. Establish a database of conventional chemical indicators of municipal sewage and toxic effect values ​​at different doses, and pre-process the data;

[0080] (1) Collect inlet and outlet water from nearly 200 representative and universal municipal sewage treatment plants in various provinces and cities across the country;

[0081] (2) Test the conventional chemical indicators of the influent and effluent of all municipal sewage plants and the toxic effect values ​​at different doses. The details of the indicators are as follows:

[0082] Chemical indicators: TOC, COD, TN, TP, NH 3 -N, NO 3 -N, NO 2 -N;

[0083] Toxic effect values ​​at different doses: lethality of model organisms at 12 hpf when exposed to REF0.25, REF0.5, REF1, and REF2 water doses (equivalent to the lethality of model organisms 12 hours after fertilization when exposed to four doses of 0.25, 0.5, 1, and 2 times the water doses), hatching inhibition rate of model organisms at 72 hpf when exposed to REF0.25, REF0.5, REF1, and REF2 water doses (equivalent to the hatching inhibition rate of model organisms 72 hours after fertilization when exposed to four doses of REF0.25, REF0.5, REF1, and REF2 water doses), deformity rate of model organisms at 120 hpf when exposed to REF0.25, REF0.5, REF1, and REF2 water doses (equivalent to the deformity rate of larvae of model organisms 120 hours after fertilization when exposed to four doses of REF0.25, REF0.5, REF1, and REF2 water doses). The lethality rate of the model organisms at 12 hpf of REF10, REF15, REF20, and REF25 effluent water (equivalent to the lethality rate of the model organisms 12 hours after fertilization when exposed to four doses of 10, 15, 20, and 25 times the effluent water), the hatching inhibition rate of the model organisms at 72 hpf of REF10, REF15, REF20, and REF25 effluent water (equivalent to the hatching inhibition rate of the model organisms 72 hours after fertilization when exposed to four doses of REF10, REF15, REF20, and REF25 times the effluent water), and the deformity rate of the model organisms at 120 hpf of REF10, REF15, REF20, and REF25 effluent water (equivalent to the deformity rate of the larvae of the model organisms 120 hours after fertilization when exposed to four doses of REF10, REF15, REF20, and REF25 times the effluent water).

[0084] (3) Calculate the dose-effect curve fitting formula based on the toxic effect values ​​at different doses and record the constants in the formula. The specific data are as follows:

[0085] The lethal dose-effect curve of the 12 hpf model organism was calculated using the lethal rates of the blank control group and the model organisms at 12 hpf in the influent REF0.25, REF0.5, REF1, and REF2. The lethal dose-effect curve of the 12 hpf model organism was calculated using the lethal rates of the blank control group and the model organisms at 12 hpf in the effluent REF10, REF15, REF20, and REF25. The LC 50 Establish a lethal effect database with Hill slope, conventional chemical indicators, and lethal toxicity effect values ​​of model organisms in water at different doses;

[0086] The hatching inhibition dose-effect curve of 72 hpf model organisms was calculated using the hatching inhibition rates of the blank control group and the model organisms of REF0.25, REF0.5, REF1, and REF2 at 72 hpf. The hatching inhibition dose-effect curve of 72 hpf model organisms was calculated using the hatching inhibition rates of the blank control group and the model organisms of REF10, REF15, REF20, and REF25 at 72 hpf. The IC 50 Establish a hatching inhibition effect database with Hill slope, conventional chemical indicators, and hatching inhibition toxicity effect values ​​of model organisms in water at different doses;

[0087] The dose-effect curve of the 120 hpf model organism was calculated using the 120 hpf deformity rates of the blank control group and the model organisms of REF0.25, REF0.5, REF1, and REF2 in the influent. The dose-effect curve of the 120 hpf model organism was calculated using the 120 hpf deformity rates of the blank control group and the model organisms of REF10, REF15, REF20, and REF25 in the effluent. The EC 50 A teratogenic effect database was established based on Hill slope, conventional chemical indicators, and teratogenic effect values ​​of model organisms in influent and effluent water at different doses.

[0088] (4) The alternative input features of the model in the three databases: conventional chemical indicators, biological toxicity effect values ​​of inlet and outlet water models at different doses were normalized by z-score to eliminate the impact of different dimensions.

[0089] S2, using feature selection algorithm, obtains the top 3 input features in terms of importance:

[0090] The mRMR feature selection algorithm was used to obtain the top three input features in terms of feature importance. According to preliminary experiments, the lethal toxicity effect database, hatching inhibition effect database, and teratogenic effect database should each include one chemical indicator, a toxicity effect value at a dose related to the influent, and a toxicity effect value at a dose related to the effluent.

[0091] S3, intentionally set missing values ​​for the toxic effect values ​​at different doses of the input features:

[0092] (1) The three input features obtained from the lethal toxicity effect database, hatching inhibition effect database, and teratogenic effect database in S2 were integrated into their respective new databases;

[0093] (2) All the toxic effect indicators of different doses obtained from the experiment were added to the new database to incorporate the logistic equation between different doses-effects. At this time, each database contained 9 input features, namely, a conventional chemical indicator, the toxic effect values ​​of 4 doses of influent (REF0.25, REF0.5, REF1, REF2), and the toxic effect values ​​of 4 doses of effluent (REF10, REF15, REF20, REF25);

[0094] (3) 50% of the data points in the newly generated database are kept as they are so that the model can learn the law of the logistic curve; for the remaining 50% of the data points, except for the top three indicators in terms of feature importance, all other indicator data are set to pseudo missing values ​​NaN. In fact, among the 9 input features of these 50% of the data points, only 2 input features have data, that is, the influent data only contains one chemical indicator and the maximum relevant dose toxic effect value of the influent, and the rest are NaN; the effluent data only contains one chemical indicator and the maximum relevant dose toxic effect value of the effluent, and the rest are NaN, so that the model can learn how to predict the dose-effect curve with single-point toxicity data.

[0095] S4, using the selected key indicators as input features to predict the dose-effect curve through the regression model:

[0096] The nine input features containing pseudo missing values ​​NaN in the three newly generated databases are used to construct three regression models for predicting EC. 50 (LC 50 or IC 50 ), the regression model needs to select algorithms such as decision trees, random forests, and gradient boosting decision trees that have special ways to deal with missing values. They can still learn data patterns when the data is incomplete and provide more reliable prediction results.

[0097] S5, prediction results are obtained using R 2 , RMSE, and MAPE are used to evaluate the model performance.

[0098] Embodiment 2:

[0099] S101. Select 5 representative municipal sewage treatment plants in all provinces and cities in China, and collect inlet and outlet water samples from the municipal sewage treatment plants.

[0100] S102. Pre-treat the water sample, filter it, and then detect all the conventional chemical indicators of the inlet and outlet water of the municipal sewage plant; after solid extraction and concentration, detect the toxic effect values ​​of the model organisms at different doses.

[0101] S103, according to the lethal effect value, hatching inhibition effect value, and teratogenic effect value at different doses, the dose-effect curve fitting formula was calculated respectively, and the constant EC in the formula was 50 (LC 50 or IC 50 ) records, and three databases were constructed with chemical indicators, lethal effect values, hatching inhibition effect values, and teratogenic effect values. Missing values, duplicate values, and abnormal values ​​were processed.

[0102] S104, z-score normalization is performed on the candidate input features in the database, i.e., chemical indicators and toxic effect values ​​at different doses, to eliminate the impact of different dimensions, EC 50 (LC 50 or IC 50 ) as output data to prepare for subsequent modeling and data analysis.

[0103] S201, using the mRMR feature selection algorithm, obtain the top 3 input features in terms of feature importance. The results are as follows Figure 2 As shown in the figure, the top three feature importance scores in the lethal toxicity effect database, hatching inhibition effect database, and teratogenic effect database are NH 3 -N, the toxic effect value of REF2 related to the influent, and the toxic effect value of REF25 related to the effluent.

[0104] S301, integrating the three input features obtained from the lethal toxicity effect database, hatching inhibition effect database, and teratogenicity effect database in S2 into their own new databases, i.e., the lethal toxicity effect database contains three input features, NH 3 -N, lethal toxicity effect value of REF2 related to influent, lethal toxicity effect value of REF25 related to effluent; the hatching inhibition effect database contains 3 input features, NH 3 -N, hatching inhibition toxicity effect value of REF2 related to influent, hatching inhibition toxicity effect value of REF25 related to effluent; the teratogenicity database contains 3 input features, NH 3 -N, the teratogenic effect value of REF2 related to influent, and the teratogenic effect value of REF25 related to effluent;

[0105] S302, all the toxic effect values ​​of different doses obtained from the experiment are added to the new database to incorporate the logistic equation between different doses-effects. At this time, each database contains 9 input features, namely NH3 -N, the toxic effect values ​​of four influent doses (REF0.25, REF0.5, REF1, REF2), and the toxic effect values ​​of four effluent doses (REF10, REF15, REF20, REF25);

[0106] S304, keep 50% of the data points in the newly generated database as they are for the model to learn the logistic curve law; for the remaining 50% of the data points, except for the top three indicators in terms of feature importance, all other indicator data are set to pseudo missing values ​​NaN. In fact, among the 9 input features of these 50% of the data points, only 2 input features have data, that is, the influent data only contains NH 3 -N and dose REF2 toxic effect value, the rest are NaN, and the effluent data only include NH 3 -N and dose REF25 toxic effect value, and the rest are NaN, so that the model can learn how to predict the dose-effect curve with single-point toxicity data.

[0107] S401, construct three regression models using the nine input features containing pseudo missing values ​​NaN in the three newly generated databases to predict EC 50 (LC 50 or IC 50 ), the regression model selects decision trees, random forests, gradient boosting decision trees and other algorithms that have special ways to deal with missing values. They can still learn data patterns when the data is incomplete and provide more reliable prediction results.

[0108] S402, EC 50 (LC 50 or IC 50 ) and the single-point dose toxicity effect value, that is, the influent REF2 toxicity effect value or the effluent REF25 toxicity effect value, are substituted into the dose-effect curve formula to obtain the Hill slope value, which is the successful prediction of the model organism dose-effect curve of municipal sewage.

[0109] S501, prediction results are based on R 2 , RMSE, and MAPE are used to evaluate the model performance. The gradient boosting decision tree is used as an example to show the improvement of the model prediction performance before and after setting pseudo missing values. Figure 3 As shown in Figure 2, after intentionally setting pseudo-missing values, the MAPE of the three models increased by an average of 46.23%.

[0110] Embodiment 3:

[0111] The model organisms in the aforementioned embodiments can all be zebrafish embryos.

[0112] In a specific implementation, the present application provides a computer storage medium and a corresponding data processing unit, wherein the computer storage medium can store a computer program, and when the computer program is executed by the data processing unit, the invention content of a method for predicting the biological toxicity dose effect of municipal sewage based on machine learning provided by the present invention and some or all of the steps in each embodiment can be run. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0113] Those skilled in the art can clearly understand that the technical solutions in the embodiments of the present invention can be implemented by means of computer programs and their corresponding general hardware platforms. Based on such an understanding, the technical solutions in the embodiments of the present invention can be essentially or partly contributed to the prior art in the form of computer programs, i.e., software products, which can be stored in a storage medium and include several instructions for enabling a device including a data processing unit (which can be a personal computer, a server, a single-chip microcomputer, an MCU or a network device, etc.) to execute the methods described in various embodiments of the present invention or certain parts of the embodiments.

[0114] The present invention provides a method and idea for predicting the biological toxicity dose effect of municipal sewage based on machine learning. There are many methods and approaches to implement the technical solution. The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the protection scope of the present invention. All components not specified in this embodiment can be implemented by existing technologies.

Claims

1. A method for predicting the biological toxicity dose effect of municipal sewage based on machine learning, characterized in that: The steps include: Step 1, establishing a municipal sewage database and preprocessing the data in the municipal sewage database; wherein the data includes: conventional chemical indicators of municipal sewage, toxic effect values ​​at different doses and dose-effect curve information; Step 2, screening input features, using a feature selection method, selecting the top three indicators ranked in importance among the conventional chemical indicators and toxic effect values ​​at different doses as input features; Step 3, preprocessing the input features, setting missing values ​​for the toxic effect values ​​at different doses in the screened input features, and obtaining the final input features; Step 4, construct a regression model, take the final input features as input, and train the regression model; predict the dose-effect curve through the trained regression model, and complete the single-point prediction of municipal sewage biological toxicity dose-effect curve based on machine learning, which is used for municipal sewage toxicity level monitoring.

2. The method for predicting the biological toxicity dose effect of municipal sewage based on machine learning according to claim 1, characterized in that: The establishment of a municipal sewage database as described in step 1 specifically includes: Step 1-1, collect influent and effluent samples from different municipal sewage treatment plants, measure and record the conventional chemical indicators and toxic effect values ​​at different doses; Step 1-2, calculating a dose-effect curve fitting formula according to the toxic effect values ​​at different doses, and recording the constants in the formula as dose-effect curve information; Step 1-3, using the average values ​​of the above conventional chemical indicators, toxic effect values ​​at different doses and dose-effect curve information collected from the sewage treatment plant to be predicted to replace the missing values, abnormal values ​​or repeated values ​​in the data recorded in step 1-1 and step 1-2; Step 1-4, storing the data processed by step 1-3 into the municipal sewage database, and standardizing the data in the database.

3. The method for predicting the biological toxicity dose effect of municipal sewage based on machine learning according to claim 2 is characterized in that: The dose-effect curve fitting formula described in step 1-2, that is, the logistic equation satisfied by the dose-effect curve, is as follows: Wherein, (X, Y) represents a point on the curve, Max represents the maximum toxic effect, which is 100%, and Min represents the minimum toxic effect, which is 0; The first constant E(L,I)C 50 represents the half-maximal toxic effect concentration, and the second constant Hillslope represents the slope. The above two constants are the predicted dose-effect curve information.

4. The method for predicting the biological toxicity dose effect of municipal sewage based on machine learning according to claim 3 is characterized in that: In the logistic equation described in step 1-2, The first constant E(L,I)C 50 , according to different toxic effects, they are respectively the half effective concentration EC 50 , median lethal concentration LC 50 or half inhibitory concentration IC 50 .

5. The method for predicting the biological toxicity dose effect of municipal sewage based on machine learning according to claim 4, characterized in that: The screening input features described in step 2 specifically include: Step 2-1, ranking the characteristic importance of the conventional chemical indicators and the toxic effect values ​​at different doses by using the maximum relevance minimum redundancy method mRMR; Step 2-2: According to the ranking result, select the top three indicators as input features.

6. The method for predicting the biological toxicity dose effect of municipal sewage based on machine learning according to claim 5, characterized in that: The top three indicators of importance described in step 2-2 include: 1 conventional chemical indicator, 1 toxic effect value with the maximum relevant dose for influent samples and 1 toxic effect value with the maximum relevant dose for effluent samples.

7. The method for predicting the biological toxicity dose effect of municipal sewage based on machine learning according to claim 6, characterized in that: The preprocessing of the input features in step 3 specifically includes: Step 3-1, integrating the top three important indicators obtained in step 2-2 into a new database; Step 3-2, adding all the toxic effect indicators of different doses obtained in step 1 into a new database; Step 3-3, setting the data in the new database processed by step 3-2 to obtain the final input features.

8. The method for predicting the biological toxicity dose effect of municipal sewage based on machine learning according to claim 7, characterized in that: Perform the settings described in step 3-3, including: 50% of the data in the new database are randomly kept as they are, and in the other 50% of the data, except for the top three indicators of importance, the rest of the data are all set to missing values ​​NaN to obtain the final input features.

9. The method for predicting the biological toxicity dose effect of municipal sewage based on machine learning according to claim 8, characterized in that: The predicted dose-effect curve described in step 4 specifically includes: Step 4-1: Take the final input features obtained in step 3 as input, and the first constant E(L,I)C 50 As output, substitute into the regression model for training; Step 4-2, using the trained regression model, inputting the toxic effect value and chemical index of the sewage treatment plant to be predicted at a single dose, and predicting the first constant E(L,I)C 50 , and substitute it into the logistic equation to calculate the second constant Hill slope; Step 4-3, combine the first constant E(L,I)C 50 and the second constant Hill slope to obtain the predicted dose-effect curve.

10. The method for predicting the biological toxicity dose effect of municipal sewage based on machine learning according to claim 9, characterized in that: The regression model described in step 4 includes: Decision Tree model, Random Forest model, or Gradient Boosted Tree model.

Images