A method for identifying key influencing factors of membrane fouling in anaerobic membrane bioreactors

Abstract

The present invention discloses a method for identifying key influencing factors of membrane fouling in anaerobic membrane bioreactors, comprising the following steps: collecting and screening raw data; constructing a data set; building a random forest model based on a decision tree; outputting prediction results; and identifying key influencing factors. Compared to traditional mathematical models, the present invention offers advantages such as flexible data selection and the absence of the need for prior assumptions. It can more accurately predict the progress of membrane fouling in anaerobic membrane bioreactors and identify key influencing factors among the many factors affecting membrane fouling. This method overcomes the limitations of traditional mathematical models, such as limited simulation effectiveness and poor versatility, and provides a reference for understanding membrane fouling mechanisms and a theoretical basis for controlling membrane fouling in anaerobic membrane bioreactors.

Description

Technical Field

[0001] The present invention relates to the field of sewage treatment, and in particular to a method for predicting membrane fouling of an anaerobic membrane bioreactor and identifying key influencing factors. Background Art

[0002] Anaerobic membrane bioreactors (AnMBRs) are primarily used in water treatment processes, including municipal sewage, industrial wastewater, and livestock wastewater. AnMBRs have attracted considerable attention due to their compact footprint, high effluent quality, low energy intensity, and high energy recovery rates, making them a promising technology for wastewater treatment. However, membrane fouling is an inevitable challenge during AnMBR operation, hindering the further promotion and application of AnMBR technology.

[0003] The main causes of membrane fouling in anaerobic membrane bioreactors include concentration polarization, membrane pore blockage, and surface deposition. The membrane fouling process is very complex and is affected by multiple variables such as biomass characteristics, membrane component characteristics, and operating parameters. It has the characteristics of complex nonlinear influencing factors and strong coupling between variables. In the past few decades, researchers have successively constructed classic mathematical models to analyze the relationship between biomass characteristics and operating parameters and membrane fouling. However, the application of traditional mathematical models is relatively simple, so their versatility is poor. Moreover, mathematical models usually require many prior assumptions during the simulation process, making it difficult to analyze the nonlinear relationship between complex factors. There are also problems such as high numerical conditions and poor simulation effects.

[0004] The development of artificial intelligence algorithms has made membrane fouling prediction possible. Random forest is a combined classifier technology composed of multiple decision trees. Compared with single classifiers such as decision trees, it has better prediction performance and better processing effect on high-dimensional data. It can effectively mine hidden patterns from large amounts of nonlinear data and make classification or regression predictions. It has certain potential in complex nonlinear problems such as membrane fouling prediction and identification of key influencing factors. Summary of the Invention

[0005] In response to the above-mentioned problems existing in the prior art, the present invention provides a method for identifying key influencing factors of membrane fouling in anaerobic membrane bioreactors. By inputting the membrane fouling-related data set during the operation of the anaerobic membrane bioreactor into the model, membrane fouling prediction and identification and analysis of key influencing factors of membrane fouling can be completed. The present invention can guide anaerobic membrane bioreactors to carry out targeted control of membrane fouling.

[0006] The technical solutions of the present invention are as follows:

[0007] A method for identifying key influencing factors of membrane fouling in anaerobic membrane bioreactors comprises the following steps:

[0008] S1. Collect and screen raw data; the raw data include but are not limited to: operating parameters of the anaerobic membrane bioreactor, biomass properties, membrane module characteristics, and membrane fouling rate;

[0009] S2. Using the membrane fouling rate during the operation of the anaerobic membrane bioreactor as the output vector, and the membrane module characteristics, operating parameters, and biomass characteristics as the input vectors, to construct a data set;

[0010] S3. Build a random forest model based on decision trees;

[0011] S4. Output target features based on the input features of the optimal random forest model to obtain a membrane fouling prediction value;

[0012] S5. Based on the relative importance value of each input vector output by the optimal random forest model, the importance index of each membrane fouling influencing factor is ranked, and the key influencing factors of membrane fouling are identified in the order of feature importance.

[0013] Furthermore, the raw data of step S1 include 14 characteristic parameters: membrane pore size, membrane effective area, membrane flux, organic load, biogas aeration rate, hydraulic retention time, volatile solids concentration, soluble microbial product protein concentration, soluble microbial product polysaccharide concentration, soluble microbial product protein to polysaccharide ratio, extracellular polymer protein concentration, extracellular polymer polysaccharide concentration, extracellular polymer protein to polysaccharide ratio, and influent COD concentration;

[0014] The above 14 feature parameters are set as the model input vector.

[0015] Furthermore, step S3 includes the following steps:

[0016] S3-1. Divide the dataset into a test set and a prediction set, and input them into the RandomForestRegressor model;

[0017] S3-2. Perform hyperparameter optimization and dataset training on the RandomForestRegressor model. Hyperparameters include but are not limited to: number of trees, number of features, maximum depth of each decision tree, minimum number of samples required for internal node repartitioning, and minimum number of samples for leaf nodes. Establish input and output matrices for the test set and prediction set, respectively. Determine the optimal parameters using a grid search method or a combination of manual parameter adjustment to complete the construction of the RandomForestRegressor model.

[0018] S3-3. Use the test set data to test the accuracy of the model, using the mean square error (MSE) and goodness of fit (R) 2Evaluate the model accuracy, the calculation formula is as follows:

[0019]

[0020]

[0021] Where n is the number of samples, is the predicted value of membrane contamination of the i-th sample, y i is the actual value of membrane contamination of sample i, The actual sample mean.

[0022] Furthermore, in step S3-1, the data set is randomly divided into a test set and a prediction set in a ratio of 80% and 20%.

[0023] Furthermore, when dividing the data set in step S3-1, if there are some missing values ​​in the data set, the missing values ​​are first interpolated and then the data set is normalized.

[0024] Furthermore, any one of the following methods, regression interpolation, hot-card filling, median replacement, and random forest interpolation, is used to interpolate the missing values ​​of the dataset.

[0025] Furthermore, the number of trees ranges from 200 to 1300 with a step size of 100;

[0026] The value range of the feature number is 1 to 14, with a step size of 1;

[0027] The maximum depth of each decision tree is none;

[0028] The minimum number of samples required for internal node repartitioning is 2;

[0029] The minimum number of samples in a leaf node is 1.

[0030] Furthermore, step S3-3 uses the mean square error (MSE) and goodness of fit (R) 2 The method for evaluating the accuracy of the model is as follows:

[0031] If R 2 If the value of the hyperparameters is ≥0.85 and the MSE is ≤0.2, the model is considered to be reasonable and effective. Otherwise, the hyperparameters of the model are readjusted and step S3-2 is repeated.

[0032] Furthermore, the random forest model training process in step S3 is carried out using the Python programming language and is built based on Matplotlib, Numpy, Sklearn, and Pandas modules.

[0033] Furthermore, step S5 calculates the feature importance of each input vector by using the average impurity reduction method.

[0034] The beneficial technical effects of the present invention are:

[0035] (1) It overcomes the problems of multiple assumptions, poor simulation results and poor versatility in the membrane fouling prediction process of traditional mathematical models, and provides a new method for predicting and analyzing membrane fouling in anaerobic membrane bioreactors;

[0036] (2) From the perspective of algorithm application, a method for predicting membrane fouling of anaerobic membrane bioreactors and identifying key factors affecting membrane fouling was proposed, enriching the theory of membrane fouling analysis and control of anaerobic membrane bioreactors;

[0037] (3) The random forest algorithm was introduced to model the membrane pollution, which enabled the classification analysis of the key factors affecting membrane pollution and intuitively displayed the degree of influence of various factors on membrane pollution. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 Schematic diagram of the method flow of the present invention;

[0039] Figure 2 This is a graph showing membrane fouling prediction results using the random forest algorithm of an embodiment;

[0040] Figure 3 The following is a diagram showing the order of importance of the key factors affecting membrane fouling in the embodiment. DETAILED DESCRIPTION

[0041] The present invention is described in detail below with reference to the accompanying drawings and embodiments. It is apparent that the embodiments described are only a portion of the embodiments of the present invention, rather than all of them. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without inventive effort are intended to fall within the scope of protection of the present invention.

[0042] The steps of the embodiment are as follows Figure 1 As shown:

[0043] S1. Extract data directly from texts and tables in published literature, and use Plot Digitizer 2.6.8 software to extract relevant data from literature illustrations to obtain original data.

[0044] S2. The membrane fouling rate during the operation of the anaerobic membrane bioreactor is used as the input vector, and the membrane component characteristics, operating parameters and biomass characteristics are used as the output vector to construct a data set.

[0045] The independent variables of the data set include 14 eigenvectors: membrane pore size, membrane effective area, membrane flux, organic load, biogas aeration rate, hydraulic retention time, volatile solids concentration, soluble microbial product protein concentration (SMP-protein), soluble microbial product polysaccharide concentration (SMP-polysaccharide), soluble microbial product protein to polysaccharide concentration ratio (SMPp / SMPc), extracellular polymeric substance protein concentration (EPS-protein), extracellular polymeric substance polysaccharide concentration (EPS-polysaccharide), extracellular polymeric substance protein to polysaccharide concentration ratio (EPSp / EPSc) and influent chemical oxygen demand concentration, which are set as model input vectors.

[0046] The dependent variable of the data set is the membrane fouling rate (kPa / d), which is set as the model output vector.

[0047] S3. Construct a random forest model based on a decision tree, including the following steps:

[0048] S3-1. Randomly select 80% of the data set generated in step S1 as the training set and the remaining 20% ​​of the data as the test set; if there are some missing values ​​in the data set, use the random forest algorithm to interpolate the missing values; and normalize the data set.

[0049] S3-2. Based on the Python programming language, build a random forest model through Matplotlib, Numpy, sklearn, and pandas modules.

[0050] The RandomForestRegressor model is hyperparameter optimized and trained on the dataset. The hyperparameters include: number of trees (n_trees), number of features (n_features), maximum depth of each decision tree (max_depth), minimum number of samples required for internal node re-splitting (min_samples_spilt), and minimum number of samples for leaf nodes (min_samples_leaf).

[0051] The input and output matrices of the test set and prediction set were established respectively. The optimal parameters were determined by using a grid search method or a combination of manual parameter adjustment to complete the construction of the RandomForestRegressor model. The number of trees ranged from 200 to 1300 with a step size of 100. The number of features ranged from 1 to 14 with a step size of 1. The maximum depth of each decision tree, the minimum number of samples required for internal node repartitioning, and the minimum number of samples for leaf nodes were set to None, 2, and 1, respectively.

[0052] S3-3. Use the test set data to test the accuracy of the model, using the mean square error (MSE) and goodness of fit (R) 2 Evaluate the model accuracy, the calculation formula is as follows:

[0053]

[0054]

[0055] Where n is the number of samples, is the predicted value of membrane contamination of the i-th sample, y i is the actual value of the membrane contamination of the i-th sample, is the actual sample average.

[0056] If R 2 If ≥0.85 and MsE≤0.2, the model is considered reasonable and effective. Otherwise, the hyperparameter values ​​of the model are readjusted and step S3-2 is repeated.

[0057] The parameters were adjusted by combining the grid search method with manual parameter adjustment. When n_trees was 1200, n_features was 14, max_depth was None, min_samples_spilt was 2, and min_samples_leaf was 1 in the RandomForestRegressor parameters, R2 ≥ 0.85 and MSE ≤ 0.2, a random forest model that met expectations was obtained.

[0058] S4. Output the target features based on the input features of the optimal random forest model to obtain the membrane fouling prediction value. Figure 2 As shown in the figure, the R2 of the membrane fouling prediction value based on the random forest algorithm model and the actual value is 0.9016, and the MSE is 0.064, indicating that the random forest model has excellent prediction effect.

[0059] S5. Based on the relative importance value of each input vector output by the optimal random forest model, the importance index of each membrane fouling influencing factor is ranked, and the key factors affecting membrane fouling are identified in the order of feature importance. Figure 3 As shown in the figure, the importance of the features related to membrane fouling rate, from high to low, is as follows: SMPp / SMPc, EPSp / EPSc, organic load, biogas circulation injection rate, membrane flux, SMP-protein, EPS-polysaccharide, volatile suspended solids, EPS-protein, hydraulic retention time, membrane effective area, SMP-polysaccharide, membrane pore size, and influent chemical oxygen demand concentration. Variable importance analysis shows that SMPp / SMPs is the most significant factor affecting membrane fouling rate, with an importance value of 0.294.

[0060] Although the embodiments of the present invention have been disclosed as above, they are not limited to the applications listed in the description and implementation methods. They can be fully applied to various fields suitable for the present invention. For those familiar with the art, for those of ordinary skill in the art, various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of the present invention. Therefore, without departing from the general concept defined by the claims and the scope of equivalents, the present invention is not limited to specific details.

Claims

1. A method for identifying key influencing factors of membrane fouling in anaerobic membrane bioreactors, characterized in that: The following steps are involved: S1. Collect and screen raw data; the raw data include but are not limited to: operating parameters of the anaerobic membrane bioreactor, biomass properties, membrane module characteristics, and membrane fouling rate; The original data include 14 characteristic parameters: membrane pore size, membrane effective area, membrane flux, organic load, biogas aeration rate, hydraulic retention time, volatile solids concentration, soluble microbial product protein concentration, soluble microbial product polysaccharide concentration, soluble microbial product protein to polysaccharide ratio, extracellular polymer protein concentration, extracellular polymer polysaccharide concentration, extracellular polymer protein to polysaccharide ratio, influent COD concentration; The above 14 feature parameters are set as the model input vector; S2. Using the membrane fouling rate during the operation of the anaerobic membrane bioreactor as the output vector, and the membrane module characteristics, operating parameters, and biomass characteristics as the input vectors, to construct a data set; S3. Construct a random forest model based on a decision tree; the following steps are included: S3-1. Divide the dataset into a test set and a prediction set, and input them into the RandomForestRegressor model; S3-2. Perform hyperparameter optimization and dataset training on the RandomForestRegressor model. Hyperparameters include but are not limited to: number of trees, number of features, maximum depth of each decision tree, minimum number of samples required for internal node repartitioning, and minimum number of samples for leaf nodes. Establish input and output matrices for the test set and prediction set, respectively. Determine the optimal parameters using a grid search method or a combination of manual parameter adjustment to complete the construction of the RandomForestRegressor model. S3-3. Use the test set data to test the accuracy of the model, using the mean square error (MSE) and goodness of fit (R) 2 Evaluate the model accuracy, the calculation formula is as follows: Where n is the number of samples, is the predicted value of membrane contamination of the i-th sample, y i is the actual value of membrane contamination of sample i, Actual sample mean; S4. Output target features based on the input features of the optimal random forest model to obtain a membrane fouling prediction value; S5. Based on the relative importance value of each input vector output by the optimal random forest model, the importance index of each membrane fouling influencing factor is ranked, and the key influencing factors of membrane fouling are identified in the order of feature importance.

2. The method for identifying key factors affecting membrane fouling in anaerobic membrane bioreactors according to claim 1, characterized in that: In step S3-1, the data set is randomly divided into a test set and a prediction set at a ratio of 80% and 20%.

3. The method for identifying key factors affecting membrane fouling in anaerobic membrane bioreactors according to claim 1, characterized in that: When dividing the data set in step S3-1, if there are some missing values ​​in the data set, the missing values ​​are first interpolated and then the data set is normalized.

4. The method for identifying key factors affecting membrane fouling in anaerobic membrane bioreactors according to claim 3, characterized in that: Use any of the following methods to interpolate missing values ​​in the dataset: regression interpolation, hot-card imputation, median replacement, or random forest interpolation.

5. The method for identifying key factors affecting membrane fouling in anaerobic membrane bioreactors according to claim 1, characterized in that: The value range of the number of trees is 200 to 1300, with a step size of 100; The value range of the feature number is 1 to 14, with a step size of 1; The maximum depth of each decision tree is none; The minimum number of samples required for internal node repartitioning is 2; The minimum number of samples in a leaf node is 1.

6. The method for identifying key factors affecting membrane fouling in anaerobic membrane bioreactors according to claim 1, characterized in that: Step S3-3 uses mean square error (MSE) and goodness of fit (R) 2 The method for evaluating the accuracy of the model is as follows: If R 2 If the value of the hyperparameters is ≥0.85 and the MSE is ≤0.2, the model is considered to be reasonable and effective. Otherwise, the hyperparameters of the model are readjusted and step S3-2 is repeated.

7. The method for identifying key factors affecting membrane fouling in anaerobic membrane bioreactors according to claim 1, characterized in that: The random forest model training process described in step S3 is based on the Python programming language and is built on Matplotlib, Numpy, Sklearn, and Pandas modules.

8. The method for identifying key factors affecting membrane fouling in anaerobic membrane bioreactors according to claim 1, characterized in that: Step S5 calculates the feature importance of each input vector by using the average impurity reduction method.

Images