A method for predicting COD in leachate from a waste transfer station
By constructing an RF-XGBoost model and combining it with an adaptive feedback optimization mechanism, the problems of accuracy and adaptability in leachate COD prediction were solved, achieving efficient leachate COD prediction, optimizing wastewater treatment process parameters, and reducing energy consumption and resource waste.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- HANGZHOU DIANZI UNIV
- Filing Date
- 2024-11-21
- Publication Date
- 2026-05-26
Smart Images
Figure CN119397251B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of leachate treatment, and more particularly to a method for predicting COD in leachate from waste transfer stations. Background Technology
[0002] Leachate from waste transfer stations is a highly polluted and concentrated wastewater, typically containing large amounts of organic pollutants, heavy metals, ammonia nitrogen, and other contaminants. It is characterized by complex water quality, high pollutant concentrations, and significant treatment challenges. Its complex composition includes organic matter, inorganic matter, and various toxic substances, posing a significant threat to the aquatic environment and human health. Chemical oxygen demand (COD), a measure of the concentration of organic matter and inorganic oxides in water, reflects the degree of pollution by reducing substances and is a crucial indicator for water quality assessment. Organic pollution of water bodies is widespread, thus COD is often used as one of the indicators reflecting the degree of organic pollution. However, the correlation between various operating parameters of wastewater treatment systems and effluent water quality indicators often exhibits non-linear and lag characteristics. Therefore, when wastewater treatment systems are impacted by changes in influent water quality and quantity, methods such as increasing aeration intensity and adding chemical agents are typically employed to ensure effluent quality. However, high-intensity aeration and chemical use significantly increase the energy consumption and operating costs of wastewater treatment plants, leading to secondary pollution and waste.
[0003] To address this issue, developing effluent quality prediction methods applicable to wastewater treatment systems is crucial. Such methods help operators understand water quality trends in advance, enabling more scientific and economical decision-making to reduce energy consumption and operating costs while ensuring effluent quality. Predictive models allow for real-time adjustments to wastewater treatment processes, ensuring treatment effectiveness while also addressing water resource management and energy conservation / emission reduction needs. For example, neural network models based on an encoder-decoder framework can predict effluent quality from wastewater treatment plants. By learning from historical data, these models can reproduce the dynamic process of water quality changes, providing a foundation for process optimization and model predictive control. Furthermore, employing a multi-indicator input strategy to predict multi-indicator output, incorporating environmental factors affecting the wastewater treatment process, allows the model to be applied in complex environments, making it practically significant.
[0004] However, current prediction methods still have some shortcomings. For example, landfill leachate has a complex composition and fluctuates significantly in pollutant concentrations. Single traditional prediction models struggle to fully capture this complex relationship, resulting in less than ideal prediction accuracy. Traditional prediction methods rely on large amounts of historical data, but due to the diverse composition of leachate and significant differences in composition and treatment difficulty across different regions and seasons, historical data lacks representativeness. This reliance on data reduces the model's applicability and accuracy in new environments. During leachate treatment, external environmental factors (such as temperature, pH, and pollutant types) have a significant impact on treatment effectiveness. Existing prediction methods often struggle to effectively incorporate these external factors, failing to achieve comprehensive prediction of multiple variables, leading to decreased accuracy under extreme conditions. Some prediction methods (such as deep learning models based on large datasets) have high computational resource requirements, with long data processing and model training times, making real-time prediction difficult. This becomes lagging when leachate treatment requires rapid response and adjustment, affecting actual wastewater treatment results. Many prediction methods perform well on specific datasets but lack adaptability to different datasets or environments. Especially when the composition of landfill leachate varies greatly and the sample size is limited, existing models are prone to overfitting and insufficient generalization ability, leading to a significant decline in predictive performance under new data or different conditions. Current methods often focus on predicting wastewater treatment effectiveness while neglecting the impact of specific process parameters on the prediction results. Different treatment processes (such as membrane processes and biological processes) have varying removal effects on different types of pollutants in leachate. There is a lack of comprehensive prediction methods that combine predictive models with actual process parameters, making it difficult to provide effective references for process selection and parameter optimization. Leachate contains numerous types of pollutants, and some traditional linear regression or simple time series models struggle to capture the relationships between these pollutants, resulting in limited prediction accuracy.
[0005] In summary, there is an urgent need for a new and efficient method for predicting COD in leachate from waste transfer stations, in order to improve the efficiency and effectiveness of wastewater treatment and reduce environmental pollution and resource waste. Summary of the Invention
[0006] The purpose of this invention is to address the problems existing in the background art by proposing a method for predicting COD in leachate from waste transfer stations, the steps of which are as follows:
[0007] S1. Collect real-time and historical data of leachate from the waste transfer station using sensors;
[0008] S2. Preprocess the collected real-time and historical data;
[0009] S3. Based on the composition characteristics of leachate, feature analysis is performed, and features are further screened and optimized through feature engineering. Feature variables that are strongly correlated with effluent COD are selected to construct a COD dataset to improve the prediction accuracy of the model.
[0010] S4. A COD prediction model RF-XGBoost is constructed using a nonlinear fitting method, and the prediction model is trained and optimized using preprocessed historical data.
[0011] S5. Input the preprocessed real-time data into the trained prediction model RF-XGBoost, and output the COD prediction value in real time; adjust the model parameters automatically according to the difference between the prediction results and the actual measurement data to achieve adaptive feedback optimization of the model.
[0012] Preferably, the preprocessing in step S2 includes standardization, removal of outlier data, imputation of missing values, and data smoothing, the specific steps of which are as follows:
[0013] S21. Construct a dataset from the real-time and historical data collected in step S1, and then perform standardization processing. Use formula (1) to scale the data to the range of [0,1].
[0014] (1)
[0015] In equation (1), x is the original data. It is normalized data. and These are the minimum and maximum values of the dataset, respectively. max and min are the limits of the scaling range, which are 0 and 1 in this case.
[0016] S22. Detect outliers based on the interquartile range (IQR), calculate the first quartile (Q1) and third quartile (Q3) of the data, and calculate the IQR (...). For less than or greater than The value can be considered an outlier and should be removed.
[0017] S23. Fill missing values with the previous or next known value and smooth the data using the moving average method formula (2);
[0018] (2)
[0019] In formula (2) The data points are smoothed, representing the average value within that window. The window size is the average number of consecutive data points. For each data point in the original data, it represents a value within the window;
[0020] Preferably, the specific steps of step S3 are as follows:
[0021] S31. Using the Pearson correlation coefficient as a reference, evaluate the correlation between each input characteristic variable and the effluent COD concentration. Calculate the corresponding values according to formula (3) and sort and select the characteristic variables.
[0022] (3)
[0023] In formula (3) The number of data samples, For the first One sample in The value of the variable, For the first One sample in The value of the variable, and These are the corresponding average values;
[0024] S32. Construct a COD dataset using the filtered feature variables.
[0025] Preferably, the selected characteristic variables include influent pH, influent COD concentration, influent C / N (carbon / nitrogen) ratio, influent C / P (carbon / phosphorus) ratio, and DO (dissolved oxygen).
[0026] Preferably, the specific steps of step S4 are as follows:
[0027] S41. Using a nonlinear fitting method, construct a random forest (RF) model and an XGBoost machine learning model;
[0028] S42. Perform cross-validation on the dataset obtained in S32, dividing it into k subsets. Each time, use one subset as the validation set and the remaining k-1 subsets as the training set, repeating this process k times, selecting a different subset as the validation set each time. Set hyperparameters, train each randomly selected hyperparameter combination, evaluate its performance using cross-validation, select the best-performing hyperparameter combination based on the cross-validation results, use the predictions of the RF model and XGBoost model as features, construct a new meta-dataset, select linear regression as the meta-learner, train using the meta-dataset, and then continuously optimize to obtain the RF-XGBoost model.
[0029] S43. Calculate the corresponding determination coefficients for different models according to formula (4). ;
[0030] (4)
[0031] In formula (4) The number of data samples, This is the actual value of COD. This represents the true average value of COD. This is the predicted value for COD.
[0032] S44. Obtain the root mean square error (RMSE) according to formula (5) and evaluate the prediction accuracy of different machine learning models.
[0033] (5)
[0034] In formula (5) The number of data samples, This is the actual value of COD. This is the predicted value for COD.
[0035] Preferably, the specific steps of S5 are as follows: The collected real-time data is input into the RF-XGBoost machine learning model, and the COD prediction value is output in real time. Based on the difference between the prediction result and the actual measurement data, the SGDRegressor is used as an incremental learning model to automatically adjust the model parameters. When the prediction error is large during the continuous operation of the model, the hyperparameters are tuned by a random search method. The hyperparameter configuration of the model is updated according to the search results. After each model update, the error is monitored again to form a loop feedback mechanism.
[0036] Compared with the prior art, the present invention has the following beneficial technical effects:
[0037] This invention employs a nonlinear fitting method to construct a COD prediction model to adapt to complex pollutant relationships. Furthermore, it proposes real-time input of collected sensor data into the trained prediction model and automatic adjustment of model parameters based on prediction errors. Through an adaptive optimization feedback mechanism, the model constructed using this method can continuously learn, adjust, and optimize during operation, improving long-term prediction accuracy and achieving precise prediction of COD in leachate from waste transfer stations. This overcomes the problem of poor prediction results caused by the large number of pollutant types. Attached Figure Description
[0038] Figure 1 This is a flowchart of a method for predicting COD in leachate from a waste transfer station according to the present invention;
[0039] Figure 2 This is a schematic diagram of the RF model for predicting COD of leachate from waste transfer stations in this invention, where A represents the training set results and B represents the test set results.
[0040] Figure 3 This is a schematic diagram of the XGBoost model for predicting COD of leachate from waste transfer stations in this invention, where A represents the training set results and B represents the test set results.
[0041] Figure 4This is a schematic diagram of the RF-XGBoost model for predicting COD of leachate from waste transfer stations in this invention, where A represents the training set results and B represents the test set results. Detailed Implementation
[0042] The present invention will be further described below with reference to the accompanying drawings and embodiments.
[0043] like Figure 1 As shown, the present invention proposes a method for predicting COD in leachate from waste transfer stations, the steps of which are as follows:
[0044] I. Data Acquisition and Preprocessing
[0045] 1.1 Real-time and historical data of leachate are collected by sensors, including relevant water quality parameters of leachate.
[0046] 1.2. The Min-Max Scaling feature standardization function is introduced to normalize the sample data, preventing large-scale attributes from dominating due to significant differences in the magnitude of the data, and simultaneously improving the model's convergence speed. Its calculation formula is as follows:
[0047]
[0048] Where x is the original data. It is normalized data. and These are the minimum and maximum values of the dataset, respectively. min and max are the limits of the scaling range, which are 0 and 1 in this case.
[0049] 1.3. Outlier detection using interquartile range (IQR): Calculate the first quartile (Q1) and third quartile (Q3) of the data, and then calculate the IQR (...). For less than or greater than The values can be considered outliers. Outliers are removed, and missing values are filled with the previous or next known value. The data is then smoothed using a moving average method, the calculation formula of which is as follows:
[0050]
[0051] in The data points are smoothed, representing the average value within that window. The window size is the average number of consecutive data points. For each data point in the original data, represent each value within the window.
[0052] II. Feature Selection
[0053] 2.1 Collect water quality parameters during the leachate treatment process of the waste transfer station, including influent pH value, influent COD concentration, influent NH3-N concentration, influent TP concentration, influent C / N ratio, influent C / P ratio, HRT (Hydraulic Retention Time), DO, and the corresponding effluent COD concentration.
[0054] 2.2 The Pearson correlation coefficient (PCC) is then used as a reference to assess the correlation between each input characteristic variable and the COD of the leachate from the waste transfer station. The calculation formula is as follows:
[0055]
[0056] in The number of data samples, For the first Each input feature For the first COD of leachate from a waste transfer station and These are the corresponding average values.
[0057] 2.3. By calculating the influent pH value, influent COD concentration, influent C / N ratio, influent C / P ratio, and DO as strongly correlated features of effluent COD, a COD dataset was constructed.
[0058] III. Establishing Machine Learning Models
[0059] 3.1. A nonlinear fitting method is adopted to adapt to complex pollutant relationships. Random Forest (RF) and XGBoost are used to construct a COD prediction model to improve the accuracy of prediction.
[0060] 3.2. Perform cross-validation on the dataset obtained in 2.3, dividing it into k subsets. Each time, use one subset as the validation set and the remaining k-1 subsets as the training set. Repeat this process k times, selecting a different subset as the validation set each time. Average the results of the k validations to obtain the final evaluation metric of the model.
[0061] 3.3 Define a range or distribution for each hyperparameter, such as learning rate, number of trees (n_estimators), and maximum depth (max_depth). Set the number of searches. Train each randomly selected hyperparameter combination and evaluate its performance using cross-validation. Based on the cross-validation results, select the best-performing hyperparameter combination. Use the predictions from RF and XGBoost as features to construct a new meta-dataset. Select linear regression as the meta-learner, train it using the meta-dataset, and then continuously optimize it to obtain the RF-XGBoost machine learning model.
[0062] IV. Model Performance Evaluation
[0063] Selecting the coefficient of determination As an evaluation criterion, its calculation formula is as follows:
[0064]
[0065] in The number of data samples, This is the actual value of COD. This represents the true average value of COD. This is the predicted value for COD.
[0066] RMSE is used as a performance metric for evaluation. A smaller RMSE value indicates a smaller error between the model's predictions and experimental values, suggesting a stronger predictive ability. The calculation formula is as follows:
[0067]
[0068] in The number of data samples, This is the actual value of COD. This is the predicted value for COD.
[0069] To further verify the model accuracy, three models—RF, XGBoost, and RF-XGBoost—were used for prediction, and the results are as follows: Figure 2-4 As shown in Table 1, the performance evaluation results of the three models are as follows:
[0070] Table 1 Evaluation results of different models
[0071]
[0072] By comparing the RF model, XGBoost model, and RF-XGBoost model and Among them, the RF-XGBoost model has the best prediction performance.
[0073] V. Model Adaptive Feedback Optimization
[0074] 5.1 Each time the model makes a prediction, it is necessary to calculate the difference between the predicted value and the actual measured value. The RMSE error metric is used to evaluate the prediction quality of the model and determine whether the model needs to be adjusted.
[0075] 5.2 Define an error threshold. When the RMSE exceeds the preset threshold, start SGDRegressor as an incremental learning model to automatically adjust the model parameters.
[0076] 5.2 After each model update, error monitoring restarts to form a cyclical feedback mechanism.
[0077] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited thereto. Various changes can be made within the scope of knowledge possessed by those skilled in the art without departing from the spirit of the present invention.
Claims
1. A method for predicting COD in leachate from a waste transfer station, characterized in that, The method includes the following steps: S1. Collect real-time and historical data of leachate from the waste transfer station using sensors; S2. Preprocess the collected real-time and historical data; S3. Based on the composition characteristics of leachate, feature analysis is performed, and features are further screened and optimized through feature engineering. Feature variables that are strongly correlated with effluent COD in historical data are selected to construct a COD dataset. S4. A COD prediction model, RF-XGBoost, is constructed using a nonlinear fitting method. The model is then trained and optimized using preprocessed historical data. Specifically: S41. Use nonlinear fitting methods to construct RF models and XGBoost models; S42. Cross-validate the dataset obtained in S32, dividing it into k subsets. Each time, use one subset as the validation set and the remaining k-1 subsets as the training set. Repeat this process k times, selecting a different subset as the validation set each time. Train each randomly selected hyperparameter combination and evaluate its performance using cross-validation. Based on the cross-validation results, select the best-performing hyperparameter combination. Use the predictions of the RF model and the XGBoost model as features to construct a new meta-dataset. Select linear regression as the meta-learner and train and optimize using the meta-dataset to obtain the RF-XGBoost model. S5. Input the preprocessed real-time data into the trained prediction model RF-XGBoost and output the COD prediction value; automatically adjust the model parameters according to the difference between the prediction result and the actual measurement data; specifically: input the preprocessed real-time data into the RF-XGBoost model and output the COD prediction value in real time. According to the difference between the prediction result and the actual measurement data, use SGDRegressor as an incremental learning model to automatically adjust the model parameters. When the prediction error is large during continuous operation, the hyperparameters are tuned using a random search method. The hyperparameter configuration of the model is updated according to the search results. After each model update, the error is monitored again to form a loop feedback mechanism.
2. The method for predicting COD in leachate from waste transfer stations according to claim 1, characterized in that, The preprocessing described in step S2 includes the following steps: S21. Scale the data features to the range [0,1] using formula (1); (1) In equation (1), x is the original data. It is normalized data. and These are the minimum and maximum values of the dataset, respectively. S22. Detect outliers based on the interquartile range (IQR). Calculate the first quartile (Q1) and third quartile (Q3) of the data, and subtract Q1 from Q3 to obtain the IQR; [The remaining text appears to be incomplete and requires further context.] or greater than The data was identified as outliers and removed; S23. Fill missing values with the previous or next known value and smooth the data using the moving average method formula (2); (2) In formula (2) The data points are smoothed, representing the average value within that window. The window size is the average number of consecutive data points. For each data point in the original data, represent each value within the window.
3. The method for predicting COD in leachate from waste transfer stations according to claim 1, characterized in that, Step S3 is as follows: S31. Use the Pearson correlation coefficient as a reference to evaluate the correlation between each input feature variable and COD. Calculate the corresponding values according to formula (3) and sort and select the feature variables. (3) In formula (3) The number of data samples, For the first Each input feature For the first COD of leachate from a waste transfer station and These are the corresponding average values; S32. Construct a COD dataset using the filtered feature variables.
4. The method for predicting COD in leachate from waste transfer stations according to claim 3, characterized in that, The selected characteristic variables include influent pH, influent COD concentration, influent C / N ratio, influent C / P ratio, and DO.
5. A COD prediction system for leachate from a waste transfer station that implements the method as described in any one of claims 1-4, characterized in that, include: The data acquisition module is used to obtain relevant water quality parameters of leachate from waste transfer stations; The data preprocessing module is used to preprocess the relevant water quality parameters of leachate from waste transfer stations, including outlier removal, missing value filling, and standardization. The feature selection module is used to perform feature analysis on the relevant water quality parameters of leachate from the waste transfer station and select feature variables that are strongly correlated with the COD value. The COD prediction module uses a well-trained, validated, and tested RF-XGBoost model to predict COD. The feedback optimization module automatically adjusts the model parameters based on the difference between the predicted and actual values.
6. An electronic device, characterized in that, It includes a processor and a memory, the memory storing machine-executable instructions that can be executed by the processor, the processor executing the machine-executable instructions to implement the method of any one of claims 1-4.
7. A machine-readable storage medium, characterized in that, The machine-readable storage medium stores machine-executable instructions that, when invoked and executed by a processor, cause the processor to implement the method of any one of claims 1-4.