Water plant coagulation dosage prediction method based on RF-BiLSTM
By using feature selection and data processing methods based on random forest and BiLSTM, the problems of high model complexity and low prediction accuracy in traditional coagulation dosing control methods are solved, and more efficient prediction of coagulation dosing and control of effluent water quality are achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-17
- Publication Date
- 2026-03-24
AI Technical Summary
Traditional coagulation dosing control methods rely on human experience or simple mathematical models, which are difficult to cope with fluctuations in raw water quality and water volume, resulting in inaccurate dosing, waste of chemicals, or substandard effluent quality. Existing neural network models ignore the influence of factors such as water volume, water temperature, and weather, and lack effective feature selection methods, resulting in high model complexity, low training efficiency, and limited prediction accuracy.
A nonlinear relationship model between raw water parameters and chemical dosage was established using a feature selection method based on random forest and a bidirectional long short-term memory network (BiLSTM). Through data preprocessing, feature screening, and bidirectional learning of time series data, a more effective method for predicting coagulation chemical dosage was constructed.
It improves the prediction accuracy of coagulation dosage, reduces chemical waste, ensures that the effluent water quality meets the standards, and the model structure has a better ability to learn time series data, thus improving training efficiency and prediction effect.
Smart Images
Figure CN121723819A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application belongs to the field of tap water treatment, and particularly relates to a water plant coagulation dosage prediction method based on a Random Forest-Bidirectional Long Short-Term Memory (RF-BiLSTM). BACKGROUND
[0002] Coagulation dosing is a key link in the water treatment process of a water plant, and its effect directly affects the treatment effect of subsequent sedimentation, filtration and other processes and the final effluent water quality. The traditional coagulation dosing control method mainly relies on manual experience or simple mathematical models, and is difficult to cope with complex working conditions such as raw water quality fluctuations and water quantity changes, resulting in inaccurate dosing, causing problems such as reagent waste or substandard effluent water quality.
[0003] In recent years, with the rapid development of artificial intelligence technology, machine learning, deep learning and other methods have been widely applied in the field of water treatment. Among them, the Recurrent Neural Network (RNN) has shown great potential in coagulation dosing prediction and control due to its advantage in processing time series data. However, the traditional RNN has problems such as gradient disappearance and gradient explosion, making it difficult to learn long-term dependencies. The Long Short-Term Memory (LSTM) network, as a special RNN, effectively solves the defects of traditional RNN by introducing memory cells and gating mechanisms, and can better capture long-term dependencies in time series data.
[0004] At present, some researches have applied neural networks to coagulation dosing prediction and control in water plants, but there are still some problems. Most of the existing researches only consider raw water quality parameters (such as turbidity, pH value, etc.) as model inputs, ignoring the influence of water quantity, water temperature, weather and other factors on coagulation effect, resulting in limited model prediction accuracy; there is a lack of effective feature selection methods, making it difficult to select the key features that have the greatest impact on coagulation dosing from numerous influencing factors, resulting in high model complexity and low training efficiency; the model structure is simple, making it difficult to fully utilize the information in time series data. SUMMARY
[0005] The present application aims to overcome the shortcomings and deficiencies of the prior art and proposes a water plant coagulation dosage prediction method based on RF-BiLSTM. The present application consists of data preprocessing, a main feature selection method based on Random Forest (RF), and a model establishing a nonlinear relationship between raw water parameters and dosing based on BiLSTM. The present application adopts the following technical solutions to achieve the above-mentioned application purposes:
[0006] The RF-BiLSTM-based coagulant dosage prediction method for water plants of the application mainly includes three parts: data preprocessing, main feature selection method based on random forest, and nonlinear relationship model between raw water parameters and dosage established based on bidirectional long short-term memory (BiLSTM). For the processing of original data, the abnormal values are divided into short-time missing, long-time missing and noise, and solutions are respectively given. For the selection of main influencing features, the input features and target values present a complex nonlinear relationship, and the commonly used correlation coefficient method usually studies the monotonicity between variables, which is not suitable for raw water feature screening. The main feature selection method based on random forest establishes a random forest model between input features and target values, shuffles the input features, compares the decrease in prediction accuracy caused by shuffling the input features to find the few main features that have the greatest impact as the input of bidirectional long short-term memory, and solves the problem that traditional feature selection methods cannot be effectively applied to raw water feature screening. The modeling problem between raw water features and coagulant dosage flow is solved by using a bidirectional long short-term memory model, which can capture forward and reverse water quality data information at a specific time step, fully explore the internal relationship between the current input and target values, and thus improve the prediction effect of the target values.
[0007] Further, the data preprocessing method for time series data, that is, the preprocessing method for the original data, processes the abnormal values by dividing them into short-time missing, long-time missing and noise, and processes them by using corresponding solutions respectively.
[0008] Further, the data preprocessing method selects the input features by calculating the importance score of each feature through a random forest model and selecting a number of features with the highest importance score as main features; the main features include but are not limited to the pH value, turbidity, temperature, conductivity and dissolved oxygen concentration of raw water.
[0009] Further, the preprocessing method for time series data includes the following steps:
[0010] For short-time missing data, linear interpolation is used for filling;
[0011] For long-time missing data, historical data is used for filling;
[0012] For noise data, a filtering algorithm is used for smoothing.
[0013] Further, the main input feature selection includes the following steps:
[0014] Step 1, for each decision tree, select the corresponding out-of-bag data and calculate the out-of-bag data error;
[0015] Step 2, for all features in the out-of-bag data , in turn, shuffle, each time only shuffle one feature (randomly change the value of the sample at the feature , and calculate the out-of-bag error At this time, under the current decision tree i, the importance score of feature j is calculated as follows:
[0016]
[0017] Step 3, weighted sum of the importance of all features. Assuming that the random forest model has T decision trees, according to steps 1 and 2, all the importance of n features, the importance score of feature The weighted sum formula is as follows:
[0018]
[0019] Thus, the importance score of n features is obtained
[0020] Step 4, normalize the importance score of all features;
[0021] Step 5, select the features with higher importance scores as the input of the neural network model, and continue to use the coagulant dosage flow as the target value.
[0022] Further, the main feature selection method based on random forest is: modeling the nonlinear relationship between raw water features and dosage flow; the bidirectional long short-term memory network model simultaneously considers forward and reverse time series data, and captures long-term dependence in time series; the input of the BiLSTM model is the main features after preprocessing and feature selection, and the output is the predicted value of the coagulant dosage.
[0023] Further, the nonlinear relationship modeling includes the following steps:
[0024] Step 1, divide the training set, test set and validation set for the new data set after feature selection, and do normalization;
[0025] Step 2, train the bidirectional long short-term memory network model; the structure of the bidirectional long short-term memory network unit is composed of two layers of opposite LSTM; the working principle of LSTM is expressed as follows:
[0026] Forget gate:
[0027]
[0028] Input gate:
[0029]
[0030]
[0031]
[0032] Output gate:
[0033]
[0034]
[0035] wherein, respectively represent the forget gate, the input gate and the output gate; and are respectively the weight and the bias vector of the forget gate; and are respectively the weight and the bias vector of the input gate; and are respectively the weight and the bias vector of the output gate; represents the current time state; represents the input raw water feature; represents the final output result of the raw water parameter after being processed by the gating device; represents the updated memory unit; represents the candidate memory unit; and are respectively the weight and the bias vector of the candidate memory unit; represents the activation function; represents the hyperbolic tangent activation function;
[0036] Step 2-1, set the hyperparameters of the bidirectional long short-term memory network model, including learning rate, hidden layer dimension, training period, input sequence length, batch size, etc.
[0037] Step 2-2, train the bidirectional long short-term memory network model, observe the changes of training error and validation error in each round, if both converge, output the model, otherwise go back to step 2-1, adjust the hyperparameters; here the loss function formula is:
[0038]
[0039] wherein, represents the number of samples, represents the true value of the i-th sample, represents the predicted value of the i-th sample.
[0040] Step 2-3, the test set is input into the trained bidirectional long short-term memory network model for prediction, and the prediction effect of the dosing flow is observed, and thus the RF-BiLSTM-based coagulant dosage prediction of the water plant is completed.
[0041] Compared with the prior art, the significant advantages brought by the present application include:
[0042] (1) The processing of time series data in the present application proposes corresponding solutions for different types of abnormal values and spans, making the restored data more reliable.
[0043] (2) The present application uses random forest for feature selection, which is more suitable for selecting the main influencing factors of the dosage than the Pearson correlation coefficient method and the mutual information method commonly used in current research, and the results obtained are more in line with the actual situation.
[0044] (3) The present application selects bidirectional long short-term memory network for the construction of the nonlinear relationship model between raw water parameters and dosage, which can learn time series from both forward and reverse directions, combining historical information and future information of the sequence. This greatly increases the model structure and enables more complete learning of the information implied in the time series data. BRIEF DESCRIPTION OF DRAWINGS
[0045] Figure 1 is an application background diagram of the present application;
[0046] Figure 2 is a general flowchart of the RF-BiLSTM-based coagulant dosage prediction method of the water plant;
[0047] Figure 3 is a feature selection algorithm flowchart based on RF;
[0048] Figure 4 is a comparison chart of the predicted value and the actual value of the dosing flow of the RF-BiLSTM-based coagulant dosage prediction method of the water plant. DETAILED DESCRIPTION
[0049] The present application will be further described below in conjunction with the drawings and examples. The following examples are used to illustrate the present application, but are not used to limit the scope of the present application.
[0050] This example takes the water treatment process flowchart as shown in Figure 1 , for example, uses the RF-BiLSTM-based coagulant dosage prediction method of the water plant to predict the coagulant dosage in the coagulation process according to the conditions of raw water. The general flowchart of the RF-BiLSTM-based coagulant dosage prediction method of the water plant is shown in Figure 2 .
[0051] Step 1: Obtain the raw dataset, select raw water data as the feature input, and coagulant dosing flow rate as the target value.
[0052] Step 2, data preprocessing, including handling missing values and data noise reduction.
[0053] Step 2-1: Handling missing values in the data. Each raw water data point is measured by its corresponding sensor. Data loss due to tank washing or sensor malfunctions can cause some raw water data to be missing. Two methods are used to fill in or delete the data: ① Historical data imputation: When the consecutive duration of missing data exceeds 5 days, the data loss time is too long, disrupting the data's trend and regularity, making it impossible to restore using interpolation or other methods. Therefore, historical data imputation is used. Data from the same historical period (such as data from the same day or week last year) is used for imputation. ② Linear interpolation: When the consecutive duration of missing data is ≤ 5 days, linear interpolation can preserve the original regularity and trend of the data as much as possible. The linear interpolation formula is...
[0054]
[0055] in, For missing values, , These are the nearest known values preceding and following the missing value, respectively. , and It is an index of the data.
[0056] Step 2-2: Processing noise in the data. When the sensor is working normally, the measured data will contain noise due to environmental interference, noise from the electronic components themselves, etc. A Hampell filter is used to reduce the noise in the data. The Hampell filter formula is:
[0057]
[0058] in Let i be the output signal at time i. Let i be the input signal at time i. For a sliding window of length L ,in , For window the median of For the set threshold, Where t is a constant.
[0059] Step 3: Based on the feature importance assessment of random forest, select the main factors affecting the target value. The feature selection algorithm based on random forest is as follows: Figure 3 As shown, the formula for calculating feature importance score is as follows:
[0060]
[0061] where T represents the number of decision trees in the random forest model, OOBt represents the out-of-bag data of the decision tree t, OOBt represents the out-of-bag data of the decision tree t, OOBt represents the out-of-bag data of the decision tree t, OOBt represents the out-of-bag data of the decision tree t, OOBt represents the out-of-bag data of the decision tree t, OOBt represents the out-of-bag data of the decision tree t, OOBt represents the out-of-bag data of the decision tree t.
[0062] Step 3-1, for each decision tree, select the corresponding out-of-bag data and calculate the out-of-bag data error. Bootstrap sampling (random sampling with replacement) is used to extract samples from the training set for training the decision tree. The extracted sample data is called in-bag (IB) data, and at this time, about 1 / 3 of the data is not extracted, which is called out-of-bag (OOB) data. Take decision tree i (assuming there are T decision trees) as an example. After training the decision tree with IB data, the OOB data is fed into the decision tree to calculate the prediction error of the decision tree .
[0063] Step 3-2, for all features in the out-of-bag data, perform shuffling in turn, each time only shuffling one feature (randomly changing the value of the sample at the feature ), and calculating the out-of-bag error again. At this time, under the current decision tree i, the importance score of feature j is calculated as follows:
[0064]
[0065] The error value before and after shuffling the bag data represents the feature importance score. The greater the error, the greater the influence of the shuffled feature on the target value.
[0066] Step 3-3, weighted sum of the importance of all features. Assuming that the random forest model has T decision trees, according to steps 3-1 and 3-2, we get T decision trees and the importance of all features n. The weighted sum formula of the importance of feature is as follows:
[0067]
[0068] Thus, the importance score of n features
[0069] Step 3-4, normalize the importance scores of all features. Scaling all feature importance scores to the [0, 1] interval can more intuitively see which features are the main influencing factors of the target value. The normalization formula is as follows:
[0070]
[0071] Step 3-5, select the features with high importance as the input of the neural network model, and continue to use the coagulant dosage flow as the target value.
[0072] Step 4, divide the training set, test set and validation set for the new data set after feature selection, and do normalization.
[0073] Step 4-1, the training set, validation set and test set are divided according to the ratio of 6:2:2.
[0074] Step 4-2, normalize the training set, and then use the normalizer of the training set to normalize the validation set and test set respectively.
[0075] Step 5, train the bidirectional long short-term memory network model. The structure of bidirectional long short-term memory network unit is composed of two layers of opposite LSTM. LSTM is based on RNN and adds control devices (input gate, forget gate and output gate). First, the forget gate determines the forgetting and retention of water quality data, then the input gate determines which new information enters the unit state, and finally the output gate determines the amount of information to be output in the current state. Its working principle is expressed as follows:
[0076] Forget gate:
[0077]
[0078] Input gate:
[0079]
[0080]
[0081]
[0082] Output gate:
[0083]
[0084]
[0085] where, , , represent the forget gate, input gate and output gate respectively; and respectively are the weight and bias vectors of the forget gate; and respectively are the weight and bias vectors of the input gate; and respectively are the weight and bias vectors of the output gate; represents the current time state; represents the input raw water characteristics; represents the final output result of the raw water parameters after being processed by the gating device; represents the update memory unit; represents the candidate memory unit; and respectively are the weight and bias vectors of the candidate memory unit; represents the activation function; represents the hyperbolic tangent activation function.
[0086] Step 5-1, set the hyperparameters of the bidirectional long short-term memory network model, including learning rate, hidden layer dimension, training period, input sequence length, batch size, etc.
[0087] Step 5-2, train the bidirectional long short-term memory network model, observe the changes of training error and validation error in each round, if both converge, output the model, otherwise go back to step 5-1, adjust the hyperparameters. The loss function formula here is:
[0088]
[0089] wherein, represents the number of samples, represents the true value of the i-th sample, represents the predicted value of the i-th sample.
[0090] Step 5-3, input the test set into the trained bidirectional long short-term memory network model for prediction, the predicted value and the actual value of the drug infusion flow are as shown in Figure 4 .
[0091] The above only describes the preferred embodiments of the present application, and the present application is not limited to the above embodiments. It can be understood that other improvements and changes directly derived or thought by those skilled in the art without departing from the spirit and concept of the present application should be considered to be included in the protection scope of the present application.
Claims
1. A method for predicting coagulation dosage in water treatment plants based on RF-BiLSTM, characterized in that, The method includes: Step 1, Data Preprocessing; Step 2: Principal feature selection based on Random Forest; Step 3: Establish a nonlinear relationship model between raw water parameters and dosage based on a bidirectional long short-term memory (BiLSTM) network.
2. The method for predicting coagulation dosage in water plants based on RF-BiLSTM according to claim 1, characterized in that, Step 1 is a preprocessing method for time series data. It involves handling outliers in the raw data, classifying outliers into short-term missing values, long-term missing values, and noise, and then using corresponding solutions to handle them.
3. The method for predicting coagulation dosage in water plants based on RF-BiLSTM according to claim 1, characterized in that, Step 2 involves filtering the input features by calculating the importance score of each feature using a random forest model and selecting the features with the highest importance scores as the main features. The main features include, but are not limited to, the pH value, turbidity, temperature, conductivity, and dissolved oxygen concentration of the raw water.
4. The method for predicting coagulation dosage in water plants based on RF-BiLSTM according to claim 2, characterized in that, The aforementioned preprocessing method for time series data includes the following steps: For data missing for a short period of time, linear interpolation is used to fill in the missing data. For data that has been missing for a long time, historical data should be used to fill in the gaps. For noisy data, the Hampel filtering algorithm is used for smoothing.
5. The method for predicting coagulation dosage in water plants based on RF-BiLSTM according to claim 3, characterized in that, The main input feature filtering includes the following steps: Step 1: For each decision tree, select the corresponding out-of-bag data and calculate the out-of-bag error; Step 2, for all features in the out-of-bag data The samples are shuffled sequentially, with only one feature being shuffled at a time (randomly changing the features of the samples). (value at point), recalculate the outside error of the bag. At this point, under the current decision tree i, the importance score of feature j is calculated using the following formula: ； Step 3: Weighted summation of the importance of all features; assuming the random forest model has T decision trees, following steps 1 and 2, we obtain T decision trees and the importance of all n features. The weighted summation formula for importance scores is as follows: ； This yields importance scores for the n features. ; Step 4: Normalize the importance scores of all features; Step 5: Select the features with higher importance scores as input to the neural network model, and continue to use the coagulant dosing flow rate as the target value.
6. The method for predicting coagulation dosage in water plants based on RF-BiLSTM according to claim 1, characterized in that, Step 3 is to establish a nonlinear relationship model between raw water characteristics and chemical dosing flow rate; the bidirectional long short-term memory network model considers both forward and reverse time series data to capture long-term dependencies in the time series; the input of the bidirectional long short-term memory network model is the main features after preprocessing and feature selection, and the output is the predicted value of coagulant dosage.
7. The method for predicting coagulation dosage in water plants based on RF-BiLSTM according to claim 6, characterized in that, The nonlinear relationship modeling includes the following steps: Step 1: Divide the new dataset after feature selection into training, testing, and validation sets, and normalize them. Step 2: Train the bidirectional Long Short-Term Memory (LSTM) network model. The bidirectional LSTM network unit structure consists of two layers of opposite Long Short-Term Memory (LSTM) networks. The working principle of the LSTM network is expressed as follows: Forgotten Gate: ； Input Gate: ； ； ； Output gate: ； ； in, , , These represent the forget gate, input gate, and output gate, respectively. and These are the weights and bias vectors of the forget gate, respectively; and These are the weights and bias vectors of the input gate, respectively; and These are the weights and bias vectors of the output gate, respectively; Represents the current time state; Represents the characteristics of the input raw water; This represents the final output result after the raw water parameters have been processed by the gating device. This indicates an update to the memory unit; Indicates candidate memory units; and These are the weights and bias vectors of the candidate memory units, respectively; Represents the activation function; Represents the hyperbolic tangent activation function; Step 2-1: Set the hyperparameters of the bidirectional long short-term memory network model, including learning rate, hidden layer dimension, training period, input sequence length, batch size, etc. Step 2-2: Train the bidirectional long short-term memory network model and observe the changes in training error and validation error in each round. If both converge, output the model; otherwise, return to step 2-1 and readjust the hyperparameters. The loss function formula here is: ； in, Indicates the number of samples. This represents the true value of the i-th sample. This represents the predicted value of the i-th sample; Steps 2-3 involve inputting the test set into the trained bidirectional long short-term memory network model for prediction and observing the prediction effect of the dosing flow rate. This completes the prediction of coagulation dosing in water plants based on RF-BiLSTM.