Outlet water COD (Chemical Oxygen Demand) prediction method based on mixed attention mechanism and codec

Abstract

The invention provides an effluent COD (Chemical Oxygen Demand) prediction method based on a mixed attention mechanism and a codec, which comprises the following steps of: acquiring an inflow parameter, an operation parameter and an effluent COD value of a sewage treatment plant, and carrying out preprocessing such as data cleaning, normalization and data set division on data; then constructing an effluent COD prediction model comprising a coder-decoder structure, a mixed attention mechanism and a regression module; taking the water inlet parameters and the operation parameters as input data, taking the effluent COD as output data, and training a model based on a back propagation algorithm; and calling the trained model to predict the effluent COD. According to the method, a coder-decoder structure is utilized to capture a complex mapping relation of a multivariate long-time sequence in the sewage treatment process, a mixed attention mechanism is designed to adaptively extract key information in time and space dimensions, the model can flexibly adapt to various working condition change conditions of a sewage treatment plant, and the prediction accuracy is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of sewage treatment, and in particular to an effluent COD prediction method based on a hybrid attention mechanism and a codec. Background Art

[0002] As a key link in the urban water cycle, improving the urban sewage treatment capacity has become an inevitable choice to improve the urban ecological environment and alleviate water shortage. The effluent COD index is one of the key parameters that reflects the stability of the operation of the sewage treatment plant in removing pollutants from sewage. At present, my country's sewage treatment plants mainly rely on laboratory testing or online instruments to monitor the effluent COD. Although the laboratory test method has the characteristics of high detection accuracy and accuracy, its detection cycle is long and there is a time lag, making it difficult to identify the phenomenon of effluent exceeding the standard in time. Although the online monitoring instrument has a short measurement cycle, it is expensive, has high maintenance costs, and poor versatility. In addition, the online instrument still requires chemical reagents for auxiliary detection, and the detection accuracy is easily affected by the purity of the reagents. Therefore, how to achieve timely measurement and even prediction of effluent COD to ensure the low-carbon and stable operation of sewage treatment plants is one of the urgent problems to be solved in my country's urban sewage treatment plants.

[0003] The sewage treatment process is a typical nonlinear, large hysteresis, strong external interference, and multivariable coupling system. The effluent COD is not only affected by the current influent water quality conditions, but also by the historical system status. Therefore, it is necessary to pay attention to the impact of related variables on the effluent COD and the impact of different historical time system states on the effluent COD. To this end, the present invention uses the decoder and encoder structure to capture the key information of the input variables, extracts the relevant parameters and historical time information based on the hybrid attention mechanism, and develops a sewage treatment plant effluent COD prediction method on this basis. Summary of the invention

[0004] Purpose of the invention: In order to overcome the influence of hysteresis, nonlinearity and multivariate factors on effluent COD prediction in the sewage treatment process and give full play to the role of each variable, this invention proposes an effluent COD prediction method based on a hybrid attention mechanism and a codec. The encoder converts the multivariate time series including the influent water quality and operating parameters of the sewage treatment plant into a feature vector, and then obtains the weight distribution of the influence of the feature vector on the future effluent COD in time and vector space based on the hybrid attention mechanism. Finally, the decoder is used to generate the predicted value of the future effluent COD.

[0005] The technical solution adopted by the present invention is:

[0006] A method for predicting outlet COD based on a hybrid attention mechanism and an encoder-decoder comprises the following steps:

[0007] S1. Data acquisition and preprocessing: Collect the influent water quality parameters, operation parameters and effluent COD parameters of the sewage treatment plant, including i auxiliary variables (influent water quality parameters, operation parameters) and 1 effluent COD parameter, remove abnormal data, fill in null values, and normalize the data to [-1,1];

[0008] S2. Dataset reconstruction: The dataset is divided into training set, validation set and test set. Then, based on the sliding window technology, a fixed time window is slid on the time series on the time axis to divide the training set and the test set into datasets containing subsequences of fixed time length.

[0009] S3. Model construction and training: Build a deep learning model, including encoder and decoder modules composed of long short-term memory neural network (LSTM), temporal and spatial feature extraction modules composed of hybrid attention mechanism, and regression module composed of fully connected neural network. Select loss function, input training set data into the model, update weight parameters, evaluate the model according to the output results of training set and validation set, optimize model hyperparameters, and finally obtain effluent COD prediction model;

[0010] S4. Prediction of effluent water quality: Call the deep learning model, input the test set into the model, and obtain the model output, which is the predicted value of effluent COD.

[0011] Furthermore, in the step S1, outliers are eliminated based on the Laida criterion, empty values ​​are filled using cubic spline interpolation, and the data are normalized to [-1, 1] based on maximum and minimum value normalization.

[0012] Furthermore, in the step S2, the ratio of the training set, the validation set and the test set is 7:1:2.

[0013] Furthermore, in the step S2, the present invention uses the data of the past τ days (or hours) to predict the effluent COD of the next day (or hour). In order to establish samples suitable for training and testing the proposed prediction model, the data set reconstruction based on the sliding window technology is achieved through the following steps: ① A time window with a fixed length of τ is set along the time axis of the auxiliary variables and the effluent water quality characteristics. All data in the time window are the first input samples, and the effluent COD at the τ+1th moment is used as the corresponding output sample; ② The time window moves one unit in the positive direction along the time axis, and the second paired sample is obtained in accordance with step ①, and so on, to obtain all samples in the training set, validation set and test set.

[0014] Furthermore, in the step S3, the encoder module is composed of a long short-term memory neural network, and the long short-term memory neural network is mainly composed of an input gate, a forget gate and an output gate. The calculation functions of each structure are as follows:

[0015] ① Input gate: controls the degree to which input data is retained in the network, calculated by the following formula:

[0016] i t =σ(W i ·[h t-1 ,x t ]+b i ) (1)

[0017] Among them, i t is the input gate, σ is the sigmoid function, W i With b i is the weight matrix of the input gate and the bias term is, h t-1 is the output signal of the LSTM unit at the previous moment, x t It is the i-dimensional auxiliary variable and the effluent COD concentration in the past τ time.

[0018] ②Forget gate: controls the retention degree of the unit state information at the previous moment, calculated by the following formula:

[0019] f t =σ(W f ·[h t-1 ,x t ]+b f ) (2)

[0020] Among them, f t is the forget gate, W f and b f is the weight matrix and bias term of the forget gate.

[0021] Based on the input gate and the forget gate, update the unit state information C t , the update process is:

[0022] C t =f t *C t-1 +i t *tanh(W C ·[h t-1 ,x t ]+b C ) (3)

[0023] Among them, C t-1 is the unit state information of the LSTM unit at the previous moment, W C and b C It is the weight matrix and bias term in the unit state information process.

[0024] ③Output gate:

[0025] o t =σ(W o ·[ht-1 ,x t ]+b o ) (4)

[0026] h t =o t *tanh(C t ) (5)

[0027] Among them, t is the forget gate, W o and b o is the weight matrix and bias term of the output gate, h t is the output signal, and * represents the Hadamard product.

[0028] The encoder input is the data of τ time steps before τ time, and its feature vector output matrix is ​​H = [h1,h2,h3,…,h τ-1 ,h τ ], the decoder is also composed of an LSTM neural network, and its input is the encoder output signal h at time τ τ After the same operation process as the encoder, the output is h τ '.

[0029] Furthermore, in the step S3, the temporal feature and spatial feature extraction module is composed of a hybrid attention mechanism. The hybrid attention mechanism mainly calculates the feature vector extracted by the encoder and constructs a hybrid attention matrix. The calculation of each structure is as follows:

[0030] First, based on H and h τ 'The cosine correlation in the time dimension and the space dimension is combined with the softmax function to calculate the attention matrix in the time scale, feature space, and mixed attention matrix respectively. The calculation process is:

[0031] α t =softmax(Hh' τ ) (6)

[0032] α f =softmax[h' τ (W α H)] (7)

[0033] α=α t α f (8)

[0034] Among them, α t is the temporal attention feature matrix, α f is the spatial attention feature matrix, α is the mixed attention matrix, and softmax represents the Softmax function.

[0035] The hybrid attention matrix is ​​used to filter important information from the feature vector of the encoder data, and the final information in the time and space dimensions is obtained through the maximum pooling operation. The calculation process is as follows:

[0036] β=α*H (9)

[0037]

[0038] Among them, β is the feature vector selected based on hybrid attention, β t is the eigenvector of the time dimension, β f is the eigenvector of the spatial dimension, represents the maximum value along the i-axis. Represents the maximum value along the j-axis.

[0039] Furthermore, in the step S3, the regression module is composed of a fully connected neural network, and the fully connected neural network is calculated by the following process:

[0040]

[0041] is the output value of the COD prediction model for effluent from the sewage treatment plant, W y are the weights of the fully connected neural network.

[0042] Beneficial effects:

[0043] (1) Aiming at the inevitable time lag and strong coupling of multiple variables when predicting effluent water quality in sewage treatment plants, the present invention proposes an effluent COD prediction model based on a hybrid attention mechanism and a codec. Specifically, the present invention uses the codec structure to capture the complex mapping relationship of multivariate long time series in the sewage treatment process, and designs a hybrid attention mechanism to adaptively extract key information. The model can flexibly adapt to various operating conditions of sewage treatment plants, thereby improving the prediction accuracy.

[0044] (2) The present invention designs a hybrid attention mechanism based on the encoder-decoder, and simultaneously extracts effective information of the time dimension and space dimension of the feature vector, which can identify the different time lags caused by water inlet flow fluctuations, reduce the impact of redundant information in multiple variables, and improve computational efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 A flow chart is constructed for the effluent COD prediction model method based on hybrid attention mechanism and encoder-decoder.

[0046] Figure 2 This is the framework diagram of the designed model.

[0047] Figure 3 This is the error graph of the model prediction results. DETAILED DESCRIPTION

[0048] In order to make the implementation path of the scheme of the present invention clearer and more specific, the present invention will be further described in detail below in conjunction with an embodiment and drawings. However, this embodiment is only a preferred embodiment of the present invention and does not limit the present invention in any form.

[0049] The implementation steps of the effluent COD prediction model method based on the hybrid attention mechanism and codec designed by the present invention are as follows: Figure 1 shown.

[0050] Step (1): Collect data from the sewage treatment plant. The parameters collected in the embodiment are: ① the influent water quality parameters are influent flow, influent COD, influent TN, influent TP, influent NH3-N, influent pH; ② the operating parameters are aerobic pool DO, MLSS, SV 30 , internal and external reflow ratio, carbon source dosage, phosphorus removal agent dosage, ③ effluent water quality parameter is effluent COD. The data was collected from 365 days of data from a sewage treatment plant in southern China.

[0051] Step (2): Preprocess the collected data. Since there may be abnormal data, missing data, and errors in the data collection process, the data needs to be preprocessed. The processing steps are as follows:

[0052] (2.1) In order to reduce the impact of outliers on the model, outliers are identified according to the Laida criterion, and the identified outliers are screened based on the time series characteristics of the data. The real outliers and zero values ​​are eliminated, and the cubic spline interpolation method is used to interpolate and supplement the null values.

[0053] (2.2) In order to reduce the amount of subsequent model calculations, the input data is normalized. Specifically, the maximum and minimum value normalization method is used to normalize the data to [-1, 1]. The calculation formula is:

[0054]

[0055] b' is the normalized data, b is the original data, b mean is the average value of a feature in the original data, b max is the maximum value of a feature in the original data, b min It is the minimum value of a feature in the original data.

[0056] Step (3): Dataset reconstruction. First, the normalized data is divided into a training set, a validation set, and a test set in a ratio of 7:1:2. Secondly, in this embodiment, the data of the past 14 days are used to predict the effluent COD of the next day. In order to establish samples suitable for training and testing the proposed prediction model, the following steps are used to achieve data set reconstruction based on the sliding window technology: ① A time window with a fixed length of 14 is set along the time axis of the auxiliary variables and the effluent water quality characteristics. All data in the time window are the first input samples, and the effluent COD on the 15th day is used as the corresponding output sample; ② The time window moves one unit in the positive direction along the time axis, and the second paired sample is obtained in accordance with step ①. And so on, all samples in the training set, validation set, and test set are obtained.

[0057] Step (4): Construct an outlet COD prediction model based on a hybrid attention mechanism and encoder-decoder structure. Figure 2 As shown in the figure, the model method mainly consists of an encoder and decoder module composed of a long short-term memory neural network (LSTM), a temporal feature and spatial feature extraction module composed of a hybrid attention mechanism, and a regression module composed of a fully connected neural network. The encoder extracts feature vectors from the input data, the decoder interprets the feature vectors, the hybrid attention focuses on the more important features in the temporal and spatial dimensions of the feature vectors, and the fully connected neural network is used to generate prediction values. The specific construction steps are as follows:

[0058] (4.1) Codec construction. LSTM has long-term and short-term memory functions, can learn data time dependencies, and can mine patterns from historical data when predicting effluent COD. The present invention uses LSTM as the basic neural network model of the codec structure, which is mainly composed of an input gate, a forget gate, and an output gate. The input gate determines the degree to which the input data is retained in the network, the forget gate determines the degree to which the previous unit information state is retained, and the output gate determines the output value. In this embodiment, the number of LSTM hidden layers is 256, and the calculation process is as follows:

[0059] i t =σ(W i ·[h t-1 ,x t ]+b i ) (2)

[0060] f t =σ(W f ·[h t-1 ,x t ]+b f ) (3)

[0061] C t =f t *C t-1 +it *tanh(W C ·[h t-1 ,x t ]+b C ) (4)

[0062] o t =σ(W o ·[h t-1 ,x t ]+b o ) (5)

[0063] h t =o t *tanh(C t ) (6)

[0064] Among them, i t is the input gate, σ is the sigmoid function, W i With b i is the weight matrix of the input gate and the bias term is, h t-1 is the output signal of the LSTM unit at the previous moment, x t is the 12-dimensional auxiliary variable and the effluent COD concentration in the past 14 days, t is the forget gate, W f and b f is the weight matrix and bias term of the forget gate, C t is the cell state information of the LSTM cell at that moment, C t-1 is the unit state information of the LSTM unit at the previous moment, W C and b C is the weight matrix and bias term in the unit state information process, o t is the forget gate, W o and b o is the weight matrix and bias term of the output gate, h t is the output signal, and * represents the Hadamard product.

[0065] The encoder input is the previous 14 days, which contains 14 time steps of data, and its feature vector output matrix is ​​H = [h1,h2,h3,…,h 13 ,h 14 ], the decoder is also composed of LSTM neural network, and its input is the encoder’s fourteenth day output signal h 14 After the same operation process as the encoder, the output is h 14 '.

[0066] (4.2) Construction of hybrid attention matrix. The sewage treatment process consists of complex biochemical reactions and physical reactions. There is time lag in the reaction process. At the same time, the treatment process is affected by multiple parameters, and there is coupling between the parameters. In order to solve the time lag and parameter coupling of the sewage treatment process, the present invention designs a hybrid attention mechanism. The main idea of ​​the attention mechanism is to selectively pay attention to important information. The essence of the hybrid attention mechanism is to pay attention to the important information in the time and space dimensions of the feature vector output by the encoder LSTM at the same time, obtain the weight vector, and improve the efficiency of information processing. The calculation process is as follows:

[0067] First, based on H and h 14 'The cosine correlation in the time dimension and the space dimension is combined with the softmax function to calculate the attention matrix in the time scale, feature space, and mixed attention matrix respectively. The calculation process is:

[0068] α t =softmax(Hh' 14 ) (7)

[0069] α f =softmax[h' 14 (W α H)] (8)

[0070] α=α t α f (9)

[0071] Among them, α t is the temporal attention feature matrix, α f is the spatial attention feature matrix, α is the mixed attention matrix, W α is the weight of the spatial attention feature matrix, and softmax represents the Softmax function.

[0072] The hybrid attention matrix is ​​used to filter important information from the feature vector of the encoder data, and the final information in the time and space dimensions is obtained through the maximum pooling operation. The calculation process is as follows:

[0073] β=α*H (10)

[0074]

[0075] Among them, β is the feature vector selected based on hybrid attention, β t is the eigenvector of the time dimension, β f is the eigenvector of the spatial dimension, represents the maximum value along the i-axis. Represents the maximum value along the j-axis.

[0076] (4.3) Fully connected neural network construction. The prediction regression module consists of a fully connected neural network, which is calculated through the following process:

[0077]

[0078] is the output value of the COD prediction model for effluent from the sewage treatment plant, W y are the weights of the fully connected neural network.

[0079] (4.4) Model training and optimization. The model is trained through back propagation, and the gradient descent method is used to minimize the loss function. The loss function uses the mean absolute error (MAE). The calculation process is as follows:

[0080]

[0081] In the formula, m is the sample size, y is the true value, is the predicted value.

[0082] The MAE error of the test set should be less than 2 mg / L. If the error is greater than this value, it is necessary to adjust and optimize the hyperparameters such as the number of hidden layers of the model, the length of the time window, the number of iterations, and the learning rate.

[0083] In this embodiment, the test set effluent COD prediction error is as follows: Figure 3 shown.

[0084] In summary, the effluent COD prediction method based on the hybrid attention mechanism and codec designed in the present invention effectively promotes the decoupling of multivariables in sewage treatment plants and makes full use of the multivariable information in sewage treatment plants. In addition, it can identify the different time lags caused by inlet flow fluctuations and realize the prediction of effluent COD.

[0085] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations and simplifications made according to the concepts or technical solutions of the present invention are all included in the protection scope of the present invention.

Claims

1. The outlet COD prediction method based on hybrid attention mechanism and encoder-decoder is characterized by: The following steps are involved: S1. Data acquisition and preprocessing: Collect the influent water quality parameters, operation parameters and effluent COD parameters of the sewage treatment plant, including i auxiliary variables and effluent COD parameters, remove abnormal data, fill in null values, and normalize the data to [-1, 1]; S2. Dataset reconstruction: The dataset is divided into training set, validation set and test set. Then, based on the sliding window technology, a fixed time window is slid on the time series on the time axis to divide the training set and the test set into datasets containing subsequences of fixed time length. S3. Model construction and training: Build a deep learning model, including encoder and decoder modules composed of long short-term memory neural networks, temporal and spatial feature extraction modules composed of hybrid attention mechanisms, and regression modules composed of fully connected neural networks. Select a loss function, input the training set data into the model, update the weight parameters, evaluate the model based on the output results of the training set and the validation set, optimize the model hyperparameters, and finally obtain the effluent COD prediction model; S4. Prediction of effluent water quality: Call the deep learning model, input the test set into the model, and obtain the model output, which is the predicted value of effluent COD.

2. The method for predicting outlet COD based on hybrid attention mechanism and codec according to claim 1, characterized in that: The S1 step removes outliers based on the Laida criterion, fills in empty values ​​using cubic spline interpolation, and normalizes the data to [-1, 1] based on maximum and minimum value normalization.

3. The method for predicting outlet COD based on hybrid attention mechanism and codec according to claim 2, characterized in that: In the step S2, the ratio of the training set, the validation set and the test set is 7:1:

2.

4. The method for predicting outlet COD based on hybrid attention mechanism and codec according to claim 3 is characterized in that: In the step S2, the data of the past τ days are used to predict the effluent COD of the next day. In order to establish samples suitable for training and testing the proposed prediction model, the following steps are used to achieve data set reconstruction based on the sliding window technology: ① A time window with a fixed length of τ is set along the time axis of the auxiliary variables and the effluent water quality characteristics. All data in the time window are the first input samples, and the effluent COD at the τ+1th moment is used as the corresponding output sample; ② The time window is moved one unit forward along the time axis, and the second paired sample is obtained in accordance with step ①. By analogy, all samples in the training set, validation set and test set are obtained.

5. The method for predicting outlet COD based on hybrid attention mechanism and codec according to claim 4, characterized in that: In the step S3, the encoder module is composed of a long short-term memory neural network, which is mainly composed of an input gate, a forget gate, and an output gate. The calculation functions of each structure are as follows: i t =σ(W i ·[h t-1 ,x t ]+b i ) (1) f t =σ(W f ·[h t-1 ,x t ]+b f ) (2) C t =f t *C t-1 +i t *tanh(W c ·[h t-1 ,x t ]+b c ) (3) the t =σ(W o ·[h t-1 ,x t ]+b o ) (4) h t =o t *tanh(C t ) (5) Among them, i t is the input gate, σ is the sigmoid function, W i With b i is the weight matrix of the input gate and the bias term is, h t-1 is the output signal of the LSTM unit at the previous moment, x t is the i-dimensional auxiliary variable and the effluent COD concentration in the past τ time, f t is the forget gate, W f and b f is the weight matrix and bias term of the forget gate, C t is the cell state information of the LSTM cell at that moment, C t-1 is the unit state information of the LSTM unit at the previous moment, W C and b C is the weight matrix and bias term in the unit state information process, o t is the forget gate, W o and b o is the weight matrix and bias term of the output gate, h t is the output signal, * represents the Hadamard product; The encoder contains τ time steps of data before τ, and its feature vector output matrix is ​​H = [h1, h2, h3, ..., h τ-1 ,h τ ], the decoder is also composed of an LSTM neural network, and its input is the encoder output signal h at time τ τ After the same operation process as the encoder, the output is h τ '.

6. The method for predicting outlet COD based on hybrid attention mechanism and codec according to claim 5, characterized in that: In the S3 step, the temporal feature and spatial feature extraction module is composed of a hybrid attention mechanism. The hybrid attention mechanism mainly calculates the feature vector extracted by the encoder and constructs a hybrid attention matrix. The calculation of each structure is as follows: First, based on H and h τ 'The cosine correlation in the time dimension and the space dimension is combined with the softmax function to calculate the attention matrix in the time scale, feature space, and mixed attention matrix respectively. The calculation process is: α t =softmax(Hh′ τ ) (6) a f =softmax[h′ τ (W α H)] (7) α＝α t α f (8) Among them, α t is the temporal attention feature matrix, α f is the spatial attention feature matrix, α is the mixed attention matrix, and softmax represents the Softmax function; The hybrid attention matrix is ​​used to filter important information from the feature vector of the encoder data, and the final information in the time and space dimensions is obtained through the maximum pooling operation. The calculation process is as follows: β＝α*H (9) Among them, β is the feature vector selected based on hybrid attention, β t is the eigenvector of the time dimension, β f is the eigenvector of the spatial dimension, represents the maximum value along the i-axis. Represents the maximum value along the j-axis.

7. The method for predicting outlet COD based on hybrid attention mechanism and codec according to claim 6, characterized in that: In the step S3, the regression module is composed of a fully connected neural network, which is calculated by the following process: is the output value of the COD prediction model for effluent from the sewage treatment plant, W y are the weights of the fully connected neural network.

Images