A BOD5 prediction method for wastewater treatment process with data imbalance

Abstract

The present invention discloses a method for predicting BOD5 in a sewage treatment process facing data imbalance, and belongs to the technical field of sewage treatment. The method embeds a spatiotemporal attention mechanism into a Siamese network, so as to better capture the relationship between process variables and the time dependency, and enhance the characterization capability of features; in view of the data imbalance problem in the multimodal sewage treatment process, a deep TL mechanism is introduced into the model to obtain a TL-STSFE model. Make full use of the sufficient process data in the source domain for model training, and use the knowledge of the source domain pre-training model to effectively transfer the learned features to the target domain. Specifically, when training the TL-STSFE model, firstly train with sunny data. After the training is completed, freeze some layers, and then train the unfrozen layers with rainy and rainstorm data to fine-tune the network parameters. While enhancing the generalization capability, it also solves the problem of target domain modeling with data imbalance.

Description

Technical Field

[0001] The invention relates to a BOD5 prediction method for a sewage treatment process facing data imbalance, and belongs to the technical field of sewage treatment. Background Art

[0002] Wastewater treatment technology is an important branch of environmental protection. Whether wastewater treatment meets the standards directly affects environmental quality. In order to ensure the effluent quality of wastewater treatment plants, some key variables are often used to measure the effect of wastewater treatment. However, some key variables are difficult to measure directly. Two methods can be used to achieve the purpose of measurement: (1) Direct measurement, that is, using online analyzers to directly measure the required parameters. However, online analyzers are often expensive and have huge errors, which brings great challenges to real-time soft measurement. Typical direct measurement methods include microbial sensor method and sampling test method; (2) Indirect measurement, that is, using auxiliary variables that are easy to measure to estimate key variables. In the sewage treatment system, the use of soft measurement technology to predict BOD5, a key variable in the sewage treatment process, can significantly reduce the cost of traditional hardware measurement.

[0003] In soft measurement technology, mechanism modeling is to determine the relationship between quality variables and auxiliary variables by establishing a mathematical model of the process based on the inherent mechanism of the process and using certain laws and principles. However, the actual sewage treatment process is affected by many factors, which makes the mathematical model of the actual production process very complex. Traditional modeling and optimization methods are difficult to establish an accurate mathematical model of complex sewage treatment objects, and thus cannot obtain more accurate measurement results. Different from traditional mechanism-based modeling methods, data-driven soft measurement models do not require prior knowledge. Typical data-driven modeling methods include artificial neural networks, support vector regression, principal component regression, etc. Although these methods, especially artificial neural networks, have been successfully implemented in soft measurement applications, due to their shallow structure, it is difficult to extract satisfactory features in complex chemical processes.

[0004] As a data-driven soft sensor modeling method, deep learning methods can extract deep features using multiple hidden layer structures. Its powerful feature extraction capability has attracted great attention from scholars. Typical deep learning methods include convolutional neural networks and deep autoencoders. In order to better explain the dynamics of complex industrial processes, dynamic deep learning methods, such as recurrent neural networks and long short-term memory (LSTM) networks, have great advantages in extracting dynamic features. In order to consider the temporal and spatial correlation between variables and quality variable predictions, a spatiotemporal attention mechanism is introduced.

[0005] As a chemical process, sewage treatment process often changes slowly. Traditional soft sensor modeling methods do not take into account the slow-changing characteristics of sewage treatment process. At present, most slow features (SFs) extraction methods rely on slow feature analysis and kernel slow feature analysis. However, slow feature analysis is difficult to handle the complex nonlinearity in sewage treatment process data; at the same time, the selection of the optimal kernel function in the kernel slow feature analysis method is a difficult problem to solve. To solve the above problems, Chiplunkar et al. proposed a supervised Siamese network to extract SFs, which was then used for soft sensor modeling and compared with the method based on slow feature analysis, showing superior performance.

[0006] However, since there are three different modes of sewage treatment process, namely sunny day, rainy day and rainstorm day, and the number of samples of rainy day and rainstorm day is far less than that of sunny day, the data of the three modes are unbalanced. This data imbalance makes it difficult to establish a reliable deep soft measurement model for rainy day and rainstorm day. Therefore, there is no soft measurement model for biochemical oxygen demand during 5 days (BOD5) for multi-modal sewage treatment process in the prior art. Summary of the invention

[0007] In order to solve the above problems, the present invention provides a transfer learning soft measurement modeling method based on slow spatiotemporal features, and applies it to the soft measurement modeling of a multimodal sewage treatment process with data imbalance characteristics, so as to establish an accurate prediction of the five-day biochemical oxygen demand BOD5 for the multimodal sewage treatment process.

[0008] A method for predicting BOD5 in a sewage treatment process for data imbalance. The method comprises:

[0009] Step 1, collecting auxiliary variables and corresponding BOD5 values ​​in sewage treatment on sunny days, rainy days and rainstorm days;

[0010] Step 2, construct the transfer learning soft sensing model TL-STSFE based on spatiotemporal slow features;

[0011] Step 3, constructing a training set and a test set based on the auxiliary variables and corresponding BOD5 values ​​in the sewage treatment collected in step 1, and training the soft sensor model TL-STSFE constructed in step 2 with sunny data. After the training is completed, some layers of the soft sensor model TL-STSFE are frozen, and then the unfrozen layers are trained with rainy day and rainstorm day data respectively to achieve fine-tuning of network parameters, and finally the trained soft sensor model TL-STSFE for predicting BOD5 values ​​on rainy days and rainstorm days is obtained;

[0012] Step 4: Use the soft sensor model TL-STSFE trained in step 3 to predict the BOD5 of sewage under different weather conditions.

[0013] Optionally, the soft measurement model TL-STSFE includes two identical sub-networks, each sub-network includes an encoder and a decoder; the encoder consists of a spatial attention layer, MLP, LSTM and temporal attention layer; wherein the spatial attention layer is used to extract the spatial relationship between input variables; MLP is used to extract low-level features, and LSTM is used to extract high-level features; the temporal attention layer is used to extract the temporal relationship between variables at different times to obtain spatiotemporal slow features; the decoder consists of an LSTM and MLP, and is used to reconstruct the spatiotemporal slow features into the original input.

[0014] Optionally, in step 3, the MLP layer in the soft sensor model TL-STSFE trained with clear-day data is frozen.

[0015] Optionally, the overall loss function of the soft sensor model TL-STSFE is:

[0016] L=min(L slow +L res +L pre +L dec )

[0017] Among them, L pre is the prediction error term, L slow is the slow feature loss term, L res is the input reconstruction term, L dec is the decorrelation term.

[0018] Optionally, the prediction error term L pre for:

[0019]

[0020] Among them, Y represents the actual value of BOD5, Indicates the BOD5 predicted value.

[0021] Optionally, the slow feature loss term is:

[0022]

[0023] Among them, α>0 is the weight of the slow feature loss term, represents the slow feature corresponding to time t, Indicates the slow feature corresponding to time t-1; Represents the square operation of the L2 norm.

[0024] Optionally, the input reconstruction term L resfor:

[0025]

[0026] Among them, β>0 is the weight of the input reconstruction term, X represents the vector of auxiliary variables in the sewage treatment process, Represents the auxiliary variables reconstructed by the decoder.

[0027] Optionally, the decorrelation term L dec for:

[0028]

[0029] Among them, γ>0, is the weight of the decorrelation term, f(X) represents the encoder output, I is the unit matrix, and cov represents the covariance operation.

[0030] Optionally, auxiliary variables in the sewage treatment process include soluble rapidly degradable organic matter, insoluble non-biodegradable particulate organic matter, insoluble slowly degradable organic matter, active heterotrophic bacterial biomass, active autotrophic bacterial biomass, inert substances produced by the decay of biosolids, dissolved oxygen concentration, nitrate nitrogen concentration, ammonia nitrogen concentration, soluble biodegradable organic nitrogen concentration, particulate degradable organic nitrogen and alkalinity.

[0031] The beneficial effects of the present invention are:

[0032] By constructing a TL-STSFE model, the spatiotemporal attention mechanism is first embedded in the Siamese network to obtain the STSFE model. The STSFE model can better capture the relationship between process variables and the temporal dependency, thereby enhancing the representation ability of features. Specifically, the STSFE model uses the encoder-decoder framework to first integrate the spatial attention mechanism into the encoder network to selectively emphasize the input variables related to quality prediction. Then, the input variables are weighted with different attention values ​​as new inputs to the low-level feature extraction layer MLP and the high-level feature extraction layer LSTM; then, the temporal attention is embedded after the LSTM layer to identify the potential states related to quality prediction. The product of the encoder hidden state and its corresponding temporal attention value is the output of the temporal attention layer, which is used for input reconstruction and quality prediction. In view of the data imbalance problem in the multimodal sewage treatment process, the deep TL mechanism is introduced into the model. The sufficient process data in the source domain is fully utilized for model training, and the knowledge of the source domain pre-trained model is used to effectively transfer the learned features to the target domain. This not only enhances the generalization ability of soft measurement, but also solves the problem of target domain modeling with imbalanced data. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0034] Figure 1 This is the Siamese network structure diagram.

[0035] Figure 2 It is a diagram of the SF-stream structure based on the encoder-decoder structure.

[0036] Figure 3 It is the structural diagram of sewage treatment equipment.

[0037] Figure 4 It is a multimodal scatter plot of sewage treatment.

[0038] Figure 5 It is the prediction curve and error curve of the sewage treatment process results. DETAILED DESCRIPTION

[0039] In order to make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0040] First, the concept of transfer learning involved in this application is introduced as follows:

[0041] Transfer learning (TL) is a technique in the field of machine learning that allows a model to apply knowledge learned on one task to another related task. This method is particularly suitable for situations where the amount of data is small or the acquisition cost is high. For example, Liu et al. proposed a multi-layer adversarial TL method (see "Liu Y, Yang C, Zhang M, et.al. Development of Adversarial Transfer Learning Soft Sensor for Multigrade Processes[J]. Industrial & Engineering Chemistry Research, 2020, 59(37): 16330–16345."), which uses historical data to generate new training data for expanding training data.

[0042] The relevant concepts involved in transfer learning include:

[0043] Domain: It is composed of data features and feature distribution and is the subject of learning. It includes source domain and target domain. Source domain refers to the domain of existing knowledge, while target domain refers to the domain to be learned.

[0044] Task: It consists of the objective function and the learning result, and is the result of learning.

[0045] Embodiment 1:

[0046] This embodiment provides a method for predicting BOD5 of a sewage treatment process for data imbalance. The method constructs a TL-STSFE model, and then uses the model to predict BOD5 of the sewage treatment process. First, the model is introduced as follows:

[0047] When constructing the TL-STSFE model, this application first embeds the spatiotemporal attention mechanism into the Siamese network to obtain the STSFE model, and then introduces the deep TL mechanism into the model, making full use of the sufficient process data in the source domain for model training, and using the knowledge of the source domain pre-training model to effectively transfer the learned features to the target domain, thereby solving the problem of inaccurate BOD5 prediction caused by the imbalance of data in three different modes of sunny days, rainy days and rainy days in the multimodal sewage treatment process.

[0048] Siamese network consists of two networks with the same structure and parameters, which are used to compare the similarity of two inputs. The twin network has two inputs, which are mapped to the feature space by the same neural network. The similarity between the features is calculated by Euclidean distance. Its structure is as follows Figure 1 As shown. In the Siamese network, the encoder network f(x) is used to convert the input x i and x j Mapping to feature z i and z j , the output of the network is the Euclidean distance of each feature pair, that is, the similarity of the features.

[0049] The objective function of the Siamese network is shown in formula (1):

[0050]

[0051] Among them, D i =d{z i ,z j} represents the slow feature pair z i and z j The Euclidean distance between the two input pairs is ξ, where the parameter ξ is the similarity between the input pairs. If ξ = 0, the inputs are exactly the same; if ξ = 1, the inputs are completely different; and δ is the threshold.

[0052] The spatiotemporal attention mechanism is embedded in the Siamese network to better capture the relationship between process variables and the temporal dependency to enhance the representation ability of features. The STSFE model uses the encoder-decoder framework to incorporate the spatial attention mechanism in the encoder network to selectively emphasize input variables related to quality prediction. The input variables are then weighted with different attention values ​​as new inputs to the low-level feature extraction layer MLP and the high-level feature extraction layer LSTM; then, temporal attention is embedded after the LSTM layer to identify latent states related to quality prediction. The product of the encoder hidden state and its corresponding temporal attention value is the output of the temporal attention layer, which is used for input reconstruction and quality prediction.

[0053] Figure 2 The architecture of the TL-STSFE model is given. The model consists of two identical sub-networks, each of which contains an encoder and a decoder. The encoder consists of a spatial attention layer, MLP, LSTM, and temporal attention layer. Among them, the spatial attention layer is used to extract the spatial relationship between the input variables; MLP is used to extract low-level features, and LSTM is used to extract high-level features; the temporal attention layer is used to extract the temporal relationship between variables at different times to obtain spatiotemporal slow features. The decoder consists of an LSTM and MLP, which are mainly used to reconstruct the spatiotemporal slow features into the original input. In the spatial dynamic attention calculation, the input variables are activated by the tanh function and then mapped by a single layer of neurons. Then, the weights are normalized using the softmax function to ensure their constrained additivity, as shown in Equation (2). The new input can be calculated from the original input and weight values ​​by Equation (3).

[0054] a t =SA(x t )=softmax(tanh(W1x t +b1)) (2)

[0055] x′ t =a t ex t (3)

[0056] Among them, a t is the spatial attention weight, SA(*) represents the spatial attention layer, W1 and b1 are the linear layer weights and biases of the spatial attention, and x t ' represents the new input, and e is the Hadamard product. Then, the spatial attention weighted variables are input into the MLP and LSTM layers for low-level and high-level feature extraction respectively; the feature z is obtained t as the input of the temporal attention layer shown in Equation (4). Then, the temporal attention weights are combined with z tMultiply to get the time attention weighted feature As shown in formula (5).

[0057] β t =TA(x t )=softmax(tanh(W2z t +b2)) (4)

[0058]

[0059] Where βt is the temporal attention weight, TA(*) represents the spatial attention layer, and W2 and b2 are the linear layer weights and biases of the temporal attention.

[0060] like Figure 2 As shown in Figure 1, the STSFE soft sensor model is a hierarchical structure model. First, the spatial attention layer is used to obtain the spatial relationship between variables, then the MLP is used to extract low-level features, and the LSTM is used to extract high-level features. Then, the temporal attention layer is used to obtain spatial-temporal slow features (STSFs), and the decoder is used for input reconstruction. In addition, STSFs and the quality variables at the previous moment are expanded for the prediction of the quality variables at the current moment. The overall loss function of the model is defined as formula (6):

[0061] L=min(L slow +L res +L pre +L dec ) (6)

[0062] Compared with the slow feature loss function term, the overall loss function increases the prediction error L pre It is used to constrain the overall prediction of the model. At the same time, this item also has a constraining effect on NFF. This loss function is divided into four sub-loss functions, which are described as follows:

[0063] 1) This part uses a slow feature extraction term, which aims to minimize the distance metric between feature pairs. This minimization is achieved by assigning weights, and the weight of this term is α ＞0 The purpose of weighting is to ensure that the features exhibit slow-changing characteristics to a certain extent, thereby improving the stability and robustness of the model. By emphasizing the importance of slow changes in input data, it aims to capture basic time dependencies for more accurate predictions. The sub-loss function is expressed as formula (7):

[0064]

[0065] 2) The main purpose of the input reconstruction term is to ensure the effectiveness of the slow features. This sub-loss function is weighted by β>0. The purpose of this weighting is to ensure that the feature can reconstruct the input data to a certain extent, thereby ensuring its relevance and usefulness in capturing basic information. Its specific expression is as shown in formula (9):

[0066]

[0067] 3) The main purpose of the prediction error term is to ensure that the slow feature can predict the quality variable with higher accuracy. Considering its importance, the weight of this term is set to 1. This ensures that the slow feature can achieve higher accuracy when predicting the quality variable. Its specific expression is shown in formula (10):

[0068]

[0069] 4) In soft sensor modeling, it is very important to ensure that the extracted features are independent and decorrelated. The proposed objective function adds a decorrelation term to make the covariance matrix of the slow feature become the unit matrix. The weight of this term is expressed as γ>0, and its specific expression is shown in formula (11):

[0070]

[0071] The values ​​of hyperparameters α, β, and γ are determined by grid search.

[0072] In order to solve the data imbalance problem in the multimodal sewage treatment process, the deep TL mechanism is introduced into the model. The sufficient process data in the source domain is fully utilized for model training. Subsequently, its low-level feature extraction layer is frozen, and the remaining part is fine-tuned using the insufficient target domain training data.

[0073] The TL-STSFE model leverages the knowledge of the source domain pre-trained model and can effectively transfer the learned features to the target domain. This not only enhances the generalization ability of soft sensing, but also solves the problem of target domain modeling with imbalanced data. Overall, the TL-STSFE soft sensing model has higher accuracy, robustness, and adaptability than traditional methods.

[0074] In order to verify the prediction effect of the TL-STSFE model proposed in this application in sewage treatment, Figure 3 The sewage treatment equipment shown in the figure is used as an example for the experiment. Figure 3 As shown in the figure, the modular design of the process has five activated sludge reactors, including two anoxic tanks and three aeration tanks. The sedimentation tank is divided into 10 layers, each 4 meters high. The reactors are of two types: internal circulation and external circulation. Figure 3 In, Q i is the sewage inflow, Q r is the sewage return flow, Qf is the sewage outflow from the aquaculture area, Q w is the excess sludge, Q e is the amount of sewage discharged into the river. The average daily flow of the sewage treatment plant is 20000m3 / D, and COD is 300mg / L. This experiment uses the outlet BOD5 as the prediction index and the root mean square error (RMSE) as the evaluation index. Soft measurement technology is used for variable prediction, that is, some measured auxiliary variables are used as the input of the system, and the predicted variable values ​​are used as the output of the system. The auxiliary variables used in this application are shown in Table 1.

[0075] Table 1 Auxiliary variables of sewage treatment process

[0076]

[0077]

[0078] Figure 4 The scatter plot of BOD5 prediction value and true value under different weather conditions using the method of this application and other existing methods reflects the advantages of the method proposed in this application over other existing related methods. Other existing related methods include TCA (Transfer Component Analysis), BDA-LSTM, Siamese network, TL-Siamese network and STSFE model; among them, TCA can refer to the prediction method introduced in patent CN114512191A-A method for predicting penicillin concentration based on migration component analysis, and use it in sewage treatment BOD5 prediction. The BDA-LSTM method can refer to the introduction in "An soft-sensor method for the biochemical reaction process based on LSTM and transferlearning"; the Siamese network adopts Figure 2 The STSFE network shown in the figure removes the spatiotemporal attention layer for BOD5 prediction. The TL-Siamese network combines transfer learning and Siamese network for BOD5 prediction. The STSFE model is Figure 2 The network shown is directly applied to BOD5 prediction without embedding TL ideas.

[0079] It can be found that under the three weather conditions, the scattered points of the method proposed in this application are more closely distributed around the 45° line. In order to more clearly show the superiority of the proposed method, Figure 5 The forecast curve and error curve under different weather conditions are given. Figure 5The left graph shows the BOD5 prediction values ​​obtained by fitting different methods; the right graph shows the error curves under three weather conditions. Figure 5 As can be seen from (a) in Figure 1, on sunny days (source domain), the deep learning method can obtain better results. This is due to the superior internal mechanism of the deep learning model. Figure 5 It can be seen from (b) and (c) in that the transfer learning method performs well in the soft sensor modeling of rainy days and rainstorms (target domain). This is because TCA and BDA-LSTM are domain adaptive methods, and the source domain can be migrated to the target domain. For the TL-Siamese and TL-STSFE methods, the source domain and the training model are reused in the target domain, and the model is fine-tuned using the target domain training set. In addition, TL-STSFE captures the spatiotemporal information between variables by introducing a spatiotemporal attention mechanism, thereby obtaining better prediction performance. The comparison results of the method proposed in this application with TCA, BDA-LSTM, Siamese and STSFE are shown in Table 2. Its indicators are calculated as:

[0080]

[0081] Table 2 Comparative indicators of sewage treatment process

[0082]

[0083] It can be seen from Table 2 that the model proposed in this application has the highest prediction accuracy and can better predict the BOD5 value in the sewage treatment process.

[0084] Some steps in the embodiments of the present invention may be implemented using software, and the corresponding software program may be stored in a readable storage medium, such as a CD or a hard disk.

[0085] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for predicting BOD5 in a sewage treatment process with data imbalance, characterized in that: The method comprises: Step 1, collecting auxiliary variables and corresponding BOD5 values ​​in sewage treatment on sunny days, rainy days and rainstorm days; Step 2, construct the transfer learning soft sensing model TL-STSFE based on spatiotemporal slow features; Step 3, constructing a training set and a test set based on the auxiliary variables and corresponding BOD5 values ​​in the sewage treatment collected in step 1, and training the soft sensor model TL-STSFE constructed in step 2 with sunny data. After the training is completed, some layers of the soft sensor model TL-STSFE are frozen, and then the unfrozen layers are trained with rainy day and rainstorm day data respectively to achieve fine-tuning of network parameters, and finally the trained soft sensor model TL-STSFE for predicting BOD5 values ​​on rainy days and rainstorm days is obtained; Step 4, using the soft sensor model TL-STSFE trained in step 3 to predict the BOD5 of sewage under different weather conditions; The soft measurement model TL-STSFE includes two identical sub-networks, each of which contains an encoder and a decoder; the encoder consists of a spatial attention layer, MLP, LSTM and temporal attention layer; the spatial attention layer is used to extract the spatial relationship between input variables; MLP is used to extract low-level features, and LSTM is used to extract high-level features; the temporal attention layer is used to extract the temporal relationship between variables at different times to obtain spatiotemporal slow features; the decoder consists of an LSTM and MLP, which is used to reconstruct the spatiotemporal slow features into the original input.

2. The method according to claim 1, characterized in that In the step 3, the MLP layer in the soft sensor model TL-STSFE trained with the clear sky data is frozen.

3. The method according to claim 2, characterized in that The overall loss function of the soft sensor model TL-STSFE is: L=min(L slow +L res +L pre +L dec ) Among them, L pre is the prediction error term, L slow is the slow feature loss term, L res is the input reconstruction term, L dec is the decorrelation term.

4. The method according to claim 3, characterized in that The prediction error term L pre for: Among them, Y represents the actual value of BOD5, Indicates the BOD5 predicted value.

5. The method according to claim 3, characterized in that: The slow feature loss term is: Among them, α>0 is the weight of the slow feature loss term, represents the slow feature corresponding to time t, Indicates the slow feature corresponding to time t-1; Represents the square operation of the L2 norm.

6. The method according to claim 3, characterized in that The input reconstruction term L res for: Among them, β>0 is the weight of the input reconstruction term, X represents the vector of auxiliary variables in the sewage treatment process, Represents the auxiliary variables reconstructed by the decoder.

7. The method according to claim 3, characterized in that The decorrelation term L dec for: Among them, γ>0, is the weight of the decorrelation term, f(X) represents the encoder output, I is the unit matrix, and cov represents the covariance operation.

8. The method according to claim 1, characterized in that: The auxiliary variables in the sewage treatment process include soluble rapidly degradable organic matter, insoluble non-biodegradable particulate organic matter, insoluble slowly degradable organic matter, active heterotrophic bacterial biomass, active autotrophic bacterial biomass, inert matter produced by biosolid decay, dissolved oxygen concentration, nitrate nitrogen concentration, ammonia nitrogen concentration, soluble biodegradable organic nitrogen concentration, particulate degradable organic nitrogen and alkalinity.

Images