A soft measurement method for wastewater effluent index BOD5 based on Transformer and Long Short-Term Memory Network

Abstract

The application discloses a BOD5 index soft measurement method based on a Transformer and a long short-term memory network. The method uses the easily measured effluent index parameters in sewage treatment and the continuous time data of the effluent index parameters affecting BOD5 to construct a complete air quality data set that can be used for prediction. Then, the data set after pretreatment is proportionally divided to construct a training set and a test set. The training set data is used to optimize the parameters of the LSTM-former model, including the number of attention head layers, model dimensions, the dimensions of Q, K and V in the attention mechanism, the number of neurons, the random inactivation rate, the initial learning rate, the number of iterations, the batch size and the like. Then, the LSTM-former model after parameter optimization that can be used for soft measurement of the BOD5 concentration is constructed, the optimization algorithm is determined, and the activation function is selected. The concentration prediction value of BOD5 is obtained based on the trained LSTM-former soft measurement model, and the model precision is improved, and the generalization ability is enhanced.

Description

Technical Field

[0001] This invention relates to the field of time-series prediction of BOD5, an effluent indicator in wastewater, and particularly to a soft measurement technique for nonlinear and complex BOD5 concentration data. This invention is a specific application of a method combining an efficient multi-head attention structure transformer network with a long short-term memory network in the field of BOD5 soft measurement in wastewater treatment. This invention improves the accuracy of soft measurement in wastewater treatment while preserving the temporal correlations hidden in the time series. Background Technology

[0002] Water is a vital component of ecosystems and human health. To ensure the sustainable use of water resources and environmental protection, wastewater treatment is of paramount importance. Whether wastewater treatment meets standards directly impacts the ecological environment and human health. Key effluent indicators are frequently used to measure the quality of wastewater treatment. For example, the five-day biochemical oxygen demand (BOD5) is a crucial indicator for assessing the concentration of organic pollutants in water bodies. This parameter is defined as the amount of dissolved oxygen required for aerobic organisms to decompose organic matter in water within a specific timeframe (typically five days) at a specific temperature. It is used to measure the amount of oxygen required for the decomposition of organic matter in water.

[0003] In real-world wastewater treatment, monitoring effluent parameters often relies on instrumental measurements. However, some crucial parameters are difficult to measure directly, involving time-consuming and expensive laboratory tests. Furthermore, the accuracy of measuring instruments is inconsistent, leading to lag and instability in effluent parameter monitoring. Therefore, to improve the monitoring capabilities of wastewater treatment effluent parameters, indirect measurement methods using soft measurement can be employed: estimating key variables using easily measurable auxiliary variables. In wastewater treatment systems, soft measurement techniques can be used to monitor key variables in product production to control product quality and reduce the cost of traditional hardware measurements.

[0004] Currently, BOD5 soft sensing modeling methods mainly include mechanistic modeling, empirical modeling, and data-driven modeling. Mechanistic modeling is a white-box model, which expresses all reactions occurring in the modeling object using formulas and equations to reveal the relationships between variables, calculating based on the relationships between easily measurable and difficult-to-measurable variables. Mechanistic models require knowledge of the chemical and physical relationships between parameters, numerous chemical and physical formulas, and operational data, resulting in high complexity, large computational load, high knowledge requirements for researchers, and low model accuracy. Empirical models only need to focus on the formulas representing the relationships between input and output variables to describe system behavior; however, these models only focus on describing stationary states and struggle to handle complex and non-stationary situations, limiting their applicability and accuracy. Data-driven modeling is a black-box model, which does not require knowledge of the internal mechanisms between parameters. It describes the relationships between input and output variables, learning these relationships to construct the output variables. Its inputs typically consist of easily measurable, low-cost, and readily available variables, while the output variables are those that are difficult to measure and expensive to collect, making it well-suited for wastewater treatment.

[0005] Data-driven modeling methods mainly include multivariate statistical methods and artificial neural network methods. BOD5 concentration in wastewater treatment is influenced by multiple factors and exhibits strong nonlinear characteristics. Neural networks, as a nonlinear science, are suitable for studying objects with multifactorial, uncertain, random, and nonlinear characteristics. However, current methods, due to the serial computation mechanism of neural networks, can only process data step by step and cannot simultaneously link all historical information of all auxiliary variables with the key variable. Therefore, they cannot accurately reflect the weight relationship between auxiliary and key variables, resulting in unsatisfactory soft measurement results. Summary of the Invention

[0006] To address the inaccuracy of soft measurement results caused by the nonlinearity and time-dependent nature of BOD5 soft measurement data, this invention provides a model combining a Transformer network and a Long Short-Term Memory (LSTM) network (LSTM-former) for soft measurement of the wastewater effluent indicator BOD5. By incorporating the LSTM network into the Q, K, and V operations of the Transformer's multi-head attention layer, and processing the data information sequentially through LSTM and multi-head attention mechanisms, the problem of inaccurate weight allocation in BOD5 soft measurement is further resolved. Compared to other neural network models, our method is unique in that it leverages the Transformer's multi-head attention mechanism to better focus on the weight relationship between auxiliary variables and BOD5, capturing key features in the BOD5 concentration time series data. Simultaneously, it utilizes LSTM to maintain attention to the long-term dependencies of each auxiliary variable in the time series data, compensating for the insufficient attention to historical information of single variables in traditional Transformer models.

[0007] The present invention adopts the following technical solution and implementation steps:

[0008] S1: Obtain continuous time data of multiple easily measurable effluent parameters and effluent parameters that affect BOD5 after a wastewater treatment plant has treated wastewater, and construct the original dataset of wastewater indicators.

[0009] S2: Preprocess the initial dataset of wastewater indicators, correct outliers, fill data gaps using interpolation, and construct a complete dataset of wastewater indicators that can be used for soft measurement.

[0010] S3: Based on the preprocessed dataset, divide it proportionally to construct training and test sets. Use the training set data to fine-tune the parameters of the LSTM-former model, including the number of attention head layers, model dimensions, dimensions of Q, K, and V in the attention mechanism, number of neurons, random inactivation rate, initial learning rate, number of iterations, and batch size.

[0011] S4: Construct a parameter-tuned LSTM-former model that can be used for soft measurement of BOD5 concentration, determine the optimization algorithm, select the activation function, and ensure dynamic consistency between the training sequence and the original real sequence.

[0012] S5: Based on the trained LSTM-former soft sensor model, predict the concentration of BOD5 on the test set. Then, evaluate the model's performance based on the predicted values ​​and the actual values.

[0013] Specifically, influenced by different weather conditions and precipitation, the complexity of BOD5 concentration increases with the severity of weather. Therefore, this experiment selected a wastewater effluent index dataset during stormy weather to perform soft-scale BOD5 concentration measurement under the corresponding weather conditions. The dataset includes the target variable BOD5 concentration data every fifteen minutes, as well as other relevant variables such as soluble non-degradable organic matter, soluble rapidly biodegradable organic matter, insoluble particulate non-degradable organic matter, insoluble slowly degradable organic matter, biomass of active heterotrophic bacteria, biomass of active autotrophic bacteria, inert substances produced by biological solid decay, dissolved oxygen, nitrate nitrogen, ammonia nitrogen, soluble biodegradable organic nitrogen, particulate biodegradable organic nitrogen, alkalinity, and total suspended solids.

[0014] Furthermore, outliers and missing values ​​are first examined, then outliers are deleted, and the missing values ​​are filled in the dataset using median interpolation.

[0015] Preferably, 85% of the dataset is divided into the training set and the remaining 15% into the test set. The number of attention head layers is 4, the model dimension is 512, the dimensions of Q, K, and V in the attention mechanism are 64, the number of neurons is 64, the random inactivation rate is 0.1, the learning rate is 0.001, the number of iterations is 200, and the batch size is 128.

[0016] Furthermore, the model comprises a basic LSTM framework, a novel Transformer framework with a multi-head attention mechanism, an encoding / decoding input / output strategy, and fully connected layers. The gate structure within the LSTM cell unit consists of three parts: a forget gate, an input gate, and an output gate. These gates are responsible for updating the cell state and hidden layer state. The multi-head attention layer employs a novel computational mechanism. After decoding the input sequence, the output variables Q, K, and V are processed by the LSTM to form new Q, K, and V. These are then processed through the original multi-head attention mechanism's dot product calculation, normalization, and other related operations to form the final attention head output. Based on the Transformer's causal computation mechanism, Q, K, and V are first processed as LSTM input, allowing the model to first analyze the long-term dependencies in the historical data of each auxiliary variable locally. Then, the original attention mechanism globally considers the weight relationship between the auxiliary and target variables, improving the modeling ability for system dynamics. The encoding / decoding input / output structure realizes the model's input and output of the target sequence. The fully connected layer is responsible for the final output of the soft sensor model of this invention.

[0017] Preferably, the loss function in the soft measurement model is the standard normalized MSE, the activation function is the RelU function, and the training process in S4 is performed using the Adam function.

[0018] Compared with the prior art, the beneficial effects of the present invention are:

[0019] (1) By integrating the Long Short-Term Memory network into the multi-head attention layer of the Transformer network, the ability to analyze the historical information of each auxiliary variable during model training is enhanced, which solves the problem of insufficient attention to the historical information of a single auxiliary variable in the traditional Transformer.

[0020] (2) The ingenious and unique LSTM-former structure design enables the model to take into account the influence of auxiliary variables on key variables globally by utilizing the parallel computing capability of Transformer, and to focus on the long-term dependency relationship in the sequence of each auxiliary variable by using LSTM, so as to capture the data features of BOD5 more accurately and improve the accuracy of the model when performing soft measurement on data with long-term dependency characteristics. Attached Figure Description

[0021] Figure 1 This is a diagram showing the overall structure of the soft measurement network proposed in the method of this invention.

[0022] Figure 2 This is a diagram of the LSTM-former network structure proposed in the method of this invention;

[0023] Figure 3 The flowchart of the operation in the new multi-head attention layer proposed in this invention;

[0024] Figure 4 This is a schematic diagram comparing the soft measurement results of the method of the present invention. Detailed Implementation

[0025] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0026] This invention proposes a novel soft measurement framework, LSTM-former, for multivariate time series data. The core idea of ​​LSTM-former is to process information sequentially through LSTM and a multi-head attention mechanism. This allows for the analysis of historical information for each auxiliary variable while also considering all positions in the input sequence globally, fully utilizing information from the entire sequence. This improves the model's ability to capture key features in BOD5 concentration time series data, thereby accurately assigning weights between auxiliary variables and BOD5. Finally, a fully connected network is constructed to output the BOD5 concentration value, achieving precise soft measurement of BOD5 concentration.

[0027] As in this invention Figure 1 The overall flowchart for soft measurement of BOD5 concentration is shown, mainly consisting of the following steps: S1: Obtain continuous time data of multiple easily measurable effluent parameters and effluent parameters affecting BOD5 from a wastewater treatment plant after wastewater treatment, constructing an original wastewater indicator dataset. S2: Preprocess the constructed initial wastewater indicator dataset, correct outliers, and fill data gaps using interpolation to construct a complete wastewater indicator dataset suitable for soft measurement. S3: Divide the preprocessed dataset proportionally to construct training and test sets. Optimize the parameters of the LSTM-former model using the training set data, including the number of attention head layers, model dimensions, dimensions of Q, K, and V in the attention mechanism, number of neurons, random inactivation rate, initial learning rate, number of iterations, and batch size. S4: Construct a parameter-optimized LSTM-former model suitable for soft measurement of BOD5 concentration, determine the optimization algorithm, select the activation function, and ensure dynamic consistency between the training sequence and the original real sequence. S5: Based on the trained LSTM-former soft measurement model, predict the BOD5 concentration on the test set. The model's performance is then evaluated based on the predicted and actual values.

[0028] S1: Affected by different weather conditions and precipitation, the complexity of BOD5 concentration increases with the severity of weather conditions, such as sunny days, rainy days, and heavy rain days. This experiment selected the wastewater effluent index dataset during stormy weather, which has the highest data complexity, and performed soft measurement of BOD5 concentration under the corresponding weather conditions. Data sets collected from a wastewater treatment plant in Beijing after wastewater treatment were obtained. These datasets include the target variable BOD5 concentration data every fifteen minutes, as well as other relevant auxiliary variables such as soluble non-degradable organic matter, soluble rapidly biodegradable organic matter, insoluble particulate non-degradable organic matter, insoluble slowly degradable organic matter, biomass of active heterotrophic bacteria, biomass of active autotrophic bacteria, inert substances produced by biological solid decay, dissolved oxygen, nitrate nitrogen, ammonia nitrogen, soluble biodegradable organic nitrogen, particulate biodegradable organic nitrogen, alkalinity, and total suspended solids.

[0029] These effluent indicators are stipulated by international standards established by the International Organization for Standardization (ISO) (ISO 14001 Environmental Management System and ISO 55001 Asset Management Standard), and relevant standards established by China's Urban Wastewater Treatment Plant Pollutant Discharge Standard (GB18918-2002) and Integrated Wastewater Discharge Standard (GB8978-1996). Except for BOD5, these variables are all effluent indicators directly measured by hardware sensors during actual wastewater treatment, but they cannot be used as key variables to determine the quality of wastewater treatment. Specific details are shown in Table 1 below:

[0030] Table 1. Variables used in the soft measurement of BOD5 concentration in this invention.

[0031]

[0032]

[0033] S2: Due to the uncertainties of data acquisition equipment malfunctions, the acquired data will always contain some missing values ​​and outliers, which will affect the accuracy of the model. Our invention proposes a bit-level interpolation method for data preprocessing. First, outliers are removed, and then all existing missing values ​​are identified. These values ​​need to be processed. Our method uses the sliding window principle combined with contextual data at the missing points to fill in the median of the missing data. This process continues from the first missing data sampling point to the last missing data sampling point. Then, we construct an M*N (M>0, N>0) dimensional initial dataset based on the data scale acquired in S1, where M is the sum of the auxiliary variables and the target variable in S1, which is 14, and N is the total number of data samples acquired. In this invention, N is 1344. The input data can be represented by the following formula:

[0034]

[0035] The specific form is

[0036]

[0037]

[0038] The input at time t is: X t =[x 2t x 2t …x 14t ] T

[0039] S3: Divide the dataset established in S2 into a training set and a test set, with the training set comprising 85% and the remainder as the test set. The former is used to build the model, while the latter is used to evaluate the model's generalization ability when predicting unknown samples. When the model's generalization ability does not meet the requirements (this experiment uses R...), 2 If the accuracy is below 80%, we need to adjust the network's hyperparameters. The specific steps are as follows:

[0040] STEP 1: Denote the neuron input as Z, with each dimension as Zi, corresponding to the weights v that can enhance or weaken the input signal. i This achieves a non-linear mapping to [0,1] or [-1,1]. The specific number of neurons will be adjusted based on the actual feedback from the model. To avoid making the overall model too complex, the number of neurons should be less than 100, using 2n as an example. n The model exhibits a consistent variation in number of neurons, and the number of neurons is selected based on the highest model accuracy. In this example, the number of neurons is determined to be 64. Increasing the number of neurons generally improves the model's accuracy, but if the number is increased too much, the model may overfit.

[0041] STEP 2: Dropout is a regularization method for neural network models. During training, it enhances the model's generalization ability by randomly ignoring some neurons. In deep network training, a portion of neurons are randomly "temporarily dropped" with a certain probability. For example, if dropout is 0.1, then 10% of the nodes are dropped. Specifically, dropout is applied to each mini-batch of training data. Due to its mechanism of randomly dropping some neurons, it's equivalent to training a neural network with a different structure in each iteration. In this example, to prevent overfitting but also to avoid dropping too many neurons, dropout is set to 0.1.

[0042] STEP 3: Select a learning rate scheme. As an important hyperparameter in supervised learning and deep learning, the learning rate determines whether and when the objective function converges to a local minimum. A suitable learning rate enables the objective function to converge to a local minimum within a reasonable time. To avoid the model's loss function changing too slowly and getting stuck in a local minimum, or the loss function failing to converge, we have developed a fractional incremental learning rate mitigation scheme. The initial learning rate α0 is set to 0.001, and the learning rate decreases according to the following formula:

[0043] α=α0 / (1+kt)

[0044] Where k is a hyperparameter used to control the rate of decrease in the learning rate. Each change in α is above 90% of the previous value, so in this example, k is set to 0.1. t is the number of training epochs, which is set to 200 to match the number of epochs.

[0045] STEP 4: Determine an appropriate number of epochs. One epoch is defined as when a complete dataset passes through the neural network once and returns correctly. The main purpose of setting epochs is to divide the entire training process into several segments, allowing for better observation and adjustment of the model's training. By iterating and comparing network accuracy by increasing the epoch by 10 each time, the improvement in network accuracy becomes less significant after reaching 200 epochs. To avoid excessively long network training times, 200 epochs were set, which also ensures the model's prediction accuracy.

[0046] STEP 5: Determine an appropriate batch size. Before using batch size, this means that during network training, all data is input into the network at once, and then their gradients are calculated and backpropagated. Set the batch size to 2. m (m>0), increment the batch size until the model accuracy no longer changes significantly. To ensure model accuracy while improving training speed, the batch size is set to 128.

[0047] S4: The main neural network for constructing the soft measurement model in this invention is LSTM-former, such as... Figure 2The text describes a Long Short-Term Memory (LSTM) network, a type of time-recurrent network commonly used in deep learning for time series prediction. The core concept of LSTM is the cell state. Through training, the gate structure controls which information to retain and which to forget. This selected information is added to the cell state through computation. As the cell state is cyclically passed between time steps, information from earlier time steps can also be used in later time steps, thus achieving long-term memory. The corresponding cell update process formula is as follows:

[0048] f t =σ(W f ·[x t h t-1 ]+b f )

[0049] Where f t Forget gate, σ is the sigmoid activation function, W f f t The weight matrix, x t Given the input at time t, h t-1 The memory at time t-1 is the hidden layer state, b f f t The bias term.

[0050] i t =σ(W i ·[x t h t-1 ]+b i )

[0051] Where i t σ is the input gate, σ is the sigmoid activation function, and W is the input gate. i For i t The weight matrix, x t Given the input at time t, b i For i t The bias term.

[0052]

[0053] Enter x in the forget gate t and hidden layer state h t-1 A new candidate value needs to be obtained by passing through a tanh layer. W c and b c

[0054] These represent the weights and biases of the candidate layer, respectively; C t This represents the output after the cell state is updated; C t-1This represents the cell state at the previous time step, and * represents the matrix element mapping multiplication operation.

[0055] o t =σ(W o ·[x t h t-1 ]+b o )

[0056] Among them o t For output gate, W o o t The weight matrix, b o o t The bias term.

[0057] h t =o t *tanh(C t )

[0058] tanh is the hyperbolic tangent function, * represents matrix element mapping multiplication, h t This indicates the final output.

[0059] The Transformer neural network is a powerful deep learning model that has proven effective in fields such as natural language processing and computer vision. The most significant feature of the Transformer is its use of self-attention, a mechanism that allows the model to consider information from all positions in the input sequence simultaneously, thus better capturing dependencies between inputs. The self-attention mechanism in the Transformer has been extended to multi-head attention. In this mechanism, the model divides the input sequence into multiple heads and applies a separate self-attention mechanism to each head. Finally, the outputs from each head are concatenated and passed to the next layer.

[0060] The following is a detailed introduction to the attention mechanism of Transformer:

[0061] Given an input sequence X = {x1, x2, ..., x...} n}, where each element x i (i is any number from 1 to n) is a vector with n elements. We will use each input vector x i This is transformed into a query vector, a key vector, and a value vector. These vectors can be calculated using matrix multiplication and a bias vector.

[0062] Query vector Q i =W Q ·x i

[0063] Key vector K i =W K ·x i

[0064] Value vector V i =W V ·x i

[0065] Among them W Q W K W V These are learnable weight matrices, and the dimensions of these vectors are the model dimensions.

[0066] Next, we will consider each query vector Q i (i is any number from 1 to n) and all key vectors K j (j is 1, 2, 3, 4...n) Calculate the dot product, for each Q i We obtain j attention scores, calculated using the following formula:

[0067]

[0068] Where d k This is the vector dimension. The purpose of the attention score is to measure the relevance between the query vector and the key vector. The dot product is divided by 0 to balance the order of magnitude of the scores. These scores are called attention weights and can be normalized using the Softmax function so that they represent the attention allocation when they sum to 1. The following is their normalization formula, obtained from the input x. i j attention weights:

[0069]

[0070] Finally, we will consider each value vector V j We sum the values ​​of the input x with the corresponding attention weights to obtain the i-th input x. i Self-attention output:

[0071] Attention(Q i ,V)=Σj AttnWeights(Q i ,K j V j

[0072] The above process is the core of the self-attention mechanism. It allows the model to extract useful information from all input positions and considers the dependencies between inputs when representing the sequence. To improve computational efficiency, the self-attention process is generally parallelized using matrix representation in practice. Therefore, the output of the entire process can be summarized by the following formula:

[0073]

[0074] In multi-head attention mechanisms, this process is applied to each attention head O of the input matrix X. i (Assuming there are a total of h attention heads, i.e., i = 1 to h): We put all the word vectors into a matrix X, and then sum them with 3 weight matrices respectively. Multiply to get Q i K i V i A matrix. Each row in matrix X represents the word vector for each word. Q i K i V i Each row in the matrix represents a Query vector, a Key vector, and a Value vector, with a vector dimension of d. k Then, applying the formula above, the attention results of h heads are concatenated to obtain the final multi-head attention result O, as shown in the formula below:

[0075]

[0076] O i =Attention(Q) i ,K i V i i = 1......h

[0077] O = Concat(O1, O2, ..., O h W O

[0078] Among them O i This is the output of the i-th attention head in the h attention heads of the multi-head attention layer, O is the final multi-head attention output, and the Concat function concatenates multiple vectors according to a specified dimension. O It is the final output weight matrix.

[0079] like Figure 3 As shown, the LSTM mechanism is integrated into the Transformer multi-head attention operation. Before the self-attention mechanism operation of Q, K, and V, they are processed by LSTM, and they are respectively used as x in the formula. t The input is fed into an LSTM, and the output is the new Q.* K * V * , and then proceed with subsequent calculations.

[0080] After setting up the core network LSTM-former, we need some optimization methods to ensure dynamic consistency between the training sequence and the original real sequence. The specific steps are as follows:

[0081] STEP 1: Select an optimization algorithm. Currently, mainstream optimization algorithms include gradient descent and Nesterov gradient acceleration. This invention uses the Adam algorithm to address the large-scale and non-stationary characteristics of the BOD5 data. The specific steps are as follows:

[0082] (1) Initialize and determine the parameters (step size), (the exponential decay rate is estimated by moment, between [0, 1)) and the stochastic objective function.

[0083] (2) Initialize the parameter vector, first-order moment vector, second-order moment vector and time step;

[0084] (3) Loop: When the parameters do not converge, iteratively update each part. That is, increment the time step by 1, update the gradient of the objective function with respect to the parameters at that time step, update the first moment estimate and the second moment estimate of the bias, then calculate the first moment estimate and the second moment estimate of the bias correction, and then use the values ​​calculated above to update the initial parameters of the model;

[0085] (4) When the parameters converge, return the model parameters.

[0086] STEP 2: Determine the activation function. The neurons in the hidden and output layers convert the weighted sum of their inputs into the neuron's output value using the activation function. To accommodate the nonlinearity of BOD5 data and improve the network's robustness, this invention employs the ReLU activation function, the formula of which is as follows:

[0087] ReLU(x) = max(0,x)

[0088] S5: Based on the trained LSTM-former soft sensor model, the PM2.5 concentration is predicted on the test set. Figure 4 A comparison chart of the generated soft measurement values ​​and the true values ​​is provided. Then, the model's performance is evaluated based on the soft measurement values ​​and the true values. The evaluation formula used in this invention is as follows:

[0089]

[0090] Where MAE is the mean absolute error and RMSE is the root mean square error. and y iThese represent the predicted and actual values, respectively. Lower MAE and RMSE indicate better model performance. Table 2 shows an evaluation example of the soft measurement of BOD5, an influent effluent indicator, under heavy rain conditions. To reduce the impact of randomness, the experiment was repeated 10 times, and the average values ​​of the two indicators were compared. The LSTM-former network used in this invention demonstrated excellent accuracy for the soft measurement of BOD5. The LSTM mechanism and multi-head attention mechanism of the model ensured its generalization ability.

[0091] Table 2: Performance evaluation of LSTM-former soft measurement after ten experiments

[0092]

[0093] The core technical challenge of this invention lies in its innovative fusion algorithm, which combines the common linear combination of LSTM and Transformer models. First, this invention ensures that the LSTM algorithm processes not the initial input dataset, but rather the three matrix variables Q, K, and V involved in the Transformer's multi-head attention mechanism. Compared to directly converting the data in the dataset into a standard matrix form, these three variables represent the dataset as a more refined word matrix, which is then used for subsequent multi-head attention calculations. Second, due to the powerful parallel computing mechanism of Transformer, integrating the LSTM algorithm into the multi-head attention mechanism preserves the overall model's characteristic of processing input information in parallel from beginning to end. This avoids the error caused by the need for LSTM to process input data serially in linear model combination, thus improving computational efficiency.

[0094] The above-disclosed embodiments are merely a few specific examples of the present invention. However, the embodiments of the present invention are not limited thereto. Any modifications, substitutions, and improvements made within the spirit and principles of the above embodiments should be included within the protection scope of the technical solution.

Claims

1. A soft measurement method for wastewater effluent index BOD5 based on Transformer and Long Short-Term Memory Network, characterized in that, Includes the following steps: S1: Obtain continuous time data of multiple easily measurable effluent parameters and effluent parameters that affect BOD5 after a wastewater treatment plant has treated wastewater, and construct the original dataset of wastewater indicators. S2: Preprocess the initial dataset of wastewater effluent indicators, correct outliers, fill data gaps using interpolation, and construct a complete wastewater effluent indicator dataset Z that can be used for soft measurement; where M is the sum of auxiliary variables and target variables, and N is the total number of data samples obtained; M is 14; ； The specific form is ； The input at time t is: ; S3: Based on the preprocessed dataset, divide it proportionally to construct training and test sets; use the training set data to fine-tune the parameters of the LSTM-former model, including the number of neurons, random inactivation rate, and learning rate. Initial learning rate Number of iterations, batch size; ； Where k is a hyperparameter used to control the learning rate. Reduce the magnitude of the reduction to 0.

1. Each transformation is above 90% of the previous one; t is the training epoch number, set to 200 to match the epoch number. S4: Construct a parameter-tuned LSTM-former model that can be used for soft measurement of BOD5 concentration. Based on the model's training performance, determine the optimization algorithm and select the activation function. This ensures dynamic consistency between the training sequence and the original ground truth sequence; the formula used for BOD5 soft measurement is as follows: First, here is the calculation formula for LSTM: ； ； ； ； ； ； Where the subscript t represents time. Indicates the input gate. Represents the Gate of Oblivion Indicates the output gate; Represents the Sigmoid function; 𝑊 𝑓 and 𝑏 𝑓 These represent the gates of forgetting. The weights and biases, 𝑊 𝑖 and 𝑏 𝑖 They represent the input gates respectively. The weights and biases, 𝑊 𝑜 and 𝑏 𝑜 They represent the input gates respectively. Weights and biases; Indicates the hidden layer state. This represents the input at the current time t, which passes through a tanh layer after passing through the forget gate to obtain a new candidate value. , 𝑐 and 𝑏 𝑐 Representing candidate layers Weights and biases; This represents the output after the cell state is updated; This indicates the cell state at the previous moment. This indicates the final output; Secondly, the calculation formula for the Transformer multi-head attention mechanism: =X· ； =X· ； =X· ； ； ； ； in , , These are three matrices obtained by transforming the input matrix X through matrix operations, and then multiplied by the trained weight matrix. , , The resulting query matrix, key matrix, and value matrix; It is the vector dimension. It is the output of the i-th attention head among the h attention heads in the multi-head attention layer. The Concat function concatenates multiple vectors according to a specified dimension. This is the final output weight matrix. This is the final multi-head attention output; finally, the LSTM mechanism is integrated into the Transformer multi-head attention operation. Before the self-attention mechanism operations on Q, K, and V, they are processed by LSTM, and these are respectively used as the formula... The input is fed into an LSTM, and the output is a new one. , , , and then perform subsequent calculations; S5: Soft measurement is performed on the test set based on the trained LSTM-former soft measurement model to obtain the soft measurement value of BOD5 concentration; and the model performance is evaluated based on the soft measurement value and the true value.

2. The soft measurement method for the wastewater effluent index BOD5 based on Transformer and Long Short-Term Memory Network as described in claim 1, characterized in that... Outliers are assigned null values, the number of missing values ​​is detected, and then all missing values ​​are filled using median interpolation, which fills in the missing values ​​by finding the value closest to the middle position of the dataset.

3. The soft measurement method for the wastewater effluent index BOD5 based on Transformer and Long Short-Term Memory Network as described in claim 1, characterized in that, In the fractional learning rate mitigation scheme, the initial learning rate is set to 0.001, and the mitigation increment is 0.

95.

4. The soft measurement method for the wastewater effluent index BOD5 based on Transformer and Long Short-Term Memory Network as described in claim 1, characterized in that, Optimization algorithm selection: Adam; Activation function selection: .

Images