TCN-LSTM water quality prediction method based on variational mode decomposition and attention mechanism

Abstract

The invention discloses a TCN-LSTM water quality prediction method based on variational mode decomposition and an attention mechanism, and belongs to the field of water quality time sequence data prediction. Comprising the following steps: (1) preprocessing water quality data; (2) carrying out VMD decomposition on the processed water quality data; (3) dividing the decomposed data into a training set, a verification set and a test set; (4) in the left branch convolution module, training set data sequentially passes through a first convolution layer, a first normalization layer, a first Dropout layer, a second convolution layer, a second normalization layer, a ReLU activation layer and a second Dropout layer; (5) at a right branch, training set data enters a convolution jump layer; and (6) adding and fusing the features of the right side branch and the features of the left side branch in an addition layer, enabling the fused features to sequentially pass through an LSTM layer, a self-attention layer, a full connection layer and a regression output layer, and outputting a prediction result of the water quality data. Water quality data of rivers, lakes and the like can be accurately predicted, and powerful support is provided for water resource management.

Description

Technical Field

[0001] The present invention belongs to the field of water quality time series data prediction, and specifically relates to a TCN-LSTM water quality prediction method based on variational mode decomposition and attention mechanism. Background Art

[0002] With the rapid development of industrialization and urbanization, water pollution problems are becoming increasingly serious. By accurately predicting the trend of water quality changes, corresponding protection measures can be taken in advance to achieve forward-looking governance. This not only improves the efficiency of governance, but also reduces the difficulty of post-governance, which is of great significance to water resource protection. In response to this problem, scholars at home and abroad have carried out relevant research and proposed many water quality prediction models. These models can be roughly divided into two categories. One is a model based on physical processes, which combines hydrodynamics with the migration and transformation process of pollutants and is known for its excellent transferability and interpretability. However, this model requires a large amount of monitoring and data related to hydrological factors, water quality and pollution sources, which makes it challenging to predict water quality using this method. The other is a data-driven model, that is, a machine learning model. The prediction results of the data-driven model come from long-term observations. These models are easier to transplant and require a smaller basic data set. They have been widely used in water quality prediction in recent years. However, many water quality prediction models today are based on the predictions of traditional or single neural network models. Traditional neural network models have natural disadvantages in processing complex and nonlinear water quality data, especially the insufficient ability to capture long-term dependencies and multi-frequency components, resulting in the inability to accurately capture the changing trends of water quality parameters; single neural network models have limitations in short-term characteristics, long-term dependencies, noise suppression, etc., and it is difficult to fully meet the characteristic requirements of complex water quality data, resulting in low model prediction accuracy. Summary of the invention

[0003] The purpose of the present invention is to provide a TCN-LSTM water quality prediction method based on variational mode decomposition and attention mechanism, wherein: variational mode decomposition (VMD) is a signal processing technology based on the variational principle, which decomposes complex signals into multiple intrinsic mode functions (IMFs) to extract multi-frequency features in non-stationary and nonlinear signals; TCN (Temporal Convolutional Network) is a temporal convolutional network, a deep learning architecture based on one-dimensional convolution, designed for predicting time series data; LSTM (Long Short-Term Memory) is a long short-term memory network, an improved recurrent neural network (RNN), specially designed to capture long-term dependencies in sequence data; attention mechanism is an important method in deep learning, which allows focusing on the most relevant parts when processing inputs. Its core idea is to imitate the attention mechanism of humans and focus more computing resources on key features. The above modules work together to solve the technical deficiencies and low prediction accuracy of traditional water quality prediction methods.

[0004] The technical solution adopted by the present invention is: a TCN-LSTM water quality prediction method based on variational mode decomposition and attention mechanism, comprising the following steps:

[0005] (1) Preprocess water quality data;

[0006] (2) Perform VMD decomposition on the processed water quality data;

[0007] (3) Divide the decomposed data into training set, validation set and test set;

[0008] (4) The training set data is input into the left branch convolution module of the TCN neural network. The training set data first passes through the first convolution layer to extract local time series features;

[0009] (5) After the first convolution layer, the first normalization layer is entered to normalize the convolution result;

[0010] (6) After the first normalization layer, the data enters the first Dropout layer, which uses Dropout to randomly discard the outputs of some neurons in this layer. Subsequently, the data passes through the second convolutional layer and the second normalization layer in sequence;

[0011] (7) The output data of the second normalization layer enters the ReLU activation layer and then enters the second Dropout layer;

[0012] (8) Inputting the training set data of step (3) into the right branch convolutional skip module of the TCN neural network, in which the training set data enters the convolutional skip layer to form a skip connection;

[0013] (9) The output features of the second Dropout layer and the output features of the convolutional jump layer simultaneously enter the addition layer for addition fusion; the fused features pass through the LSTM layer, which is used to capture long-term dependencies in time series data;

[0014] (10) The features output by the LSTM layer pass through the self-attention layer to assign attention weights to different time steps;

[0015] (11) After passing through the fully connected layer, the multi-dimensional features extracted from the attention layer are integrated into an output vector of fixed size;

[0016] (12) Finally, the data output by the fully connected layer passes through the regression output layer to output the prediction results of the water quality data and is displayed in the test set prediction result diagram.

[0017] Step (1) is as follows: obtaining the water quality index data of a certain water area over a continuous period of time, including 11 indexes, including water temperature, pH, dissolved oxygen, total phosphorus, total nitrogen, ammonia nitrogen, turbidity, chlorophyll, permanganate, algae density and conductivity, and then performing preprocessing such as deleting outliers and interpolating missing values;

[0018] Step (3) is as follows: the decomposed data set is divided into 8:1:1 ratios, i.e., 80% training set, 10% validation set, and 10% test set.

[0019] Step (2) is as follows:

[0020] (21) Determine the modal number S of the decomposition, that is, the number of IMFs obtained;

[0021] (22) Initialize the center frequency ω of each mode s and the initial value u of each mode s ;

[0022] (23) Set the convergence condition; the formula is as follows:

[0023]

[0024] Where: u s (t) is the sth mode function; ω s is the center frequency of each mode; S is the number of modes; is the time derivative; * is the convolution operation; j is the imaginary unit; δ(t) is the Dirac function used to calculate the instantaneous frequency; t represents the time variable.

[0025] Step (4) is as follows:

[0026] (41) Setting convolution parameters;

[0027] (42) Fill in input data;

[0028] (43) Perform convolution operation; the formula is as follows:

[0029] x1=W1*x+b1

[0030] Where: x is the input data of the training set; x1 is the output value of the first convolution layer; W1 is the first convolution kernel; b1 is the first layer bias; * is the convolution operation;

[0031] Step (5) is as follows:

[0032] (51) Calculate the batch mean μ and variance σ 2 ;

[0033] (52) Normalize the input;

[0034] (53) Output the normalized result; the formula is as follows:

[0035]

[0036] Among them: x1 is the output value of the first convolutional layer; x2 is the output value of the first normalization layer; μ, σ 2 are the batch mean and variance respectively; ε is a constant, its range is 10 -5 ~10 -8 ; γ, β are learnable parameters.

[0037] Step (6) is as follows:

[0038] (61) The first Dropout layer randomly “discards” a portion of the neurons in this layer. The formula is as follows:

[0039] x3=Dropout(x2)=x2·mask2

[0040] Among them: x3 is the output of the first Dropout layer; x2 is the output of the first normalization layer; mask2 is a binary matrix with the same shape as x2, and each element is randomly 0 or 1 according to the set dropout probability.

[0041] (62) Then the second convolution operation is performed to further process the extracted features; the formula is as follows:

[0042] x4=W2*x3+b2

[0043] Among them: x4 is the output of the second convolutional layer; x3 is the output of the first Dropout layer; W2 is the second convolution kernel; b2 is the bias term of the second layer; * is the convolution operation.

[0044] (63) The data then passes through the second normalization layer, the formula of which is as follows:

[0045]

[0046] Among them: x5 is the output of the second normalization layer; x4 is the output value of the second convolutional layer; μ, σ 2 are the batch mean and variance respectively; ε is a constant whose range is 10 -5 ~10 -8 , used to prevent division by zero; γ and β are learnable parameters.

[0047] Step (7) is as follows:

[0048] (71) The ReLU activation function introduces nonlinearity and sets negative values ​​to zero. Its formula is as follows:

[0049] x6=ReLU(x5)=max(0,x5)

[0050] Among them: x6 is the output value of the ReLU activation layer; x5 is the output of the second normalization layer; ReLU is an activation function that sets negative values ​​to 0 and keeps positive values ​​unchanged;

[0051] (72) The second Dropout layer further prevents overfitting, and its formula is as follows:

[0052] x7=Dropout(x6)=x6·mask

[0053] Among them: x7 is the output of the second Dropout layer; x6 is the output value of the ReLU activation layer; mask is a binary matrix with the same shape as x6, and each element is randomly 0 or 1 according to the set drop probability.

[0054] Step (8) is as follows:

[0055] (81) Inputting the training set data of step (3) into the convolutional skip layer of the right branch convolutional skip module of the TCN neural network to generate a skip connection;

[0056] (82) The formula for the output of the convolutional jump layer is as follows

[0057] x skip =W skip *x+b skip

[0058] Where: x skipis the output of the convolutional jump layer; x is the original input data; W skip is the convolution kernel of the convolutional skip layer, b skip is the bias term of the skip convolutional layer.

[0059] Step (9) is as follows:

[0060] (91) The addition layer is used to add the output of the second Dropout layer on the left and the output of the convolutional jump layer on the right to achieve feature fusion. The formula is as follows:

[0061] x add =x7+x skip

[0062] Where: x add is the output of the addition layer; x7 is the output of the second Dropout layer; x skip is the output of the convolutional skip layer.

[0063] (92) The added data is then input into the LSTM layer of the LSTM network, and each unit of the LSTM layer performs the following steps in sequence:

[0064] (921) The forget gate calculates an output value that controls the degree of forgetting through the output value at the previous moment and the current input value;

[0065] f t =σ(W f ·[h t-1 ,x t ]+b f )

[0066] (922) The input gate combines the output value at the previous moment and the current input value to calculate the output value of the input gate and the candidate unit state;

[0067] i t =σ(W i ·[h t-1 ,x t ]+b i )

[0068]

[0069] (923) When updating the current cell state, the output of the forget gate is multiplied by the state at the previous moment, and then the output of the input gate is multiplied by the candidate cell state to obtain the current cell state, that is:

[0070]

[0071] (924) Output goalkeeper h t-1 and input x tThe output gate is obtained by taking the input value of the input gate at time t:

[0072] o t =σ(W o ·[h t-1 ,x t ]+b o )

[0073] h t =o t tanh(C t )

[0074] In the formula, f t is the forget gate; σ is the standard sigmoid activation function; W is the weight matrix; h t-1 is the unit of the previous moment; x t is the input vector at time t; b is the bias term; C t is the state unit of the hidden layer at time t; is the state unit currently input; i t is the input gate; tanh(·) is the hyperbolic tangent activation function; o t is the output gate; W f is the weight matrix of the forget gate; b f is the bias term of the forget gate; W i is the weight matrix of the input gate; b i is the bias term of the input gate; W c is the weight matrix of the candidate memory unit; b c is the bias term of the candidate memory unit; W o is the weight matrix of the output gate; b o is the bias term of the output gate; h t is the result of the output gate.

[0075] Step (10) is as follows:

[0076] (101) The self-attention layer calculates the attention weights at different time steps in the input sequence, and the formula is as follows:

[0077] (102) Query, key and value vectors:

[0078] Q=W Q ·x lstm , K=W K ·x lstm , V=W V ·x lstm

[0079] (103) Self-attention weight calculation:

[0080]

[0081] Where: Q is the query matrix; K is the key matrix; V is the value matrix; x lstm is the output of the LSTM layer; d k is the dimension of the key vector; softmax is the normalization function.

[0082] Step (11) is as follows:

[0083] (111) The fully connected layer maps the features extracted from the self-attention layer to the output space, and its formula is as follows:

[0084] x fc =W fc ·x attention +b fc

[0085] Where: x fc is the output of the fully connected layer; W fc is the weight matrix of the fully connected layer; b fc is the bias term; x attention is the output of the self-attention layer;

[0086] Step (12) is as follows:

[0087] (121) The regression output layer is used to generate continuous prediction values. This layer has no activation function, so the output can be any real value. Its formula is as follows:

[0088]

[0089] in: Represents the prediction result of the input time series data; x fc is the output of the fully connected layer.

[0090] Compared with the prior art, the present invention has the following beneficial effects:

[0091] The VMD module in the present invention has significant advantages in processing non-stationary water quality time series data. By decomposing the original water quality signal into a set of intrinsic mode functions (IMFs) through VMD, noise can be effectively removed, and complex fluctuations can be decomposed into components of different frequencies, thereby improving the ability to understand complex time series data. At the same time, TCN is responsible for extracting local features and short-term dependencies to reduce computational costs, and LSTM is responsible for capturing long-term dependencies and improving memory capabilities. In addition, the inherent noise resistance, long time series processing capabilities, gradient stability, and higher prediction accuracy make TCN-LSTM have obvious advantages in processing complex time series data, especially multi-level dependent tasks. The introduction of the self-attention mechanism enables the present invention to focus on more important information, thereby improving prediction accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0092] Figure 1 It is a schematic diagram of the process of the present invention;

[0093] Figure 2 This is a test set prediction effect diagram of the present invention. DETAILED DESCRIPTION

[0094] The technical solution of the present invention is further described below in conjunction with the accompanying drawings.

[0095] Example 1: Figure 1 As shown, the present invention provides a TCN-LSTM water quality prediction method based on variational mode decomposition and attention mechanism, comprising the following steps:

[0096] (1) Preprocessing the water quality data; specifically as follows: The water quality data of this embodiment comes from the China National Environmental Monitoring Center, which releases monitoring data every 4 hours, including 11 indicators including water temperature, pH, dissolved oxygen, total phosphorus, total nitrogen, ammonia nitrogen, turbidity, chlorophyll, permanganate, algae density and conductivity. The experimental data time range is from January 1, 2023 to December 31, 2023, with a total of 2086 data. Subsequently, outliers are deleted and missing values ​​are interpolated. The missing value interpolation adopts cubic spline interpolation to improve the quality of the data set.

[0097] (2) VMD decomposition is performed on the processed water quality data, which decomposes the complex non-stationary signal into multiple independent modal components in order to extract the characteristics of different frequencies, thereby improving the accuracy and stability of data analysis; the details are as follows:

[0098] (21) Determine the modal number S of the decomposition, that is, the number of IMFs obtained. In this experiment, the modal number is set to 8;

[0099] (22) Initialize the center frequency ω of each mode s and the initial value u of each mode s ;

[0100] (23) Set the convergence condition; the formula is as follows:

[0101]

[0102] Where: u s (t) is the sth mode function; ω s is the center frequency of each mode; S is the number of modes; is the time derivative; * is the convolution operation; j is the imaginary unit; δ(t) is the Dirac function used to calculate the instantaneous frequency; t represents the time variable.

[0103] (3) Divide the decomposed data into training set, validation set and test set; specifically, divide the decomposed data set into a ratio of 8:1:1, i.e., 80% training set, 10% validation set and 10% test set; the training set is used for learning, the validation set is used for tuning and preventing overfitting, and the test set is used to evaluate the final performance.

[0104] (4) Input the training set data into the left branch convolution module of the TCN neural network. The training set data first passes through the first convolution layer, whose core function is to extract basic local features from the data, reduce the number of parameters, increase smoothness and robustness, and provide a robust feature basis for subsequent layers. The details are as follows:

[0105] (41) Setting convolution parameters;

[0106] (42) Fill in input data;

[0107] (43) Perform convolution operation; the formula is as follows:

[0108] x1=W1*x+b1

[0109] Among them: x is the input data of the training set; x1 is the output value of the first convolution layer; W1 is the first convolution kernel; b1 is the first layer bias term; * is the convolution operation.

[0110] (5) After the convolution layer, there is the first normalization layer, which normalizes the convolution result; its core function is to standardize feature distribution, reduce internal covariate shift, improve generalization ability, and accelerate convergence speed. The details are as follows:

[0111] (51) Calculate the batch mean μ and variance σ 2 ;

[0112] (52) Normalize the input;

[0113] (53) Output the normalized result; the formula is as follows:

[0114]

[0115] Among them: x1 is the output value of the first convolutional layer; x2 is the output value of the first normalization layer; μ, σ 2 are the batch mean and variance respectively; ε is a small constant, its range is 10 -5 ~10 -8 , used to prevent division by zero; γ and β are learnable parameters used to adjust the standardized output.

[0116] (6) Next is the first Dropout layer, which randomly discards the output of some neurons in this layer to prevent overfitting. Subsequently, the data passes through the second convolutional layer and the second normalization layer in turn to further explore more important features. The details are as follows:

[0117] (61) The first Dropout layer randomly “discards” (i.e. sets to 0) a portion of the neurons in this layer to avoid excessive co-adaptation between neurons and reduce over-reliance on specific features of the training data, thereby reducing the risk of overfitting. The formula is as follows:

[0118] x3=Dropout(x2)=x2·mask2

[0119] Among them: x3 is the output of the first Dropout layer; x2 is the output of the first normalization layer; mask2 is a binary matrix with the same shape as x2, and each element is randomly 0 or 1 according to the set dropout probability.

[0120] (62) Then the convolution operation is continued to further process the extracted features; the formula is as follows:

[0121] x4=W2*x3+b2

[0122] Among them: x4 is the output of the second convolution layer; x3 is the output of the first Dropout layer; W2 is the second convolution kernel; b2 is the bias term of the second layer; * is the convolution operation.

[0123] (63) The data then passes through the second normalization layer to further normalize the data distribution and balance the scale of the feature values. Multiple normalizations make the release of deep features stable and can effectively suppress the internal covariate shift problem during training. The formula is as follows:

[0124]

[0125] x5 is the output of the second normalization layer; x4 is the output value of the second convolutional layer; μ, σ 2 are the batch mean and variance respectively; ε is a small constant, its range is 10 -5 ~10 -8 , used to prevent division by zero; γ and β are learnable parameters.

[0126] (7) The ReLU activation layer introduces nonlinearity; finally, the second Dropout layer is used to further prevent overfitting and pass the features of the left convolution module to the subsequent layers. The details are as follows:

[0127] (71) The ReLU activation function introduces nonlinearity, enabling it to learn complex data distributions. By setting negative values ​​to zero, the network's expressive power is enhanced. Its formula is as follows:

[0128] x6=ReLU(x5)=max(0,x5)

[0129] Among them: x6 is the output value of the ReLU activation layer; x5 is the output of the second normalization layer; ReLU is an activation function that sets negative values ​​to 0 and keeps positive values ​​unchanged.

[0130] (72) The second Dropout layer further prevents overfitting. This layer can be regarded as the “summary” layer of the left branch of the TCN, which makes an important contribution to the subsequent addition layer. Its formula is as follows:

[0131] x7=Dropout(x6)=x6·mask6

[0132] Among them: x7 is the output of the second Dropout layer; x6 is the output value of the ReLU activation layer; mask6 is a binary matrix with the same shape as x6, and each element is randomly 0 or 1 according to the set dropout probability.

[0133] (8) Input the training set data of step (3) into the right branch convolutional jump module of the TCN neural network. In the right branch convolutional jump module, the training set data enters the convolutional jump layer to form a jump connection to prevent the gradient from disappearing and retain important information. The details are as follows:

[0134] (81) Convolutional skip layers are used to generate skip connections, which pass input features directly to subsequent layers. This skip connection can help preserve the information in the initial input and alleviate the problem of information loss during the transmission process in the deep layers of the network.

[0135] (82) The formula for the output of the convolutional skip layer is as follows:

[0136] x skip =W skip *x+b skip

[0137] Where: x skip is the output of the convolutional jump layer; x is the original input data; W skip is the convolution kernel of the convolutional skip layer; b skip is the bias term of the skip convolutional layer.

[0138] (9) In the addition layer, the features output by the right convolutional jump layer are added and fused with the final output features of the left branch. The fused features are passed through the LSTM layer. LSTM is used to capture long-term dependencies in time series data. The details are as follows:

[0139] (91) The addition layer is used to add the output of the left branch (the features obtained after multiple layers of convolution and nonlinear activation, that is, the output of the second layer of Dropout layer) and the output of the right skip connection branch to achieve feature fusion, which helps to improve the prediction ability. The formula is as follows:

[0140] x add =x7+x skip

[0141] Where: x add is the output of the addition layer; x7 is the output of the second Dropout layer; x skip is the output of the convolutional skip layer.

[0142] The added data is then input into the LSTM network. The LSTM (Long Short-Term Memory) layer is used to capture long-term dependencies in the sequence. It is a special recurrent neural network (RNN) that is very suitable for time series prediction. The LSTM layer controls the flow of information through memory units and gating mechanisms (input gate, forget gate, and output gate), so that the present invention can maintain important information over a longer time step and avoid the gradient vanishing problem. The LSTM layer uses the results of the previous feature fusion to further learn the long-term patterns and dependencies in the time series features. Each unit performs the following steps in sequence:

[0143] (921) The forget gate calculates an output value that controls the degree of forgetting through the output value at the previous moment and the current input value.

[0144] f t =σ(W f ·[h t-1 ,x t ]+b f )

[0145] (922) The input gate combines the output value at the previous moment and the current input value to calculate the output value of the input gate and the candidate unit state.

[0146] i t =σ(W i ·[h t-1 ,x t ]+b i )

[0147]

[0148] (923) When updating the current cell state, the output of the forget gate is multiplied by the state at the previous moment, and then the output of the input gate is multiplied by the candidate cell state to obtain the current cell state, that is:

[0149]

[0150] (924) Output goalkeeper h t-1 and input x t The output gate is obtained by taking the input value of the input gate at time t:

[0151] o t =σ(W o ·[h t-1 ,x t ]+b o )

[0152] h t =o t tanh(C t )

[0153] In the formula, f t is the forget gate; σ is the standard sigmoid activation function; W is the weight; h t-1 is the unit of the previous moment; x t is the input vector at time t; b is the bias value vector; C t is the state unit of the hidden layer at time t; is the state unit currently input; i t is the input gate; tanh(·) is the hyperbolic tangent activation function; o t is the output gate. f is the weight matrix of the forget gate; b f is the bias term of the forget gate; W i is the weight matrix of the input gate; b i is the bias term of the input gate; W c is the weight matrix of the candidate memory unit; b c is the bias term of the candidate memory unit; W o is the weight matrix of the output gate; b o is the bias term of the output gate; h t is the result of the output gate.

[0154] (10) The features output by the LSTM layer pass through the self-attention layer, which enhances the ability to capture global dependencies by assigning attention weights to different time steps. The details are as follows:

[0155] (101) The self-attention layer can allocate more attention to important time steps in the sequence by calculating the attention weights of different time steps in the input sequence, which helps to capture long-range dependencies and is not limited to local dependencies in the sequence, thereby improving prediction accuracy. Its formula is as follows:

[0156] (102) Query, key and value vectors:

[0157] Q=W Q ·xlstm , K=W K ·x lstm , V=W V ·x lstm

[0158] (103) Self-attention weight calculation:

[0159]

[0160] Where: Q is the query matrix; K is the key matrix; V is the value matrix; x lstm is the output of the LSTM layer; d k is the dimension of the key vector, which is used for scaling; softmax is a normalization function used to generate attention weights.

[0161] (11) After passing through the fully connected layer, the multi-dimensional features extracted from the self-attention layer are integrated into an output vector of a fixed size, ready to generate the final prediction result. The details are as follows:

[0162] (111) The fully connected layer maps the features extracted from the self-attention layer to the output space. It integrates all the features extracted in the previous layers and is a bridge connecting the feature extraction part and the output part. Its formula is as follows:

[0163] x fc =W fc ·x attention +b fc

[0164] Where: x fc is the output of the fully connected layer; W fc is the weight matrix of the fully connected layer; b fc is the bias term; x attention is the output of the self-attention layer.

[0165] (12) Finally, the data passes through the regression output layer, outputs the prediction results of the water quality data, and is displayed in the test set prediction result graph. The details are as follows:

[0166] (121) The regression output layer is used to generate continuous prediction values. This layer has no activation function, so the output can be any real value. Its formula is as follows:

[0167]

[0168] in: Represents the prediction result of the input time series data; x fc is the output of the fully connected layer.

[0169] After multiple experiments, the prediction performance of the proposed method on the validation set and the test set is as follows: the average value of the mean absolute error (MAE) is maintained at about 0.12, the average value of the root mean square error (RMSE) is maintained at about 0.16, and the coefficient of determination (R 2 )The average value remains around 0.97, which has a high prediction accuracy.

[0170] like Figure 2 As shown in the figure, the performance evaluation results of the TCN-LSTM water quality prediction method based on variational mode decomposition and attention mechanism on the test set are shown, covering 100 sets of data. In the figure, the red curve represents the real water quality data value, and the blue curve represents the predicted value. It can be intuitively seen from the graph that the predicted value and the real value are highly consistent in trend, indicating that the present invention can well learn and capture the changing law of water quality data. Especially in the peak and valley parts of the data, the prediction results of the present invention can closely fit the real value, which shows its accuracy in processing data fluctuations. In addition, the blue prediction curve is relatively smooth, without severe jitter phenomenon, indicating that the present invention has a certain noise resistance in the prediction process, and can effectively filter out the random noise that may exist in the data. This noise resistance property enables the present invention to extract more effective information and key features from the data, rather than being interfered by noise. It is worth noting that although there is a small amount of error, this error has little effect on the judgment of the overall trend. In general, the present invention performs well in processing complex patterns and nonlinear fluctuations, can capture the detailed changes of water quality data, and takes into account both stability and accuracy. This result shows that the present invention has strong practicality and wide application value in water quality prediction tasks, and can provide reliable support, especially in scenarios where rapid response or stable prediction is required in practice.

[0171] The invention should not only be able to quickly respond to and process a large amount of real-time data, but also have a high degree of accuracy to capture long-term trends and short-term fluctuations in water quality and prevent sudden pollution incidents. The method can accurately predict water quality data of rivers, lakes, etc., providing strong support for water resources management.

[0172] The specific implementation modes of the present invention are described in detail above in conjunction with the accompanying drawings, but the present invention is not limited to the above implementation modes, and various changes can be made within the knowledge scope of ordinary technicians in this field without departing from the purpose of the present invention.

Claims

1. A TCN-LSTM water quality prediction method based on variational mode decomposition and attention mechanism, characterized by: The following steps are involved: (1) Preprocess water quality data; (2) Perform VMD decomposition on the processed water quality data; (3) Divide the decomposed data into training set, validation set and test set; (4) The training set data is input into the left branch convolution module of the TCN neural network. The training set data first passes through the first convolution layer to extract local time series features; (5) After the first convolution layer, the first normalization layer is entered to normalize the convolution result; (6) After the first normalization layer, the data enters the first Dropout layer, which uses Dropout to randomly discard the outputs of some neurons in this layer. Subsequently, the data passes through the second convolutional layer and the second normalization layer in sequence; (7) The output data of the second normalization layer enters the ReLU activation layer and then enters the second Dropout layer; (8) Inputting the training set data of step (3) into the right branch convolutional skip module of the TCN neural network, in which the training set data enters the convolutional skip layer to form a skip connection; (9) The output features of the second Dropout layer and the output features of the convolutional jump layer simultaneously enter the addition layer for addition fusion; the fused features pass through the LSTM layer, which is used to capture long-term dependencies in time series data; (10) The features output by the LSTM layer pass through the self-attention layer to assign attention weights to different time steps; (11) After passing through the fully connected layer, the multi-dimensional features extracted from the attention layer are integrated into an output vector of fixed size; (12) Finally, the data output by the fully connected layer passes through the regression output layer to output the prediction results of the water quality data and is displayed in the test set prediction result diagram.

2. A TCN-LSTM water quality prediction method based on variational mode decomposition and attention mechanism according to claim 1, characterized in that: Step (1) is as follows: obtain the water quality index data of a certain water area over a continuous period of time, including 11 indicators including water temperature, pH, dissolved oxygen, total phosphorus, total nitrogen, ammonia nitrogen, turbidity, chlorophyll, permanganate, algae density and conductivity, and then perform preprocessing by deleting outliers and interpolating missing values; step (3) is as follows: divide the decomposed data set into a ratio of 8:1:1, i.e., 80% training set, 10% validation set and 10% test set.

3. A TCN-LSTM water quality prediction method based on variational mode decomposition and attention mechanism according to claim 1, characterized in that: Step (2) is as follows: (21) Determine the modal number S of the decomposition, that is, the number of IMFs obtained; (22) Initialize the center frequency ω of each mode s and the initial value u of each mode s ; (23) Set the convergence condition; the formula is as follows: Where: u s (t) is the sth mode function; ω s is the center frequency of each mode; S is the number of modes; is the time derivative; * is the convolution operation; j is the imaginary unit; δ(t) is the Dirac function used to calculate the instantaneous frequency; t represents the time variable.

4. According to claim 1, a TCN-LSTM water quality prediction method based on variational mode decomposition and attention mechanism is characterized in that: Step (4) is as follows: (41) Setting convolution parameters; (42) Fill in input data; (43) Perform convolution operation; the formula is as follows: x1=W1*x+b1 Where: x is the input data of the training set; x1 is the output value of the first convolution layer; W1 is the first convolution kernel; b1 is the first layer bias; * is the convolution operation; Step (5) is as follows: (51) Calculate the batch mean μ and variance σ 2 ; (52) Normalize the input; (53) Output the normalized result; the formula is as follows: Among them: x1 is the output value of the first convolutional layer; x2 is the output value of the first normalization layer; μ, σ 2 are the batch mean and variance respectively; ε is a constant, its range is 10 -5 ~10 -8 ; γ, β are learnable parameters.

5. A TCN-LSTM water quality prediction method based on variational mode decomposition and attention mechanism according to claim 1, characterized in that: Step (6) is as follows: (61) The first Dropout layer randomly "discards" some neurons in this layer. The formula is as follows: x3=Dropout(x2)=x2·mask2 Among them: x3 is the output of the first Dropout layer; x2 is the output of the first normalization layer; mask2 is a binary matrix with the same shape as x2, and each element is randomly 0 or 1 according to the set dropout probability; (62) Then the second convolution operation is performed to further process the extracted features; the formula is as follows: x4=W2*x3+b2 Among them: x4 is the output of the second convolution layer; x3 is the output of the first Dropout layer; W2 is the second convolution kernel; b2 is the bias term of the second layer; * is the convolution operation; (63) The data then passes through the second normalization layer, the formula of which is as follows: Among them: x5 is the output of the second normalization layer; x4 is the output value of the second convolutional layer; μ, σ 2 are the batch mean and variance respectively; ε is a constant whose range is 10 -5 ~10 -8 , used to prevent division by zero; γ and β are learnable parameters.

6. A TCN-LSTM water quality prediction method based on variational mode decomposition and attention mechanism according to claim 1, characterized in that: Step (7) is as follows: (71) The ReLU activation function introduces nonlinearity and sets negative values ​​to zero. Its formula is as follows: x6=ReLU(x5)=max(0,x5) Among them: x6 is the output value of the ReLU activation layer; x5 is the output of the second normalization layer; ReLU is an activation function that sets negative values ​​to 0 and keeps positive values ​​unchanged; (72) The second Dropout layer further prevents overfitting, and its formula is as follows: x7=Dropout(x6)=x6·mask Among them: x7 is the output of the second Dropout layer; x6 is the output value of the ReLU activation layer; mask is a binary matrix with the same shape as x6, and each element is randomly 0 or 1 according to the set drop probability.

7. A TCN-LSTM water quality prediction method based on variational mode decomposition and attention mechanism according to claim 1, characterized in that: Step (8) is as follows: (81) Inputting the training set data of step (3) into the convolutional skip layer of the right branch convolutional skip module of the TCN neural network to generate a skip connection; (82) The formula for the output of the convolutional jump layer is as follows x skip =W skip *x+b skip Where: x skip is the output of the convolutional jump layer; x is the original input data; W skip is the convolution kernel of the convolutional skip layer, b skip is the bias term of the skip convolutional layer.

8. According to claim 1, a TCN-LSTM water quality prediction method based on variational mode decomposition and attention mechanism is characterized in that: Step (9) is as follows: (91) The addition layer is used to add the output of the second Dropout layer on the left and the output of the convolutional jump layer on the right to achieve feature fusion. The formula is as follows: x add =x7+x skip Where: x add is the output of the addition layer; x7 is the output of the second Dropout layer; x skip is the output of the convolutional skip layer; (92) The added data is then input into the LSTM layer of the LSTM network, and each unit of the LSTM layer performs the following steps in sequence: (921) The forget gate calculates an output value that controls the degree of forgetting through the output value at the previous moment and the current input value; f t =σ(W f ·[h t-1 ,x t ]+b f ) (922) The input gate combines the output value at the previous moment and the current input value to calculate the output value of the input gate and the candidate unit state; i t =σ(W i ·[h t-1 ,x t ]+b i ) C~ t =tanh(W c ·[h t-1 ,x t ]+b c ) (923) When updating the current cell state, the output of the forget gate is multiplied by the state at the previous moment, and then the output of the input gate is multiplied by the candidate cell state to obtain the current cell state, that is: (924) Output goalkeeper h t-1 and input x t The output gate is obtained by taking the input value of the input gate at time t: the t =σ(W o ·[h t-1 ,x t ]+b o ) h t =o t ·tanh(C t ) In the formula, f t is the forget gate; σ is the standard sigmoid activation function; W is the weight matrix; h t-1 is the unit of the previous moment; x t is the input vector at time t; b is the bias term; C t is the state unit of the hidden layer at time t; is the state unit currently input; i t is the input gate; tanh(·) is the hyperbolic tangent activation function; o t is the output gate; W f is the weight matrix of the forget gate; b f is the bias term of the forget gate; W i is the weight matrix of the input gate; b i is the bias term of the input gate; W c is the weight matrix of the candidate memory unit; b c is the bias term of the candidate memory unit; W o is the weight matrix of the output gate; b o is the bias term of the output gate; h t is the result of the output gate.

9. A TCN-LSTM water quality prediction method based on variational mode decomposition and attention mechanism according to claim 1, characterized in that: Step (10) is as follows: (101) The self-attention layer calculates the attention weights at different time steps in the input sequence, and the formula is as follows: (102) Query, key and value vectors: Q=W Q ·x lstm ,K=W K ·x lstm ,V=W V ·x lstm (103) Self-attention weight calculation: Where: Q is the query matrix; K is the key matrix; V is the value matrix; x lstm is the output of the LSTM layer; d k is the dimension of the key vector; softmax is the normalization function.

10. A TCN-LSTM water quality prediction method based on variational mode decomposition and attention mechanism according to claim 1, characterized in that: Step (11) is as follows: (111) The fully connected layer maps the features extracted from the self-attention layer to the output space, and its formula is as follows: x fc =W fc ·x attention +b fc Where: x fc is the output of the fully connected layer; W fc is the weight matrix of the fully connected layer; b fc is the bias term; x attention is the output of the self-attention layer; Step (12) is as follows: (121) The regression output layer is used to generate continuous prediction values. This layer has no activation function, so the output can be any real value. Its formula is as follows: in: Represents the prediction result of the input time series data; x fc is the output of the fully connected layer.

Images