A hybrid deep learning water quality prediction method based on time-frequency feature extraction

Abstract

The application relates to the technical field of data processing, and specifically discloses a mixed deep learning water quality prediction method based on time-frequency feature extraction, which comprises the following steps: step S1, collecting water quality pollutants, hydrological and meteorological hourly data; step S2, using time discrete stationary wavelet transformation to decompose the data into sub-signal sequences with unified length and different frequency domains, and meanwhile, integrating time features such as seasons and dates into the sequences to form a time series deep learning model input data set; step S3, constructing an Informer Encoder to perform sequence feature enhancement coding; step S4, strengthening the local feature correlation of a half-cascaded sequence by gradually decreasing and stacking a bidirectional LSTM; step S5, taking RMSE, MAE and MAPE as evaluation parameters; and step S6, analyzing a correlation matrix learned by the deep learning model.The mixed deep learning water quality prediction method based on time-frequency feature extraction can reduce cost and improve prediction accuracy.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and particularly relates to a mixed deep learning water quality prediction method based on time-frequency feature extraction. BACKGROUND

[0002] In the prior art, water pollution prediction is the research focus of intelligent city water environment protection and pollutant monitoring. In recent years, due to the gradual increase of industrialization and urbanization, the ecological environment quality of many river basins has decreased, seriously affecting the healthy development of the water environment of the river basin. Hydrology, meteorology and other factors are important factors affecting water environment quality, which can provide an important basis for continuous prediction of water quality changes. A large number of water quality monitoring sites have been established in key river basins in China, forming a continuous time series dataset of water quality, hydrology and meteorology, and the basic conditions for water quality prediction simulation have been met. In order to reduce the harm caused by water pollution, researchers have introduced various models to predict changes in water pollution so that appropriate decisions can be made in a timely manner. However, the transmission process and mechanism of water pollution are complex, resulting in different frequency factors mixed in the water quality time series, which makes it difficult for traditional models to predict changes in water pollution. Therefore, it is necessary to carry out research on the correlation of different frequency domains of multivariate water quality time series to improve the accuracy of water quality prediction model simulation and prediction of water quality changes. At present, common river basin water quality simulation models, such as SWAP (Water Quality Analysis Simulation Program), SWAT (Soil and Water Assessment Tool), and HSPF (Hydrological Simulation Program FORTRAN) model, can accurately simulate the transmission of pollutants in water, and the model parameters have strict physical interpretation. However, these models are based on ideal physical models. Usually, modeling needs to be based on a deep understanding of physical and chemical processes and physical factors, and the problems of difficult parameter calibration, complex model structure, numerous parameters, high computational cost, etc. limit their application in river basin water quality prediction. Traditional machine learning methods are also widely used in water quality prediction, mainly including time series ARIMA, support vector regression (SVR), multiple linear regression (MLR), and random forest (RF) model. These machine learning models can learn high-level features from raw features and improve prediction accuracy to some extent, but they have limitations in enhancing time lag or long-term dependence for time series data. As an efficient time series feature information extraction tool, deep learning models are used for water pollution prediction. For example, convolutional neural network (CNN), long short-term memory model LSTM, attention network, and Transformer, etc. These deep learning models can extract more information than physical models and machine learning, which can greatly improve the prediction accuracy of water pollution. However, water quality sequences often contain fluctuation information of different time-frequency characteristics, and deep learning models directly extracting features from raw data is not sufficient, which will seriously affect the prediction accuracy of water quality, and the model prediction behavior is difficult to explain.Fortunately, the wavelet transform (WT) can decompose time series data into two-dimensional time-frequency information, has efficient signal processing capability, and can extract local transforms in time and frequency domains at the same time. Researchers have always combined wavelet transform with deep learning models to explore water quality data rules. For example, a double-layer bidirectional Seq2Seq hybrid model (W-Bi2Seq2Seq) based on wavelet decomposition denoising and LSTM is established to predict four water quality index data of Yantai Menlou Reservoir. In order to simulate and predict the concentration of chlorophyll a (Chla) in Kunming Dianchi Lake, a long-term prediction hybrid model (WDTDLSTM-WMF) combining wavelet domain threshold denoising (WDTD), wavelet mean fusion (WMF) and long short-term memory (LSTM) is developed. A hybrid model based on synchronous wavelet transform (SWT) denoising and improved long short-term memory (LSTM) is established to predict water quality parameters such as dissolved oxygen (DO) in Yongding River Basin in China. However, only low-frequency signals are used in these hybrid models obtained by wavelet decomposition, while high-frequency signals are ignored, which has a greater impact on data-driven water quality deep learning prediction models affected by high-frequency signal fluctuations. Therefore, they can only improve the prediction accuracy to a certain extent. SUMMARY

[0003] The present application aims to at least solve one of the above-mentioned technical problems, and provides a hybrid deep learning water quality prediction method based on time-frequency feature extraction, which can reduce cost and improve prediction accuracy.

[0004] In order to achieve the above-mentioned purpose, the technical scheme adopted by the present application is as follows: a hybrid deep learning water quality prediction method based on time-frequency feature extraction, comprising the following steps:

[0005] Step S1, collecting and preprocessing water pollution, hydrology and meteorology hourly data;

[0006] Step S2, using time-discrete stationary wavelet transform to decompose the data into sub-signal sequences with uniform length and different frequency domains, and at the same time, integrating time features such as season and date into the sequence to form a time series deep learning model input data set;

[0007] Step S3, constructing an Informer Encoder to do sequence feature enhancement coding to discover multivariate global time correlation;

[0008] Step S4, strengthening the local feature correlation of the half-cascaded sequence by gradually decreasing the stack bidirectional LSTM to build an adaptive hybrid deep learning model for predicting target pollutant changes;

[0009] Step S5, using RMSE, MAE and MAPE as evaluation parameters, the parameters of the deep learning model are optimized by Adam optimizer, and the effect is verified by comparison with other benchmark models;

[0010] Step S6, analyzing the correlation matrix learned by the deep learning model, the deeper time-frequency rules of water quality and hydro-meteorological variables are obtained.

[0011] Preferably, in step S2, the time series data is separated into low-frequency components of seasonal trends and high-frequency components of short-term dramatic changes by using discrete stationary wavelet transform to construct a time-frequency filter decomposer.

[0012] Preferably, the Informer Encoder includes a multi-head Probsparse self-attention layer and a distilling layer, wherein the multi-head Probsparseself-attention layer captures different information through different spatial dimensions, and a residual connection and a normalization layer are applied after each attention block module; the distilling layer is at the end of the encoder, which is used to reduce the output length and reduce the relevant redundant features when the input time series is too long and causes information redundancy.

[0013] Preferably, in the training process of the deep learning model, the L2 regularization weight decay method, the packet loss method and the early stopping method are combined to suppress overfitting.

[0014] Preferably, in step S3, the following steps are included:

[0015] (1) After the time-frequency filter, different frequency band sequences of water quality and hydro-meteorological data are obtained, and then local time features and global time features are fused;

[0016] (2) After embedding, the input tensor X is obtained, and X is mapped to different dimensions through different linear layers to obtain Q, K and V;

[0017] (3) The main Q ～ and K are selected and measured, the weight coefficients between the variables are calculated, and then the correlation matrix is obtained by using the softmax activation function after normalization;

[0018] (4) The correlation matrix is multiplied by the V matrix, and the index positions not calculated by the Score are filled with Mean(V) to obtain enhanced data feature information. If a multi-head attention mechanism is used, x needs to be divided into nhead groups of different Q, K and V, and each group needs to be calculated by the attention mechanism.

[0019] (5) Concatenate the attention scores of each group to obtain the final enhanced feature encoding information;

[0020] (6) After the obtained feature encoding is subjected to Dropout, residual connection and normalization, Conv1d, ELU and MaxPool are sequentially performed to obtain more obvious dominant feature information and reduce the sequence length;

[0021] (7) Multi-layer multi-head self-attention mechanism and distillation operation are alternately processed.

[0022] Preferably, in the step S4, the following steps are included:

[0023] (1) The feature map of the water quality sequence is input into the first layer BiLSTM for forward and backward local feature information extraction;

[0024] (2) The sequence is continuously input into the next layer BiLSTM for the same operation until the last layer BiLSTM is processed to obtain each frequency domain time correlation matrix;

[0025] (3) The frequency domain correlation matrix after local feature extraction is subjected to a global average pooling layer (GAP) to reduce the dimension;

[0026] (4) The feature information is combined together through a fully connected layer to predict the future target water quality sequence.

[0027] The beneficial effects are that, compared with the prior art, the mixed deep learning water quality prediction method based on time-frequency feature extraction provides an intelligent hybrid model combining deep learning and time-frequency domain feature decomposition method to explore the influence of hydrology, meteorology and other factors on the water quality of the study basin, and further improves the model explainability; meanwhile, the feature extraction module combining the InformerEncoder and the multi-layer bidirectional LSTM is proposed to obtain the global feature correlation of the sequence and take into account the sequence local feature dependence enhancement; and the experimental results show that, due to the application of discrete stationary wavelet transform decomposition time sequence, the prediction precision is effectively improved and the hysteresis is reduced, so it can be seen that the prediction method has important practical value in water quality prediction real-time monitoring and forecasting, provides a new method for water environment quality long sequence data prediction and model behavior explainability, provides a strong foundation for relieving water pollution, helps to reduce cost, protect ecology and improve human health. BRIEF DESCRIPTION OF DRAWINGS

[0028] The specific embodiments of the present application are further described in detail below with reference to the accompanying drawings, in which:

[0029] Figure 1 Structure diagram of LSMT;

[0030] Figure 2 Distribution of basin monitoring stations in the selected research area of the present application;

[0031] Figure 3 Structure diagram of Informer Encoder;

[0032] Figure 4 Diagram of prediction results of EMD-IE-SBiLSTM, SWT-IE-SBiLSTM and ESWT-IE-SBiLSTM three models;

[0033] Figure 5 Diagram of attention correlation matrix found by self-attention mechanism in the encoder;

[0034] Figure 6 Diagram of prediction values and observed values of water quality index concentration predicted by ESWT-IE-SBiLSTM and seven baseline models. DETAILED DESCRIPTION

[0035] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0036] It should be noted that when a component is referred to as being "fixed" to another component, it can be directly on the other component or there can be intervening components. When a component is referred to as being "connected" to another component, it can be directly connected to the other component or there can be intervening components. When a component is referred to as being "disposed on" another component, it can be directly on the other component or there can be intervening components. When a component is referred to as being "disposed in the middle", it is not only disposed in the middle position, but also within the range defined by the middle. The terms "vertical", "horizontal", "left", "right", and similar terms used herein are for illustrative purposes only.

[0037] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in the description of the application herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0038] As Figure 1 shown, the application discloses a mixed deep learning water quality prediction method based on time-frequency feature extraction, comprising the following steps:

[0039] Step S1, collecting and preprocessing water pollution, hydrology and meteorological hourly data;

[0040] Step S2, using time-discrete stationary wavelet transform to decompose the data into sub-signal sequences with uniform length and different frequency domains, and at the same time, integrating time features such as season and date into the sequence to form a time series deep learning model input data set;

[0041] Step S3, constructing an Informer Encoder to do sequence feature enhancement coding to discover multivariate global time correlation;

[0042] Step S4, through step-by-step decreasing stack bidirectional LSTM, the local feature correlation of the half cascade sequence is strengthened to construct a mixed deep learning model with strong adaptability for predicting target pollutant changes;

[0043] Step S5, taking RMSE, MAE and MAPE as evaluation parameters, using Adam optimizer to optimize the parameters of the deep learning model, and comparing with other benchmark models to verify the effect;

[0044] Step S6, analyzing the correlation matrix learned by the deep learning model to obtain deeper time-frequency rules of water quality and hydrology and meteorology variables.

[0045] Compared with the prior art, the mixed deep learning water quality prediction method based on time-frequency feature extraction provides an intelligent hybrid model combining deep learning and time-frequency domain feature decomposition method to explore the influence of hydrology, meteorology and other factors on the water quality of the study basin, and further improve the model explainability; meanwhile, the feature extraction module combining Informer Encoder and multi-layer bidirectional LSTM is proposed to obtain the global feature correlation of the sequence, and the sequence local feature dependence enhancement is also considered; and the experimental results show that the method of the application effectively improves the prediction accuracy and reduces the hysteresis due to the application of discrete stationary wavelet transform to decompose the time sequence, so the prediction method of the application has important practical value in water quality prediction real-time monitoring and forecasting, and provides a new method for water environment quality long sequence data prediction and model behavior explainability, which provides a strong foundation for relieving water pollution, helps to reduce cost, protect ecology and improve human health.

[0046] Specifically, in the watershed water quality model, hydrological and meteorological parameters are used as influencing factors, which have different degrees of influence on the target sequence parameters and are autocorrelated. Discrete stationary wavelet transform (ESWT) is used to decompose these influencing factors in the frequency domain, which helps to better express the correlation. At the same time, the global correlation features of the sequences at different frequencies obtained by decomposition can be well captured by the attention mechanism in the Informer Encoder, and then the local correlation features are extracted in the Decoder, so as to enter a time series deep learning model for training. To improve the prediction accuracy and have the interpretability of different frequency influences.

[0047] The watershed water quality time series data mixes factors of different frequencies, making it difficult for deep learning models to effectively capture their correlation features. The original water quality data contains low-frequency components and high-frequency components, which can also be interpreted as seasonal long-term trends and short-term dramatic changes. Therefore, the discrete stationary wavelet transform (ESWT) is introduced to decompose the original sequence. ESWT is an extension of the discrete stationary wavelet transform (SWT), which has stronger time-frequency decomposition capability than general wavelet transform.

[0048] Wavelet transform inherits and develops the characteristics of short-time Fourier transform, effectively overcoming the shortcoming of window size not changing with frequency. It uses an effective long wavelet function that decays over time, which can decompose data into low-frequency signals representing overall trends and high-frequency signals representing short-term changes. Discrete stationary wavelet transform (SWT) has shift invariance, which is beneficial to explore the rules of time series data. In the SWT decomposition process, instead of downsampling, zero values are inserted between the filter coefficients to implement filter expansion, thereby ensuring the consistency of the length of the original sequence and the subsequence. The original data is decomposed into one low-frequency signal and a series of high-frequency signals [cD1, cD2, …, cDn] by the SWT filter. Conventional wavelet transform and SWT only decompose the low-frequency component further, often ignoring the high-frequency component. However, in the process of multi-scale wavelet decomposition, if the decomposition of the high-frequency component is ignored, it will cause the leakage of time series signals, leading to incomplete decomposition of the volatile water quality sequence, which is not conducive to the prediction and training of deep learning models. The formula of general wavelet transform is as follows:

[0049]

[0050] where is the normalization factor, f(t) is the input signal, ψ(t) is the mother wavelet function, a is the scale index parameter, and b is the time shift parameter.

[0051] The data is decomposed into high and low frequency components using the discrete stationary wavelet transform (ESWT). Its greatest feature is that it not only focuses on low-frequency components, but also fully decomposes high-frequency components with large fluctuations. Therefore, through ESWT decomposition, more effective, more accurate, and more complete information can be obtained to express the original water quality sequence.

[0052] After the time series of water quality and hydro-meteorological data are decomposed in the time-frequency domain, the water quality signal with complex and unclear characteristics becomes more concise, stable, and clear, and high and low frequency sub-signal sequences are obtained. The multivariate global time correlation in the time-frequency domain is better discovered by the Encoder, and the dominant correlation characteristics are obtained to better analyze and understand the evolution law of water pollution. The Informer Encoder is mainly composed of a multi-head Probsparse self-attention layer and a distilling layer, which is similar to the encoder of the Transformer as a whole. The multi-head Probsparseself-attention layer captures different information through different spatial dimensions, which enhances the time sequence feature extraction capability of the encoder. In particular, a residual connection and a normalization layer are applied after each attention block module. The distilling layer is at the end of the encoder, which is used to reduce the output length when the input time sequence is too long, reduce the relevant redundant features, and thus improve the calculation efficiency, and the features will be more distinctive.

[0053] Multi-head attention is to use different calculation methods to calculate the weight value of attention to obtain different spatial dimension information. The multi-head Probsparseself-attention layer is the most critical sparse self-attention mechanism (Probsparseself-attention) mechanism, which solves the problem of high computational complexity of traditional self-attention and has the performance of capturing sequence dependencies.

[0054] The Transformer model is fundamentally different from the classic convolutional neural network CNN and recurrent neural network RNN, and the key lies in its attention mechanism applied to the entire network structure. The attention mechanism cleverly solves the problem of RNN over-reliance on the previous time, and can be calculated in parallel, improving the operation efficiency. Vaswan et al. proposed the classic scaled dot-product self-attention mechanism. The input form is (Query, Key), and then the scaled dot-product is performed, that is:

[0055]

[0056] in, d is the input dimension, and the probabilistic form of the attention coefficient for the i-th query is:

[0057]

[0058] in, k(q i k i Choose an asymmetric exponent

[0059] Traditional scaled dot product self-attention mechanisms use quadratic time complexity click operations to calculate probabilities, which is a significant obstacle to improving model prediction accuracy. Zhou et al.'s research on the probability distribution of self-attention revealed its sparsity; that is, not all time points are closely related, meaning a few dot products contribute significantly to the main attention, while others contribute negligibly. Therefore, they proposed the Probsparse self-attention mechanism. A key component is calculating the distance between the probability distribution of query-key dot product pairs and a discrete uniform distribution, which filters out the most important dot product pairs. Based on KL divergence, they proposed an evaluation formula M to measure query sparsity:

[0060]

[0061] First, the input embedding is mapped to different spaces using different linear layers, resulting in Q, K, and V. Second, K is randomly sampled to obtain K_sample, and for each q... i Find the value of M for u ∈Q, and find the q elements with the largest M value. i , forming Q - For this Top-u q i We calculate the score for K to obtain the primary attention score, S1. Then, to ensure that the input and output sequence lengths are consistent, the unselected items are usually used as Q. - For the other q values, the missing score values ​​are replaced by mean(V), resulting in S0. Finally, the enhanced data features S = (S1, S0) are obtained. The mathematical formula for probabilistic sparse attention is:

[0062]

[0063] The encoder applies the Probsparse self-attention to extract the global dependency of the long sequence input, which causes the mean (V) value redundancy combination in the feature mapping. Therefore, the Distilling operation at the end of the encoder is to give higher weights to the dominant features and reduce the redundant features of the sequence. First, after the multi-head Probsparse self-attention and the key operations such as Residual connection and layer normalization in the attention block, one-dimensional convolution operation Conv1d is performed on the time series features, then the ELU activation function is used, and finally the maximum pooling operation is performed. Thus, the feature sequence is down-sampled, the feature dimension is compressed, and more important information is extracted. The formula of the Distilling operation is:

[0064]

[0065] The Informer encoder structure is very important in exploring the evolution process of water quality, but the local correlation of multivariate in time-frequency domain is easily ignored. In order to effectively enhance the local feature correlation of time-frequency data in the water quality prediction process, the Decoder composed of SBiLSTM, GAP and FCN is introduced. Decoder uses a stack of bidirectional LSTM with decreasing step to capture the local time dependency of feature map time information, thereby making up for the Encoder's neglect of capturing local feature information. Then, global average pooling (GAP) is used to reduce the dimension of the fully connected layer to prevent overfitting caused by excessive parameter quantity.

[0066] Based on the long short-term memory network (LSTM) derived from the recurrent neural network (RNN), a stack of bidirectional LSTM modules with decreasing step is designed to enhance the local dependency of the water quality frequency domain time series feature map. Compared with LSTM, bidirectional LSTM can consider the relationship between the previous signal state and the next signal state in the same sequence to predict the current state, and in addition, the stack of bidirectional LSTM can accurately extract more effective features from the water quality sequence.

[0067] The structure of SBiLSTM is composed of multiple layers of bidirectional LSTM, which has forward and backward calculation processes at the same time. With the increase of the number of layers, the number of unit weights of each hidden layer gradually decreases to improve the prediction performance of the model. The structure of LSMT is shown in Figure 1 The LSTM introduces three "gate" structures and cell state C to selectively affect the state of information at each time. The specific propagation process of LSTM is as follows:

[0068] ft =σ(W f x t +U f h t- 1+b f ), #(7)

[0069] i t =σ(W i x t +U i h t-1 +b i ), #(8)

[0070] O t =σ(W o x t +U o h t-1 +b o ), #(9)

[0071]

[0072]

[0073] h t =O t tanh(C t ), #(12)

[0074] Among them, f t i t o t , where W is the current output value of the forget gate, input gate, and output gate; t W i W o W c These are the forget gate, input gate, output gate, and candidate cell state at the current time x. t The weight matrix; b f b i b O It is the bias constant.

[0075] Therefore, this application selects the Li River in Guilin, Guangxi, China as the study area. This basin belongs to the subtropical monsoon hot and humid climate zone, with concentrated and abundant rainfall throughout the year, reaching a maximum annual rainfall of 2000 mm. Water volume increases in summer and decreases in winter, with alternating rainy and dry seasons, and significant differences between the wet and dry periods. Rainfall and flow are closely related. Major hydrological and meteorological indicators such as rainfall and flow exhibit obvious seasonal characteristics and short-term dramatic variations. Water quality and pollutant data also show seasonal variations. Therefore, the Li River basin possesses the necessary conditions for time-frequency filtering decomposition of water quality time-series data, making it suitable for validating the effectiveness of the research model. The distribution of monitoring stations in the study area is as follows: Figure 2 As shown.

[0076] The research data is from the multi-point continuous online monitoring station of Guilin Ecological Environment Center Station. There are 47 water quality monitoring stations in the Lijiang River Basin, including 25 regular monitoring stations and 22 micro monitoring stations. The average hourly data of hydro-meteorological and water quality pollutant concentrations from 2018 to 2021 is used as the basic data set. As shown in Table 1, the hydro-meteorological data and pollutant data are listed in the basic data set. The hydro-meteorological data includes rainfall (PCP), runoff (Q), water temperature (WT), pH, conductivity (EC), and turbidity (TB). The pollutant data includes total phosphorus (TP), total nitrogen (TN), permanganate (CODMn), ammonia nitrogen (NH3-N), and dissolved oxygen (DO). The core indicators of water quality evaluation, DO, CODMn, NH3-N, and TP, are selected as the prediction targets of the model.

[0077] Table 1. Variable data

[0078]

[0079] In response to this, the present application proposes a prediction model framework for the basin. The framework includes data collection and processing, time-frequency filter decomposition, model construction, result analysis, and frequency domain correlation analysis. First, water quality pollutants, hydrological and meteorological hourly data are collected and preprocessed. Second, the original data is decomposed into sub-signal sequences with uniform length and different frequency domains using a time-frequency filter decomposer. At the same time, special consideration is given to integrating time features such as season and date into the sequence to form the input data set of the time series deep learning model. Third, the Informer Encoder is constructed to enhance the coding of sequence features to discover the global time correlation of multiple variables. Fourth, the local feature correlation of the half-cascaded sequence is strengthened by progressively decreasing the stacking of bidirectional LSTM, thereby constructing a hybrid deep learning model with strong adaptability to predict the target pollutant changes. Fifth, RMSE, MAE, and MAPE are used as evaluation parameters, and the Adam optimizer is used to optimize the parameters of the deep learning model. The effectiveness is verified by comparing with other benchmark models. Finally, the correlation matrix learned by the model is analyzed to obtain the deeper time-frequency rules of water quality and hydro-meteorological variables.

[0080] Informer Encoder, as an effective method for extracting high-level global features from multi-channel long-time feature sequences, is first applied to water quality time-frequency feature extraction. The encoder is constructed through multi-head self-attention layers and distillation layers. Sparse self-attention is used to calculate the correlation matrix of water quality and hydro-meteorological variables, and the dominant feature information is adaptively selected and enhanced. Then, the distillation mechanism is used to eliminate redundant features and reduce the dimension of sequence features. Its structure is shown in Figure 3 .

[0081] First, the water quality and hydro-meteorological data are filtered by a time-frequency filter to obtain different frequency sequences, and then local time features (position encoding) and global time features (season, month, date, etc. encoding) are fused. Second, the input tensor X is obtained after embedding, and X is mapped to different dimensions by different linear layers to obtain Q, K, and V. Third, the main Q ～ and K are multiplied by the transpose matrix to calculate the weight coefficient between variables, and then normalized to obtain the correlation matrix using the softmax activation function. Fourth, the correlation matrix is multiplied by the V matrix, and the index position not calculated Score is filled with Mean(V) to obtain enhanced data feature information. Note that if a multi-head attention mechanism is used, x needs to be divided into nhead groups of different Q, K, and V, and each group needs to be calculated by the attention mechanism. Fifth, the attention score of each group is spliced to obtain the final enhanced feature encoding information. Sixth, to further reduce redundant features, the obtained feature encoding is processed by Dropout, residual connection and normalization, and then Conv1d, ELU, MaxPool are performed in sequence to obtain more obvious dominant feature information and reduce the sequence length. Seventh, after alternating processing by multiple layers of multi-head self-attention mechanism and distillation operation, more accurate global features are extracted. Finally, the stronger the correlation with the target water quality index, the more obvious the feature enhancement, thereby enhancing the feature information of the dominant factors and reducing the feature information of other factors.

[0082] The Decoder is mainly composed of stacked bidirectional LSTM. It is used to extract the local feature dependence of water quality in the time-frequency domain, better discover the local correlation of adjacent data, and thus extract more effective water quality related feature information. The overall Decoder structure is shown in Fig4:

[0083] First, the feature map of the water quality sequence is input into the first layer BiLSTM for forward and backward local feature information extraction, which is more conducive to discovering deep local feature information of water quality time series. Second, continue to input into the next layer BiLSTM for the same operation in sequence until the last layer BiLSTM is processed to obtain the frequency domain correlation matrix. Third, the frequency domain correlation matrix after local feature extraction is reduced in dimension by the global average pooling layer (GAP). Fourth, the feature information is combined together by the fully connected layer to predict the future target water quality sequence.

[0084] The pollutant and hydro-meteorological data are decomposed into sub-sequences of different frequency bands based on the discrete stationary wavelet transform (ESWT) and the time characteristics of the data are fused to construct a multi-dimensional time series input dataset. According to the indicators of these characteristic parameters in the past few hours, the InformerEncoder-SBiLSTM model is input, and the target water quality sequence at the next moment is predicted. This scheme adopts rolling prediction. Repeat this process until the prediction of the target pollutant is completed.

[0085] Let the time characteristic indicators be T{T 1 , T 2 , …, T n}, the hydro-meteorological parameters be H{H 1 , H 2 , …, H n}, and the pollutant indicators be P{P 1 , P 2 , …, P n}. In the model, the input matrix X{x1,x2,…,x n} is as shown in Eq(14). The target sequence E′{e′1,e′2,…,e′ n} is as shown in Eq(15), where f(X) is the training function of the Informer Encoder-SBiLSTM neural network. The entire model training process is explained by Eq(14) and Eq(15). Let the rolling window size be l, as shown in Eq(16) and Eq(17).

[0086]

[0087]

[0088] X t = [x t , x t+1 , …, x t+l-1 ] T , t∈1,2,…,n-l+1 (16)

[0089] e′ t = f([x t , x t+1 , …, x t+l-1 ]), t∈1,2,…,n-l+1 (17)

[0090] The prediction performance of the proposed ESWT-IE-SBiLSTM model was evaluated. First, the effectiveness of the wavelet decomposition module in the ESWT-IE-SBiLSTM model was explored, overcoming the problem of data deviation at the peak and trough of the wave. And respectively embedded empirical mode decomposition (EMD), discrete stationary wavelet (SWT) to construct EMD-IE-SBiLSTM and SWT-IE-SBiLSTM models, compared with ESWT-IE-SBiLSTM model, highlights the ESWT is more suitable for the decomposition of water quality time series data than EMD and SWT. Finally, compared with other models, autoregressive moving average model (ARIMA) based on statistical theory, random deep forest based on machine learning, RNN, LSTM, and Transformer based on deep learning were established, and ESWT-IE-SBiLSTM model showed better results.

[0091] In order to better quantify and evaluate the accuracy of the experimental prediction results, the evaluation index uses the root mean square error (RMSE), the mean absolute error (MAE) and the mean absolute percentage error (MAPE). The three represent the deviation of the model prediction from the true value fitting, the smaller the better the result. As Eq(12) to Eq(14).

[0092]

[0093]

[0094]

[0095] Where n is the total number of samples, y i is the monitored time series, y′ i is the predicted time series.

[0096] The number of encoder layers in the deep learning model is set to 2, the number of heads in the multi-head sparse self-attention mechanism is 8, the sampling factor is (96, 25), the Conv1ds convolution operation in the distillation layer has a kernel size of 3, a stride of 1, a padding of 1, and a channel number of 18, the MaxPool pooling operation has a kernel size of 3, a stride of 2, and a padding of 1, and the number of stacked bidirectional LSTM layers is 5. During training, the batch size is 32. The training process is repeated 300 times on the training data set. To avoid local overfitting, the dropout probability between layers is set to 0.2. The network weights are initialized using the kaiming normal initializer. During the model training process, the optimizer uses Adam. In addition, the L2 regularization weight decay method, the packet loss method, and the early stopping method are combined to suppress overfitting. In water quality prediction, the value of the hyperparameters directly affects the performance and accuracy of the model. Therefore, selecting appropriate hyperparameter values is particularly important. The detailed hyperparameters are shown in Table 2.

[0097] The time-frequency feature decomposition stage uses the discrete stationary wavelet transform (ESWT) to obtain sub-sequences in different frequency bands. The selection of the wavelet decomposition scale is particularly important for the experimental results. As shown in Table 3, the wavelet decomposition scale is selected from the set (2, 3, 4, 5). The experimental results show that as the decomposition scale increases, the performance of the model's prediction first increases and then decreases. From the experimental results, it can be seen that when the number of decomposition scales is 4, the RMSE and MAE are the smallest, and the model's prediction accuracy is higher. Therefore, the decomposition scale of 4 is selected as the experimental parameter.

[0098] Table 2 Model Hyperparameter Settings

[0099]

[0100] Table 3 EMD-IE-SBiLSTM Model Decomposition Scale Selection Results

[0101]

[0102] Traditional deep learning models cannot effectively predict extreme values due to the entanglement of different frequency bands in historical water quality data, while the discrete stationary wavelet transform (ESWT) can free the original sequence from multi-frequency entanglement. To evaluate the prediction performance of the proposed model on extreme values and the superiority of the ESWT module, the following experiments are conducted: Figure 4As shown, the concentration changes of DO and CODMn within 48 hours of extreme values ​​were selected on the test set. EMD-IE-SBiLSTM, SWT-IE-SBiLSTM, and ESWT-IE-SBiLSTM models were established, and the prediction results were compared with those of the three models. The prediction results show that the maximum relative errors between the predicted and observed values ​​are 7.35% and 8.96%, respectively. Figure 4 The comparison results demonstrate that the combination of ESWT and IE-SBiLSTM is the most suitable for the target water quality prediction problem. Because Empirical Mode Decomposition (EMD) uses global wavelet decomposition of the original water quality data, rather than local wavelet decomposition, it struggles to effectively separate seasonal trends and localized sharp fluctuations, resulting in poor model prediction performance. Compared to Discrete Wavelet Transform (SWT), Discrete Stationary Extended Wavelet Transform (ESWT), by considering the influence of high-frequency components in the water quality time series data, is sensitive to high-frequency fluctuations. This allows the ESWT embedded in the IE-SBiLSTM model to more accurately predict peaks and troughs in the data, resulting in more timely and meaningful predictions.

[0103] Frequency domain correlation analysis aims to determine the deeper-level impact of hydrological and meteorological parameters on water quality target sequences and to analyze the interpretability of the model. For example, to analyze the factors influencing the variation of the water quality index DO in different frequency bands, Figure 5 The data presents the attention correlation matrix discovered by the self-attention mechanism in the encoder. Among water pollution factors, the correlation between DO and TP and TN is mainly observed in the low-frequency range and the slower high-frequency range, with weaker correlation in the high-frequency range. The correlation between DO and CODMn is strong in the low-frequency range and weak in the high-frequency range. The correlation between DO and NH3-N is mainly observed in the slower high-frequency range, with lower correlation in the low-frequency range and the faster fluctuating high-frequency range. Among hydrometeorological factors, DO is strongly correlated with pH, ​​TB, and water temperature in the low-frequency range and the slower high-frequency range, while the correlation with rainfall is mainly observed in the low-frequency range. The correlation between DO and EC is mainly observed in the high-frequency range and weak in the low-frequency range.

[0104] The effects of TN, TP, CODMn, pH, TB, water temperature, and rainfall on water quality DO are mainly reflected on a wide time scale, exhibiting seasonal trends. In contrast, the effects of EC and NH3-H on DO are mainly reflected in higher frequency bands, showing short-term, drastic abrupt changes. This phenomenon indicates the discovery of time-frequency patterns among the variables, and the predictive behavior of the model can be explained by analyzing the attention correlation matrix.

[0105] The superiority and advancement of the ESWT-IE-SBiLSTM model established by using the benchmark models of MLP, SVR, RNN, GRU, LSTM, Transformer, and Informer are verified. The quantitative results of RMSE, MAE, and SMAPE of the eight models for the DO and CODMn indexes at the same time step (48 h) are listed in Table 4. By comparing the benchmark models with the ESWT-IE-SBiLSTM model, it is found that the ESWT-IE-SBiLSTM model has the best prediction effect. Since the machine learning model cannot effectively capture the complex relationship of water quality time series data, the prediction accuracy of the MLP and SVR models is low, and the error is large. The RNN, LSTM, and GRU models are obviously superior to the non-time series models MLP and SVR in terms of performance. Among them, the LSTM and GRU models effectively solve the "gradient vanishing and explosion" compared with the RNN model, thus having the ability to capture the long sequence data dependence of water quality, so the prediction performance of LSTM and GRU is better than that of RNN. The prediction performance of the Transformer model is better than that of LSTM and GRU, because the Transformer model structure contains a self-attention mechanism, and the self-attention mechanism has global modeling capability. The prediction performance of the Informer model is better than that of the Transformer model, because the Informer model uses a sparse self-attention mechanism and a distillation mechanism to achieve feature enhancement of long sequence data of water quality. The ESWT-IE-SBiLSTM model coupled with wavelet transform and time series model has higher water quality time series prediction performance than the Informer model. This is because the ESWT-IE-SBiLSTM model can separate hidden time-frequency signals to mine more reliable water quality time series data rules, reduce the interference of noise, improve the prediction accuracy and stability, and has good adaptability.

[0106] Table 4 prediction results of other models

[0107]

[0108]

[0109] In order to more intuitively show the prediction effect of DO and CODMn, Figure 6The predicted values and observed values of water quality index concentration predicted by ESWT-IE-SBiLSTM and seven baseline models were compared. The prediction curves of each model were basically consistent with the observed curve trend. Compared with deep learning models LSTM, GRU and RNN, the predicted values of MLP and SVR machine learning models had large differences with the observed values, indicating that capturing time correlation had an important influence on the prediction results in water quality time series prediction. Both Transformer and Informer models captured time correlation based on self-attention mechanism and position encoding vector, but Informer solved the high computational cost and memory consumption of self-attention calculation in long sequence input problems, and also enhanced the ability of feature extraction. Therefore, the Informer model was often better than the traditional Transformer in water quality sequence prediction. Compared with the above models, the prediction effect of ESWT-IE-SBiLSTM model was the best, which was closer to the real water quality trend and more sensitive to the local sharp mutation and peak value of water pollution. The main reason was that ESWT-IE-SBiLSTM model could perform time-frequency analysis on water quality time series signal and extract more rich time-frequency domain information, which could better handle non-stationary signals and noise interference, and also better capture the local changes of the signal. The established ESWT-IE-SBiLSTM model provided a new method for water quality prediction.

[0110] In summary, an accurate and reliable water environment quality prediction system can effectively monitor and predict water quality evolution, which helps decision-makers to discover and diagnose water environmental pollution problems in time and take necessary measures. It has important significance to alleviate water pollution and solve the deterioration of water environment in the basin. A hybrid deep learning model combining ESWT, Informer Encoder and SBiLSTM modules is proposed in the present application. The training data set is composed of time series data of hydro-meteorological parameters, pollutant parameters and other parameters. The ESWT module is used for time-frequency feature decomposition. In the time-frequency filtering module, the extended wavelet transform is used to effectively separate the high and low frequency signals. The Informer Encoder is used to capture the long-term dependence relationship of the time-frequency feature sequence. The SBiLSTM module is used to enhance the mapped local short-term dependence relationship. The established model not only can overcome the problem of local mutation and extreme value prediction, but also can reflect the influence of different hydro-meteorological conditions on the trend of water quality evolution. The performance of the established model is evaluated, and compared with seven baseline models such as MLP, SVR, RNN, etc. The water quality indexes DO and CODMn are selected as performance evaluation indexes. The mean absolute error (MAE) is 0.329 and 0.121 respectively; the root mean square error (RMSE) is 0.217 and 0.057 respectively; the symmetric mean absolute percentage error (SMAPE) is 0.022 and 0.063 respectively. The results show that the ESWT-IE-SBiLSTM model established in the present application can better predict the trend of water quality change. The prediction accuracy of the ESWT-IE-SBiLSTM model for water quality is better than that of the machine learning models such as MLP, SVR, etc. and the traditional deep learning models such as RNN, LSTM, GRU, Transformer, Informer, etc. The evaluation results also show that by coupling the extended wavelet transform and deep learning in water quality time prediction, the model established in the present application can improve the prediction accuracy of local mutation and extreme value of water quality.

[0111] The traditional deep learning method is limited to the original water quality time series data set mixed with frequencies, and the prediction behavior of the model is not interpretable, which is very unfavorable for water environment managers to take necessary measures to alleviate water pollution at the right time. The ESWT-IE-SBiLSTM model proposed in the application can separate different frequency signals hidden in the original data, learn the correlation between signal sequences through a sparse self-attention network, realize effective dominant feature adaptive filtering with a time complexity of O(LlogL), and extract the dominant feature map through a semi-serial sequence. The prediction behavior of the model can be explained by analyzing the adaptive attention correlation matrix, and the correlation between different frequency signals and the water quality prediction target sequence can be found. For example, the hydrological, meteorological and water quality parameter variables from April 1 to 30, 2021 are selected as the model input, and DO is selected as the prediction target sequence. Through attention matrix feature visualization analysis, it is found that the correlation between DO and TN, TP, CODMn mainly reflects in the low frequency band, and the correlation between DO and EC, NH3-H mainly reflects in the high frequency band. The correlation between DO and PH, TB, water temperature, rainfall mainly reflects in the low frequency interval. This shows that the model has strong interpretability and can provide support for watershed water environment pollution prevention and control.

[0112] On the basis of the results, the application provides a new method for water environment quality long sequence data prediction and model behavior interpretation, which provides a strong basis for alleviating water pollution, helps to reduce costs, protect ecology and improve human health. At the same time, the method of the application is suitable for single site or single city watershed water environment quality prediction in small and medium-sized cities.

[0113] The above examples are only used to illustrate the technical solutions of the present application and not to limit it, any modification or equivalent replacement within the spirit and scope of the present application should be covered in the scope of the technical solutions of the present application.

Claims

1. A hybrid deep learning method for water quality prediction based on time-frequency feature extraction, characterized in that, Includes the following steps: Step S1: Collect hourly data on water pollutants, hydrology, and meteorology; Step S2: Use the discrete stationary wavelet transform to decompose the data into sub-signal sequences with uniform lengths but different frequency domains, and incorporate time features such as season and date into the sequences to form the input dataset for the time series deep learning model. Step S3: Construct an Informer Encoder to perform sequence feature enhancement encoding in order to discover multivariate global temporal correlations; Step S4: Enhance the local feature correlation of the halved cascade sequence by progressively decreasing stacked bidirectional LSTMs to construct a highly adaptive hybrid deep learning model for predicting changes in target pollutants. Step S5: Using RMSE, MAE, and MAPE as evaluation parameters, optimize the parameters of the deep learning model using the Adam optimizer, and verify the effect by comparing with other benchmark models. Step S6: Analyze the correlation matrix learned by the deep learning model to obtain deeper time-frequency patterns of water quality and hydrological meteorological variables.

2. The hybrid deep learning water quality prediction method based on time-frequency feature extraction according to claim 1, characterized in that, In step S2, a time-frequency filter decomposer is constructed using discrete stationary wavelet transform to separate the time-series data into low-frequency components with seasonal trends and high-frequency components with short-term drastic changes.

3. The hybrid deep learning water quality prediction method based on time-frequency feature extraction according to claim 1, characterized in that, The Informer Encoder includes a multi-head Probsparse self-attention layer and a distilling layer. The multi-head Probsparse self-attention layer captures different information through different spatial dimensions, and a residual connection and a normalization layer are applied after each attention block. The distilling layer is at the end of the encoder and is used to reduce the output length and reduce relevant redundant features when the input time sequence is too long and causes information redundancy.

4. The hybrid deep learning water quality prediction method based on time-frequency feature extraction according to claim 1, characterized in that, During the training process of the deep learning model, a combination of L2 regularization weight decay, packet loss, and early stopping methods is used to suppress overfitting.

5. The hybrid deep learning water quality prediction method based on time-frequency feature extraction according to claim 1, characterized in that, Step S3 includes the following steps: (1) Water quality and hydro-meteorological data are filtered by time and frequency to obtain different frequency band sequences, and then local time features and global time features are fused together; (2) After embedding, the input tensor X is obtained. X passes through different linear layers to map the time-frequency data of water quality and hydrological meteorology to different dimensions to obtain Q, K, and V. (3) The main Q values ​​that have been screened and measured ～ Multiply by the transpose of K to calculate the weight coefficients between variables, then normalize and use softma. x The activation function yields the correlation matrix; (4) Multiply the correlation matrix with the V matrix and fill the index positions of the uncalculated Score with Mean(V) to obtain the enhanced data feature information. If a multi-head attention mechanism is used, x needs to be divided into different Q, K, and V groups of nhead. Each group needs to be calculated by the attention mechanism. (5) Concatenate the attention scores of each group to obtain the final enhanced feature encoding information; (6) After the obtained feature encoding is processed by Dropout, residual connection and normalization, Conv1d, ELU and MaxPool are performed in sequence to obtain more obvious dominant feature information and reduce the sequence length. (7) A multi-layer, multi-head self-attention mechanism and distillation operation are used for alternating treatment.

6. The hybrid deep learning water quality prediction method based on time-frequency feature extraction according to claim 1, characterized in that, Step S4 includes the following steps: (1) Input the feature map of the water quality sequence into the first layer of BiLSTM to extract local feature information in the forward and backward directions; (2) Continue to input the same operation into the next layer of BiLSTM in sequence until the last layer of BiLSTM is processed, and obtain the time correlation matrices in the frequency domain; (3) The frequency domain correlation matrix after local feature extraction is reduced in dimensionality by a global average pooling layer (GAP). (4) The feature information is combined together through a fully connected layer to predict the future target water quality sequence.

Images