A transformer-bilstm water quality prediction method based on shap explainability analysis

Abstract

The application discloses a kind of based on shap explainability analysis's Transform-BiLSTM water quality prediction method, belongs to the field of water quality prediction and explainability artificial intelligence under deep learning.First, the multi-parameter water quality monitoring data of target water area is obtained, and the data is cleaned and normalized.The data is divided into training set, validation set and test set, a hybrid neural network model composed of Transform encoder and bidirectional long short-term memory network is constructed, Transform-BiLSTM training data, evaluation index is calculated, finally SHAP is used to explain and analyze the fusion model, accurately identify the driving factor and the direction and degree of action affecting water quality change.The application significantly improves the accuracy of water quality prediction, enhances the credibility and explainability of the model, and provides intuitive and reliable decision-making basis for water environment management.

Description

Technical Field

[0001] This invention belongs to the field of water quality prediction and interpretable artificial intelligence under deep learning, specifically involving a Transformer-BiLSTM water quality prediction method based on SHAP interpretability analysis. Background Technology

[0002] Water quality prediction is crucial in environmental protection and water resource planning. Traditional techniques mainly fall into three categories: traditional statistical models, process-driven models, and neural network prediction models. Traditional statistical models include autoregressive (AR) and support vector regression (SVR), which rely on historical observation data, making it difficult to capture nonlinear relationships and complex coupling characteristics, resulting in limited prediction accuracy. Process-driven models can reflect the physicochemical processes of water bodies, but they are extremely dependent on parameters and computationally complex. Conventional neural networks, such as convolutional neural networks (CNN) or long short-term memory networks (LSTM), have a certain prediction accuracy, but they always suffer from the gradient vanishing problem. Summary of the Invention

[0003] This invention addresses the low prediction accuracy, insufficient long-term time-series modeling, and lack of interpretability in existing water quality prediction technologies by proposing a Transformer-BiLSTM water quality prediction method based on SHAP interpretability analysis. Bidirectional Long Short-Term Memory (BiLSTM) networks possess strong time-series capture capabilities, while the Transformer considers various time-series locations to better capture long-distance dependencies. This invention combines the advantages of Transformer and BiLSTM, performs SHAP interpretability analysis, and forms a novel STBL water quality prediction model. This model achieves high-precision, interpretable, multi-parameter water quality prediction, providing strong technical support for scientific decision-making and water environment management.

[0004] The technical solution adopted in this invention is as follows: Historical monitoring data of multiple parameters in the target water area is acquired and cleaned and normalized; a hybrid neural network composed of Transformer and BiLSTM is constructed. The Transformer utilizes a multi-head self-attention mechanism to extract temporal features, while the BiLSTM captures bidirectional changes, achieving high-precision prediction of multiple water quality parameters; SHAP is introduced to calculate the shap value, revealing the contribution of each feature and its positive and negative impact. This invention integrates global feature extraction and bidirectional temporal modeling, effectively improving the accuracy and stability of water quality prediction, filling gaps in the fields of water environment governance and water resource management, and demonstrating significant practical value and promising prospects for application.

[0005] A Transformer-BiLSTM water quality prediction method based on SHAP interpretability analysis includes the following steps:

[0006] (1) Obtain multi-parameter water quality monitoring data of the target water area.

[0007] (2) Clean and normalize the data.

[0008] (3) Divide the data into training set, validation set and test set.

[0009] (4) The Transformer encoder performs embedding encoding and position encoding, and uses a self-attention mechanism to calculate the time step.

[0010] (5) The BiLSTM neural network receives the feature sequence output by the Transformer encoder, learns the dependencies between time series through forward and backward LSTM, and concatenates the forward and backward hidden states of the time step to obtain bidirectional encoded features.

[0011] (6) Calculate the mean squared error (RMSE), mean absolute error (MAE), and coefficient of determination (R²) to evaluate the prediction results.

[0012] (7) Select a deep neural network interpreter, calculate the SHAP value, and interpretively analyze the prediction results of the hybrid neural network model.

[0013] Step (1) is as follows: Obtain the time series of water quality data from a water quality monitoring station in a certain water area, including pH, COD, and DMn. Water quality factors include TN, TP, DO, water temperature, conductivity, turbidity, chlorophyll, and algae density.

[0014] Step (2) is as follows: Use a time-series-based cubic spline function to smooth the missing data. The cubic spline function is as follows:

[0015]

[0016] in, v It is a missing data point. These are known data points. , , , These are the interpolation coefficients.

[0017] Outliers are handled using the Z-score method. If the Z-score is greater than a set threshold, the point is considered an outlier. The formula is as follows:

[0018]

[0019] in, The original data values, n The mean, It is the standard deviation. For standardized data

[0020] Remove duplicate values ​​to ensure sample uniqueness.

[0021] Scale the data to [0,1] and perform min-max normalization, as shown in the following formula:

[0022]

[0023] in, To be the minimum value, It is the maximum value. , Sample feature values.

[0024] Step (3) is as follows: ensure that the model fully learns the patterns of historical data (80% of the training set), use the validation set (10%) for hyperparameter tuning, and evaluate the model’s generalization ability through the test set (10%).

[0025] Step (4) is as follows: The Transformer encoder obtains the water quality sequence and learns the time step dependencies in the sequence.

[0026] Input shape is .

[0027] in, T For time steps, F It is the characteristic number.

[0028] Each time step incorporates sequence information through position encoding, which is then used to locate the position and create a high-dimensional vector. The formula is as follows:

[0029]

[0030]

[0031] in, PE For position encoding, pos For location index, i For dimensional indexing, This represents the feature dimension of the model.

[0032] Positional encoding is added to the feature vector at each time step, and a multi-head self-attention mechanism captures the dependencies between different time steps. The self-attention mechanism calculates the correlation between time steps and generates a weighted output. The input feature vector undergoes three matrix transformations to obtain the Query, Key, and Value, respectively. The formulas are as follows:

[0033] .

[0034] Where Q(Query) is the query vector, representing the information processed at the current time step, with the shape (T, K (Key) is the key vector, representing the content features of the time step, and has the same shape as Q. V (Value) is the value vector, representing the actual information of the time step, and has the shape (T, ...). T is the time step. It is the dimension of the key vector. It is the dimension of the value vector.

[0035] After attention calculation, the output passes through residual connections, layer normalization, and a feedforward network. After multiple layers are stacked, the Transformer encoder outputs a high-dimensional representation sequence. The residual connection formula is as follows:

[0036] Output=LayerNorm(X+SubLayer(X))

[0037] L( )=

[0038] in, μ The vector mean The vector standard deviation, γ It is a learnable scaling parameter. β For learnable bias parameters, Let L( be the input vector of the current sublayer). ) represents layer normalization, and X is the input tensor.

[0039] Step (5) is as follows: The forward Lstm processes the sequence starting from the first item to obtain the forward hidden state, and the backward Lstm processes the sequence starting from the last item to obtain the backward hidden state. In the forget gate, invalid water quality data is filtered out by the Sigmoid function. The input gate filters out valid water quality data through independent neural network layers and the Sigmoid function. The formula is as follows:

[0040]

[0041] Where p is the input variable and e is the natural constant.

[0042] The forget gate and input gate jointly update the state of the data unit, while the output gate determines whether the memory information of the unit is displayed or hidden at the current time. The formula is as follows:

[0043]

[0044]

[0045]

[0046]

[0047]

[0048]

[0049] in, For the input gate vector, Forget gate vector, For the output gate vector, This represents the current cell state. For memory state, Candidate memory vectors, It is currently in a hidden state. It is the hidden state from the previous moment. For Hadama accumulation, Here, is the sigmoid function, tanh is the hyperbolic tangent function, and W is the weight matrix. , , , These are the forget gate bias vector, input gate bias vector, candidate cell state bias vector, and output gate bias vector, respectively. , These are the forget gate weight matrix and the output gate weight matrix, respectively.

[0050] After the data unit state is updated, the forward and backward hidden states are concatenated to obtain bidirectional encoded features, and the prediction result is output.

[0051] Step (6) is as follows: Calculate RMSE, MAE, and RMSE is the global standard deviation of the prediction error, reflecting the degree of deviation between the predicted and actual values. MAE is the mean absolute difference between the predicted and actual values, representing the average prediction deviation. R² is the coefficient of determination, reflecting the goodness of fit of the model to the fluctuations in the real data; the closer its value is to 1, the better the fit. The formula is as follows:

[0052]

[0053]

[0054]

[0055] Where n is the number of samples, For the observed values, These are predicted values.

[0056] Step (7) is as follows: the interpreter uses DeepExplainer to determine the baseline prediction value, the feature subset evaluates the model output, the Shapley formula is used to calculate the marginal contribution of each feature, the sorted samples are decomposed, and the Shaple value is calculated.

[0057] Using DeepExplainer as the SHAPE interpreter, the baseline prediction value is calculated by taking the mean of all samples in the training set. The formula is as follows:

[0058]

[0059] in, The training set consists of samples, and M is the number of background samples. This is the model's predicted output.

[0060] calculate and ,in This represents the predicted value after using only the features of a subset S as input to the model. This means adding features to S. The predicted values ​​are then used. After obtaining the subset of predicted values, water quality characteristics are calculated. The SHAP value is calculated using the following formula:

[0061]

[0062] in, For the size of the subset, The total number of features.

[0063] After obtaining the contribution of sample features, the predicted value is decomposed as follows:

[0064] +

[0065] Take the absolute value of the SHAP average: .

[0066] The formula for calculating the contribution of water quality characteristics is as follows:

[0067]

[0068] The SHAP matrix is ​​obtained after the calculation:

[0069] Where q is the sample size and p is the feature quantity.

[0070] The interpretation is visualized by integrating the SHAP values ​​of all samples and presenting the global feature importance, values, and influence direction using scatter plots or bar charts. The overall contribution ranking is then displayed by sorting the samples by average absolute SHAP values.

[0071] A positive SHAP value indicates that the indicator pushes the predicted value up at the corresponding time step, while a negative SHAP value indicates a suppressive effect.

[0072] The beneficial effects of this invention are:

[0073] (1) The Transformer encoder captures the interaction relationship in water quality time series through self-attention mechanism, extracts deep features from multi-parameter and multi-scale monitoring data, and overcomes the shortcomings of traditional models in long sequence modeling.

[0074] (2) BiLSTM uses historical and future sequence features in bidirectional information transmission, which improves the sensitivity and prediction stability of water quality change trends, and is especially suitable for sudden fluctuations and nonlinear water quality data.

[0075] (3) The SHAP quantitative decomposition prediction results reveal the impact of key water quality indicators, help identify major pollutants and potential risks, and provide a scientific basis for water environment governance. Attached Figure Description

[0076] Figure 1 This is a schematic diagram of the process of the present invention;

[0077] Figure 2 This is a schematic diagram of the Transformer model structure;

[0078] Figure 3 This is a schematic diagram of the BILSTM model structure;

[0079] Figure 4 This is a schematic diagram of the Transformer-BiLSTM model structure;

[0080] Figure 5 The diagrams above and below show the shap values ​​calculated for two monitoring stations. The top diagram shows the shap value calculated for the Fumin Bridge monitoring station, and the bottom diagram shows the shap value calculated for the Cuijiazhuang monitoring station.

[0081] Figure 6 The diagrams above and below illustrate the order of importance of water quality characteristics at the Fumin Bridge monitoring point and Cuijiazhuang monitoring point, respectively. Detailed Implementation

[0082] To make the objectives, technical solutions, and advantages of this invention clearer, the embodiments of this invention are further described below.

[0083] Example 1: To construct a neural network model for accurately predicting water quality, see... Figure 1 This invention proposes a water quality prediction method based on the Transformer-BiLSTM fusion model using SHAP interpretability analysis, comprising the following steps:

[0084] (1) Obtain water quality monitoring data for a specific body of water. Specifically: The water quality data in this example was obtained from the China National Environmental Monitoring Centre, using data from the Cuijiazhuang and Fumin Bridge monitoring stations, including pH, COD, and DMn. TN, TP, DO, water temperature, conductivity, and turbidity. Data set time range: 2020–2024, data time step: 4 hours.

[0085] (2) Data processing: The missing data is smoothed using a cubic spline function, as shown in the following formula:

[0086]

[0087] in, v It is a missing data point. These are known data points. , , , These are the interpolation coefficients.

[0088] The Z-score method is used to handle outliers; if a value exceeds a set threshold, that value is considered an anomaly. The formula is as follows:

[0089]

[0090] in, The original data values, n The mean, It is the standard deviation. This is the data after standardization.

[0091] Removing duplicate values ​​ensures that the sample is unique.

[0092] Scale the data to [0,1] and then normalize it using the following formula:

[0093]

[0094] in, To be the minimum value, It is the maximum value. , Sample feature values.

[0095] (3) Divide the dataset: Divide the water quality data into 80% training set, 10% validation set, and 10% test set.

[0096] (4) Input the water quality sequence into the Transformer encoder to learn the relationship between each time step, such as... Figure 2 As shown.

[0097] Input shape is .

[0098] in, T For time steps, F It is the characteristic number.

[0099] Location encoding incorporates sequence information, and a high-dimensional vector is created after localization. The formula is as follows:

[0100]

[0101]

[0102] in, PE For position encoding, pos For location index, i For dimensional indexing, This represents the feature dimension of the model.

[0103] Positional encoding is added to the feature vector at each time step, and a multi-head self-attention mechanism captures the dependencies between different time steps. The self-attention mechanism calculates the correlation between time steps and generates a weighted output. Each input feature is transformed into Query, Key, and Value through matrix transformation. The formula is as follows:

[0104] ,

[0105] Where Q(Query) is the query vector, representing the information processed at the current time step, with the shape (T, K (Key) is the key vector, representing the content features of the time step, and has the same shape as Q. V (Value) is the value vector, representing the actual information of the time step, and has the shape (T, ...). T is the time step. It is the dimension of the key vector. It is the dimension of the value vector.

[0106] The output passes through residual connections, layer normalization, and a feedforward network. After multiple layers are stacked, the Transformer encoder outputs a high-dimensional representation sequence. The residual connection formula is as follows:

[0107] Output=LayerNorm(X+SubLayer(X))

[0108] L( )=

[0109] Where μ is the vector mean. Let be the vector standard deviation, γ be a learnable scaling parameter, and β be a learnable bias parameter. Let L( be the input vector of the current sublayer). ) represents layer normalization, and X is the input tensor.

[0110] (5) Set a time window to predict future total phosphorus concentration. The input shape is "number of samples, 12, number of features", and the output shape is "number of samples, 1". The forward Lstm processes from the first item in the sequence to obtain the forward hidden state, and the backward Lstm processes from the last item in the sequence to obtain the backward hidden state. In the forget gate, invalid water quality data is filtered out by the Sigmoid function, and the input gate filters out valid water quality data through independent neural network layers, such as... Figure 3 As shown. The formula is as follows:

[0111]

[0112] Where p is the input variable and e is the natural constant.

[0113] The forget gate and input gate update the state of the data unit, while the output gate determines whether the information in the unit's memory is visible or hidden at the current moment. The formula is as follows:

[0114]

[0115]

[0116]

[0117]

[0118]

[0119]

[0120] in, For the input gate vector, Forget gate vector, For the output gate vector, This represents the current cell state. For memory state, Candidate memory vectors, It is currently in a hidden state. It is the hidden state from the previous moment. For Hadama accumulation, Here, is the sigmoid function, tanh is the hyperbolic tangent function, and W is the weight matrix. , , , These are the forget gate bias vector, input gate bias vector, candidate cell state bias vector, and output gate bias vector, respectively. , These are the forget gate weight matrix and the output gate weight matrix, respectively.

[0121] (6) Evaluate the deviation between predicted and actual values ​​using statistical indicators, assess the model's generalization ability and prediction accuracy, and calculate RMSE, MAE, and The formula is as follows:

[0122]

[0123]

[0124]

[0125] Where n is the number of samples, For the observed values, For predicted values

[0126] (7) Using DeepExplainer as the SHAPE interpreter, the mean of the predicted values ​​from the training set samples is used to calculate the baseline predicted value. The formula is as follows:

[0127]

[0128] in, The training set consists of samples, and M is the number of background samples. This is the model's predicted output.

[0129] calculate and , This represents the predicted value after using only the features of a subset S as input to the model. This means adding features to S. The predicted values ​​are then used. After obtaining the subset of predicted values, water quality characteristics are calculated. The SHAP value is calculated as follows: Figure 4 , 5 The formula is as follows:

[0130]

[0131] in, For the size of the subset, Let be the number of features. After obtaining the feature contribution of each sample, the predicted value is decomposed using the following formula:

[0132] +

[0133] The average absolute value of the SHAP values ​​across all samples is used for evaluation.

[0134] .

[0135] The formula for calculating the additional contribution of pairwise interactions of water quality characteristics is as follows:

[0136]

[0137]

[0138] Where q is the number of samples and p is the number of features.

[0139] Finally, the explanation will be visualized to achieve interpretability.

[0140] Compare the results of the Transformer-LSTM fusion model and the Transformer-BiLSTM fusion model in total phosphorus prediction.

[0141] Table 1: Predictions of total phosphorus by different models at two water quality monitoring stations.

[0142] ;

[0143] The table shows that the MAE, RMSE, and RMSE of total phosphorus at the Cuijiazhuang and Fumin Bridge water quality monitoring points were obtained using the Transformer-LSTM fusion model and the Transformer-BiLSTM fusion model. Performance of the three evaluation indicators. Lower RMSE and MAE values ​​indicate better performance. A R² value closer to 1 indicates a better fit. The MAE and RMSE values ​​of the Transformer-BILSTM fusion model are lower than those of the Transformer-LSTM fusion model; however, the R² value is higher, fully demonstrating the good predictive performance of the Transformer-BILSTM fusion model and providing a reliable basis for water quality prediction.

[0144] exist Figure 5 In the diagram, blue to red indicates that the sample values ​​are from smallest to largest, and the color of the scatter points represents the size of the sample values. The scatter points are located at... x Points to the right of axis 0 indicate that the sample plays a positive role in the prediction. Figure 5 (Above) it can be seen that at the Fumin Bridge measuring point, turbidity has a positive effect on the prediction results. Conductivity has a multi-stage relationship with the prediction results: when conductivity is in the low and part of the middle range, the sample contributes positively to the predicted value. The SHAP value distribution of pH is narrow, and the high and low value samples are monotonically indistinct. From... Figure 5 (Below) it can be seen that at the Cuijiazhuang measuring point, total nitrogen, ammonia nitrogen, and turbidity have a positive effect on the prediction results. As the characteristic value increases, the SAP value increases.

[0145] Figure 6The macroscopic data shows the correlation between different cross-sectional characteristic variables and total phosphorus concentration. The correlation at the Fumin Bridge monitoring point, from largest to smallest, is: turbidity, conductivity, pH, ammonia nitrogen, total nitrogen, permanganate index, dissolved oxygen, and water temperature. The correlation at the Cuijiazhuang monitoring point, from largest to smallest, is: total nitrogen, ammonia nitrogen, turbidity, water temperature, permanganate index, pH, conductivity, and dissolved oxygen.

[0146] The specific embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of the present invention.

Claims

1. A Transformer-BiLSTM water quality prediction method based on SHAP interpretability analysis, characterized in that, Includes the following steps: (1) Obtain historical multi-parameter water quality monitoring data of the target water area; (2) Clean and normalize the data; (3) Divide the data into training set, validation set and test set; (4) The Transformer encoder performs embedding encoding and positional encoding, and uses a self-attention mechanism to calculate the time step; (5) The BiLSTM neural network receives the feature sequence output by the Transformer encoder, learns the dependencies between time series by forward and backward LSTM, and concatenates the forward and backward hidden states of each time step to obtain bidirectional encoded features. (6) Calculate the mean square error RMSE, mean absolute error MAE, and coefficient of determination R²; (7) Calculate the SHAP value and interpret the Transformer-BiLSTM water quality prediction results; Step (4) is as follows: After the water quality sequence is input into the Transformer encoder, the relationship between time steps is learned. The sequence information is introduced by the position encoding. The position encoding is added to the feature vector of the time step. The self-attention mechanism calculates the correlation between time steps and generates a weighted output. Each input feature vector is transformed by three matrices to obtain the Q, K, and V matrices. The path is passed through residual connection, layer normalization and feedforward network. The Transformer encoder outputs a high-dimensional representation sequence. Step (5) is as follows: the forward Lstm processes the first sequence to obtain the forward hidden state, and the backward Lstm processes the last sequence to obtain the backward hidden state. In the forget gate, the Sigmoid function filters out invalid water quality data, and the input gate filters out valid water quality data through independent neural network layers and the Sigmoid function. After the data unit state is updated, the forward and backward hidden states are concatenated to obtain bidirectional encoded features and output the prediction results. Step (7) is as follows: the interpreter uses DeepExplainer to determine the baseline prediction value, the feature subset evaluates the model output, the Shapley formula is used to calculate the marginal contribution of each feature, the sorted samples are decomposed, and the Shaple value is calculated.

2. The Transformer-BiLSTM water quality prediction method based on SHAP interpretability analysis according to claim 1, characterized in that, Step (1) is as follows: Obtain the water quality time series of a certain water area monitoring point, including pH, , TN, TP, DO, water temperature, conductivity, and turbidity.

3. The Transformer-BiLSTM water quality prediction method based on SHAP interpretability analysis according to claim 1, characterized in that, Step (2) is as follows: smooth missing data using a time series-based cubic spline function; process outliers using the Z-score method; and delete duplicate values.

4. The Transformer-BiLSTM water quality prediction method based on SHAP interpretability analysis according to claim 1, characterized in that, Step (3) is as follows: Divide the data into training set, validation set and test set in a ratio of 8:1:1.

Images