Pipeline direct drinking water monthly water supply prediction method based on LSTM-GRU-Attention model

Abstract

The invention discloses a method for predicting the monthly water supply amount of pipeline direct drinking water based on an LSTM-GRU-Attention model. External factors such as temperature and historical monthly water supply are taken as an input time sequence, a long short-term memory network (LSTM), a gating cycle unit (GRU) and Attention are combined in a series connection mode, and firstly, the LSTM and the GRU capture a long-term dependency relationship in the time sequence and deal with a gradient disappearance or explosion problem; secondly, Attention weights different time steps in the sequence, and pays attention to the time step most relevant to the current task; and finally, outputting a prediction result. Through the method, the prediction precision is greatly improved, the prediction error is reduced, the prediction effect is also good under the condition of more water station data, a decision basis and technical support can be provided for urban water supply prediction, the model can efficiently predict the monthly water supply of the pipeline direct drinking water, a production plan is reasonably arranged for a water supply enterprise, and the economic benefit is improved. The scientific management level is improved; and a certain application value is realized.

Description

Technical Field

[0001] The present invention belongs to the technical field of pipeline direct drinking water supply prediction, and in particular relates to a pipeline direct drinking water monthly supply prediction method based on an LSTM-GRU-Attention model. Background Art

[0002] With the continuous advancement of urbanization and the continuous improvement of people's living standards, the growing needs of urban residents for a better life have led to the emergence of an important form of high-quality drinking water - piped drinking water, and the demand for healthy piped drinking water is increasing. In the operation of piped drinking water, the water supply sometimes has a structural surplus or shortage. Therefore, accurate and effective prediction of monthly water supply is of great significance to ensure the stability of residents' water use and optimize water resource allocation. As the basis for decision-making and planning of direct drinking water supply enterprises, the forecast of piped drinking water supply can help direct drinking water enterprises reasonably arrange production plans, reduce operating costs, and improve scientific management.

[0003] Existing prediction methods and models can provide great help for water supply prediction. At present, many scholars use intelligent algorithms to predict water supply in order to improve the accuracy of water supply prediction, which has improved the prediction accuracy and generalization ability to a certain extent. However, some models have large computational complexity, long training process, cumbersome tuning process, and low fitting degree. In view of the limitations of a single model, researchers have proposed the concept of a combined prediction model.

[0004] In summary, this method developed a method for predicting the monthly supply of piped drinking water based on the LSTM-GRU-Attention model. Summary of the invention

[0005] The present invention provides a method for predicting the monthly supply of piped drinking water based on the LSTM-GRU-Attention model. This method greatly improves the prediction accuracy, reduces the prediction error and the complexity of calculation, and can help drinking water companies to reasonably arrange production plans, reduce operating costs, and improve the level of scientific management.

[0006] In order to solve the above technical problems, a technical solution adopted by the present invention is: to provide a method for predicting the monthly supply of direct drinking water from pipelines based on the LSTM-GRU-Attention model, comprising the following steps:

[0007] 1) Feature engineering: The collected data are the monthly water supply data of direct drinking water from pipes, as well as the number of users, holidays, temperature and other meteorological data;

[0008] 2) Preprocessing of monthly piped drinking water supply data: When there is missing time data, the average method is used to fill it in to obtain a complete data set; the complete data set and external feature engineering are divided into training set, test set and validation set.

[0009] 3) Build LSTM-GRU-Attention neural network model:

[0010] Use LSTM and GRU models serially to capture long-term dependencies in time series and deal with the problem of gradient disappearance or explosion; the Attention layer weights the different time steps in the output sequence of the LSTM-GRU model and focuses on the time step that is most relevant to the current task; finally, the model outputs a prediction.

[0011] 4) Determine the LSTM-GRU-Attention neural network model:

[0012] In the training phase, the training data set is input into the LSTM-GRU-Attention neural network model to train the neural network parameters of the model;

[0013] In the verification phase, the verification data set is input into the constructed LSTM-GRU-Attention neural network model to fine-tune the neural network parameters of the model.

[0014] During the testing phase, the test data set is input into the constructed LSTM-GRU-Attention neural network model, and the independent variable input values ​​in the test data are input into the trained neural network model to obtain the test results. When the prediction accuracy requirements are met, the model is determined to perform monthly water supply prediction.

[0015] In step 1, data collected are the monthly water supply, monthly temperature, number of users at each station, etc. of an independent water station of a water supply enterprise in northern China from January 2016 to June 2024. The water supply and number of users can be collected by sensors or manually recorded, and meteorological data such as temperature can be obtained from China Weather Network.

[0016] In step 2, the Pearson correlation coefficient is used to measure the linear correlation between the variables of the monthly water supply data of direct drinking water from the pipeline, the number of users, holidays, and meteorological data such as temperature, so as to find the external factors related to the monthly water supply.

[0017] The LSTM-GRU-Attention model in step 3 uses a layer of LSTM neural network, a layer of GRU neural network, a layer of Attention layer and a layer of Dense fully connected network.

[0018] LSTM neural network module: LSTM generates a long gradient path by introducing a self-loop design. The weight of the self-loop changes with each iteration. The input gate (i-gate), forget gate (f-gate) and output gate (o-gate) are added, so that the error does not need to be attributed layer by layer during the propagation process, and some errors can be directly passed to the next layer of the network, effectively solving problems such as long-term dependence, gradient disappearance and gradient explosion.

[0019] GRU neural network module: GRU is a neural network improved on LSTM. It has all the functions of LSTM. Its simplified internal structure of LSTM network reduces the parameters required for training and improves the training speed of the model. t and the moment before h t-1 Increased linear correlation.

[0020] Attention layer module: The attention layer can better learn the peaks and troughs of the time series and capture the patterns and trends of water volume changes.

[0021] Dense fully connected network module: prevents unstable network layer stacking.

[0022] The LSTM-GRU-Attention model establishment module in step 4 is used to implement the following stages:

[0023] During the training phase, the training data set is input into the LSTM-GRU-Attention neural network model, and the neural network parameters of the training model are:

[0024] The preprocessed training data is used to input the features into the LSTM neural network prediction model for feature extraction; the monthly water supply of each water station, the closely affected temperature value, and the number of users are used as input time series to flow into the expanded LSTM layer, and the output is a sequence of memory state updates for each time step.

[0025] The update sequence output from the LSTM layer flows into the GRU that is also expanded. The output length remains unchanged and the state update sequence is remembered at each time step. At the same time, the model accurately remembers the traffic prediction target in each time period and grasps the overall traffic change trend.

[0026] The Attention layer is then used to calculate the dot product between each time step in the sequence and the attention weight, and the output sequence is weighted to obtain the time step most relevant to the current task, that is, to better learn the peaks and troughs of the time series and capture the laws and trends of water volume changes.

[0027] In order to prevent the instability of network layer stacking, the fully connected neural network (Dense) layer is used to adjust the output vector, and finally the fully connected layer is output to obtain the prediction result.

[0028] Use the predicted value to input into the loss function and compare it with the true value to reverse the gradient to adjust the parameters in the neural network to improve the prediction accuracy;

[0029] During the verification phase, fine-tune the LSTM-GRU-Attention neural network model parameters based on a small batch verification dataset:

[0030] It is executed after the data training of each batch training phase is completed. By inputting the verification data set, the verification output value is obtained based on the LSTM-GRU-Attention neural network model, and the parameters of the neural network are fine-tuned according to the gap between the predicted value and the true value;

[0031] During the testing phase, the test data set is input into the constructed LSTM-GRU-Attention neural network model, and the independent variable input values ​​in the test data are input into the trained neural network model to obtain the test results. When the prediction accuracy requirements are met, the model is determined to perform monthly water supply prediction.

[0032] Traditional time series forecasting methods have difficulty in obtaining high-precision forecasts. This method can capture the dependencies in the data, thereby obtaining a more comprehensive representation of the input data.

[0033] Beneficial effects: The combined prediction model is obtained by weighting the LSTM model, GRU model and Attention in series. This method can effectively improve the prediction accuracy, reduce the prediction error, and has good prediction effect when there are more data sets. Therefore, it can provide decision-making basis and technical support for water supply enterprises. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 LSTM-GRU-Attention network structure diagram

[0035] Figure 2 A comparison chart of LSTM-GRU-Attention and other models at a random water station without external factors

[0036] Figure 3 Comparison chart of LSTM-GRU-Attention and other models under external factors at a random water station DETAILED DESCRIPTION

[0037] The method of the present invention is further described below in conjunction with the accompanying drawings and specific implementation methods.

[0038] The method for predicting monthly piped drinking water supply based on the LSTM-GRU-Attention model implemented in this embodiment includes the following steps:

[0039] 1) Feature Engineering: The experiment collects monthly piped drinking water supply data, as well as meteorological data such as the number of users, holidays, and temperature. Specifically, the process is as follows:

[0040] A total of 10,506 sets of data were collected, including the monthly water supply from January 2016 to June 2024, as well as the monthly temperature and the number of users at each site.

[0041] 2) Preprocessing of monthly piped drinking water supply data: Preprocess the monthly water supply time series based on data preprocessing technology: When there is missing time data, use the average method to fill it in to obtain a complete data set; divide the complete data set and external feature engineering into training set, test set and validation set. When dividing the data set, the specific division is as follows:

[0042] The data from January 2016 to December 2020 is used as the training set to train the model to obtain the optimal parameters; the data from January 2021 to December 2021 is used as the validation set to verify the generalization ability of the model; the data from January 2022 to June 2024 is used as the test set to evaluate the performance of the model under the model structure.

[0043] Selection of external feature engineering:

[0044] In order to verify the correlation of the influence of each external eigenvector on the monthly water supply forecast, the Pearson correlation coefficient is used to measure the linear correlation between two variables. The calculation formula of the correlation coefficient R is:

[0045]

[0046] Where: X i and Y i They are two different data sets, i represents the i-th pair of values, and is their respective average, n is the total number, and the correlation coefficient R ranges from -1 to 1.

[0047] 3) Build LSTM-GRU-Attention neural network model:

[0048] Use LSTM and GRU models serially to capture long-term dependencies in time series and deal with the problem of gradient disappearance or explosion; the Attention layer weights the different time steps in the output sequence of the LSTM-GRU model and focuses on the time step that is most relevant to the current task; finally, the model outputs a prediction.

[0049] The specific process of inputting the monthly water supply time series of pipeline direct drinking water through LSTM is as follows:

[0050] 1) Use the forget gate to forget the water supply data that has little impact on the current prediction and filter out valid data.

[0051] f t =σ(W f ·[h t-1 , x t ]+b f ) (2)

[0052] 2) The current important monthly water supply data is determined through the input gate and then added to the candidate unit status to reflect the latest water supply situation.

[0053] i t =σ(W i ·[h t-1 , x t ]+b i ) (3)

[0054]

[0055] 3) The cell state is the memory part of the LSTM network, which is updated according to the output of the forget gate and the input gate to reflect the historical trend and latest changes of the water supply.

[0056]

[0057] 4) The output gate processes the prediction output of the hidden state, which is used to generate the prediction of monthly water supply.

[0058] o t =σ(w o ·[h t-1 , x t ]+b o ) (6)

[0059] h t =o t *tanh(C t ) (7)

[0060] Where: At time t, C t , is the content of the memory unit; W f , W i , W c are the weight matrices of the corresponding functions; h t-1 is the input value at time t-1; x t is the input value at time t, b, b i is the deviation vector; h t is the hidden layer quantity at time t, σ and tanh are activation functions.

[0061] The time series output by LSTM is used as the input of the monthly water supply time series of pipeline direct drinking water. The specific steps are as follows through GRU:

[0062] 1) The update gate determines the state h before the current time step t-1 For the new state h t The influence of the model determines how much information from previous months is retained when predicting the water supply in the current month.

[0063] r t =σ(W xr x t +W hr h t-1 +b r ) (8)

[0064] 2) The reset gate determines the degree of influence of the current input on the hidden state, and it can ignore historical information that is not relevant to the current prediction.

[0065] z t =σ(W xh x t +W hz h t-1 +b z ) (9)

[0066] 3) The candidate hidden state represents the comprehensive state based on the current month's data and historical information, which can reflect the seasonal changes or trends in water supply. Then, the final hidden state updates the parameters.

[0067]

[0068]

[0069] Where: r t 、z t , Represent the reset gate, update gate and input vector respectively; ⊙ is the Hadamard product; W xr , W hr , W xh , W hz , W hh are the weight matrices of the corresponding functions respectively.

[0070] The time series output by GRU uses the Attention layer to calculate the dot product between each time step in the sequence and the attention weight, and weights the output sequence to obtain the time step most relevant to the current task, that is, to better learn the peaks and troughs of the time series and capture the laws and trends of water volume changes. The Attention function is:

[0071]

[0072] Where: w s , b s , v s is the network parameter, N represents the length of the input sequence, and the softmax function is used to generate a set of weights to indicate which parts of the input data the model should focus on when making predictions. Assume that the current time is i, and the output time series of the GRU at the kth time in the past is h i-k . i-k Input into the Attention mechanism layer to obtain the corresponding monthly water supply feature attention distribution coefficient α i-k , which represents the output time series h i-k The importance of the monthly water supply forecasting task. i-k The output time series is weighted and summed as the weighting coefficient.

[0073] To prevent the network layer stacking from being unstable, the output vector of Attention is adjusted using the fully connected neural network (Dense) layer, and finally the fully connected layer is output.

[0074] 4) Determine the LSTM-GRU-Attention neural network model:

[0075] During the training phase, the training data set is input into the LSTM-GRU-Attention neural network model to train the neural network parameters of the model.

[0076] The validation phase is performed after each batch of training data is trained. By inputting the validation data set, the validation output value is obtained based on the LSTM-GRU-Attention neural network model, and the parameters of the neural network are fine-tuned according to the gap between the predicted value and the true value.

[0077] In the testing phase, the LSTM-GRU-Attention neural network obtained in the training and validation phases is used to input the test data set to obtain the output test prediction value; the test prediction value is used to evaluate the prediction accuracy of the neural network model using the MAPE, MAE, and RMSE evaluation indicators. When the prediction accuracy meets the requirements, the model is determined to perform monthly water supply prediction.

[0078] The mean absolute error (MAE), root mean square error (RMSE), and mean absolute percentage error (MAPE) were selected to comprehensively evaluate the performance and quality of the model:

[0079]

[0080]

[0081]

[0082] Where: n represents the total number of monthly piped drinking water supply flow sequence; y i represents the measured flow value at time i; f(x i ) represents the deterministic prediction of traffic flow at time i.

Claims

1. A method for predicting monthly supply of direct drinking water from pipelines based on LSTM-GRU-Attention model, characterized in that: The method comprises the following steps: 1) Feature engineering: The collected data are the monthly supply of piped drinking water, as well as meteorological data such as the number of users, holidays, and temperature. 2) Preprocessing of monthly piped drinking water supply data: When there is missing time data, the average method is used to fill it in to obtain a complete data set; the complete data set and external feature engineering are divided into training set, test set and validation set. 3) Build LSTM-GRU-Attention neural network model: Use LSTM and GRU models serially to capture long-term dependencies in time series and deal with the problem of gradient disappearance or explosion; the Attention layer weights the different time steps in the output sequence of the LSTM-GRU model and focuses on the time step that is most relevant to the current task; finally, the model outputs a prediction. 4) Determine the LSTM-GRU-Attention neural network model: During the training phase, the training data set is input into the LSTM-GRU-Attention neural network model to train the neural network parameters of the model. In the verification phase, the verification data set is input into the constructed LSTM-GRU-Attention neural network model to fine-tune the neural network parameters of the model. During the testing phase, the test data set is input into the constructed LSTM-GRU-Attention neural network model, and the independent variable input values ​​in the test data are input into the trained neural network model to obtain the test results. When the prediction accuracy requirements are met, the model is determined to perform monthly water supply prediction.

2. A method for predicting monthly supply of direct drinking water based on LSTM-GRU-Attention model according to claim 1, characterized in that: In step 1, data such as monthly water supply from January 2016 to June 2024, monthly temperature, and the number of users at each site are collected.

3. The method for predicting monthly supply of direct drinking water from pipelines based on the LSTM-GRU-Attention model according to claim 1 is characterized in that: In step 2, the Pearson correlation coefficient is used to measure the linear correlation between the monthly water supply data of piped drinking water, the number of users, holidays, and meteorological data such as temperature, and to find the external factors related to the monthly water supply.

4. The method for predicting monthly supply of direct drinking water based on the LSTM-GRU-Attention model according to claim 1 is characterized in that: The LSTM-GRU-Attention model in step 3 uses one LSTM neural network, one GRU neural network, one Attention layer, and one Dense fully connected network.

5. The method for predicting monthly supply of direct drinking water based on the LSTM-GRU-Attention model according to claim 1 is characterized in that: The LSTM-GRU-Attention model in step 4 establishes the module to implement the following stages: During the training phase, the training data set is input into the LSTM-GRU-Attention neural network model, and the neural network parameters of the training model are: 1) Use the preprocessed training data to input the features into the LSTM neural network prediction model for feature extraction; the monthly water supply of each water station, the closely affected temperature value, and the number of users are used as input time series to flow into the expanded LSTM layer, and the output is a sequence of memory state updates for each time step. 2) The update sequence output from the LSTM layer flows into the GRU that is also expanded, and the output length remains unchanged and the state update sequence is memorized at each time step. At the same time, the model accurately memorizes the traffic prediction target in each time period and grasps the overall traffic change trend. 3) The Attention layer is then used to calculate the dot product between each time step in the sequence and the attention weight, and the output sequence is weighted to obtain the time step most relevant to the current task, that is, to better learn the peaks and troughs of the time series and capture the laws and trends of water volume changes. 4) To prevent the instability of network layer stacking, the fully connected neural network (Dense) layer is used to adjust the output vector, and finally the fully connected layer is output to obtain the prediction result. 5) Use the predicted value to input into the loss function and compare it with the true value to reverse the gradient to adjust the parameters in the neural network to improve the prediction accuracy; The validation phase is executed after the data training of each batch of training phase is completed. By inputting the validation data set, the validation output value is obtained based on the LSTM-GRU-Attention neural network model, and the parameters of the neural network are fine-tuned according to the gap between the predicted value and the true value; During the testing phase, the test data set is input into the constructed LSTM-GRU-Attention neural network model, and the independent variable values ​​in the test data are input into the trained neural network model to obtain the test results. When the prediction accuracy requirements are met, the model is determined to perform monthly water supply prediction.

Images