A Time Series Data Prediction Method Based on Time Series Decomposition and LSTM

Abstract

The present invention discloses a time series data prediction method based on time series decomposition and LSTM, including: Step 1, collect time series data, preprocess it to obtain a time series sample set that meets the data requirements for establishing a prediction model, and divide it into a training set and a test set to obtain a first training set and a first test set; Step 2, establish a first neural network for trend component and remainder prediction based on LSTM, train and tune parameters through the first training set, use the trained first neural network model to predict the first training set, obtain the prediction results of the trend component and remainder of the first training set, and further process them into a second training set; Step 3, establish a second neural network based on ANN, train and tune parameters through the second training set; Step 4, use the trained first neural network and the second neural network model to jointly predict the test set to obtain the predicted results of the fitted time series data.

Description

Technical Field

[0001] The present invention relates to a time series data prediction method based on time series decomposition and LSTM, and belongs to the technical field of time series data prediction. Background Art

[0002] Time series data prediction is to simulate the variation law of data through the analysis of historical data, and then predict the data at future time points. Time series prediction is of great significance in many applications such as stock analysis, air pollution monitoring, traffic flow scheduling, etc.

[0003] Common time series data prediction methods include statistical methods and machine learning methods. In recent years, with the continuous development of deep learning, recurrent neural networks (RNNs) and long short-term memory networks (LSTMs) have shown good prediction performance in data prediction. However, common methods cannot well predict complex non-linear time series. Although the performance of deep learning methods has been improved, the training time is long and there is an overfitting phenomenon. In addition, the problem of prediction lag still exists, and it cannot adapt to the variation characteristics of time series data, especially the prediction of non-stationary burst data is inaccurate.

[0004] Time series decomposition plays an important role in time series analysis. Time series data is generally decomposed into three parts: trend component, periodic component, and remainder component. Summary of the Invention

[0005] The present invention designs and develops a time series data prediction method based on time series decomposition and LSTM. By using time series decomposition to separate the trend and period of time data and combining it with LSTM, the prediction accuracy can be effectively improved. In a traffic flow monitoring system, through the real-time prediction of traffic conditions, it supports the pre-scheduling of traffic flow and can solve the traffic congestion phenomenon.

[0006] The technical solution provided by the present invention is as follows:

[0007] A time series data prediction method based on time series decomposition and LSTM, comprising:

[0008] Step 1: Collect time series data, establish a time series sample set required for the prediction model data, and divide it into a training set and a test set to obtain a first training set and a first test set;

[0009] Step 2: Based on LSTM, establish a first neural network for predicting trend components and residuals, train and tune parameters through the first training set, use the trained first neural network model to predict the first training set, obtain the prediction results of the trend components and residuals of the first training set, and further process them into a second training set;

[0010] Step 3: Based on ANN, establish a second neural network, train and tune parameters through the second training set, and obtain a trained second neural network;

[0011] Step 4: Jointly predict the first test set using the trained first neural network and the second neural network model to obtain the prediction results of the fitted time series data.

[0012] Preferably, the said Step 1 includes:

[0013] Perform first-order difference and second-order difference on the original time series data, and decompose the original time series data into periodic components, trend components, and residuals;

[0014] Determine the input and output of the first neural network by the method of sliding window. Let the window size be n, and the window slides 1 each time. The input is: the first n - 1 of six kinds of data before the window, and the output is: the nth trend component of the window and the nth residual of the window;

[0015] Among them, the six kinds of data include: the original sequence, the first-order difference sequence, the second-order difference sequence, the periodic component sequence, the trend component sequence, and the residual sequence;

[0016] Divide the input and output data samples of the first neural network according to 0.95:0.05, and perform normalization respectively to obtain the first training set and the first test set.

[0017] Preferably, the said Step 2 includes:

[0018] Use the first training set to train and tune the parameters of the first neural network model to obtain a trained first neural network model;

[0019] Predict the trend components and residuals of the first training set and the first test set through the trained first neural network.

[0020] Preferably, the said Step 3 includes:

[0021] Use the trend components, residuals, and corresponding periodic components predicted by the first neural network as the input of the second neural network, and the corresponding original time series data as the output of the second neural network, and construct the second neural network based on ANN;

[0022] Train and tune the parameters of the second neural network through the second training set to obtain a trained second neural network.

[0023] Preferably, step four includes:

[0024] Process the first test set using the first neural network, and use the predicted trend component, remainder, and corresponding periodic component as the input of the second neural network, and the corresponding original time series data as the output of the second neural network to obtain the second test set.

[0025] Predict the second test set using the trained second neural network, and denormalize the prediction result to obtain the predicted result of the finally fitted time series data.

[0026] Preferably, in step one, the original time series data is decomposed by the additive model of the decomposition method, and the additive model is:

[0027] f(t) = S(t) + T(t) + R(t);

[0028] In the formula, S(t) is the periodic component, T(t) is the trend component, and R(t) is the remainder.

[0029] Preferably, in step one, a period of 60 is selected when decomposing the time series, and the moving average is used when decomposing the time trend.

[0030] The beneficial effects of the present invention are as follows:

[0031] 1) The present invention uses time series decomposition to separate the trend component, periodic component, and remainder of traffic flow / network flow time series data, predicts the trend component and remainder, and then synthesizes the predicted trend component, remainder, and periodic component into time series data. Combining the neural network with time series difference and decomposition makes the overall data prediction become three independent partial predictions, reduces the difficulty of traffic flow / network flow prediction, and thus effectively improves the accuracy of traffic flow / network flow prediction.

[0032] 2) The present invention uses time series decomposition to obtain a clear periodicity of traffic flow / network flow, and has a better prediction effect in the case of a sudden increase in the periodicity of traffic flow / network flow time series data, improving the performance of traffic scheduling / network monitoring.

[0033] 3) In the streaming computing platform, the application-oriented proactive elastic resource scheduling is based on the prediction of input stream data. Accurate prediction is beneficial to improving the performance of resource scheduling, can save resource usage, and thus save system energy consumption.

[0034] 4) In the traffic flow monitoring system, through the real-time prediction of traffic conditions, it supports the pre-scheduling of traffic flow and can solve the traffic congestion phenomenon Description of the Drawings

[0035] Figure 1Schematic diagram of the principle of the prediction method of the present invention

[0036] Figure 2 Schematic flowchart of the time series data prediction method provided by the embodiment of the present invention

[0037] Figure 3 Decomposition result diagram of the first 500 data time series of the embodiment of the present invention

[0038] Figure 4 Comparison diagram of the prediction results of the embodiment of the present invention

[0039] Figure 5 Comparison diagram of the results predicted by the traditional LSTM network Detailed implementation manners

[0040] The following further describes the present invention in detail with reference to the accompanying drawings, so that those skilled in the art can implement it according to the description in the specification.

[0041] As Figures 1-5 shown, the present invention provides a time series data prediction method based on time series decomposition and LSTM, which separates the trend and period of time data by time series decomposition and combines it with LSTM, thereby effectively improving the prediction accuracy.

[0042] Using python software, the version number is: python3.7, tensorflow1.14, the running software environment: windows10, the hardware configuration: processor AMD Ryzen 5 4600H with Radeon Graphics(12 CPUs), 3.0GHz, memory 16G RAM, graphics card NVIDIA GeForce GTX 1650;

[0043] The first neural network model: As Figure 1 shown in the LSTM network in: The first layer is the LSTM layer with 100 neurons. The second layer is the Dense layer with 50 neurons. The third layer is divided into two parts. One part is used to predict the trend component. In the embodiment of the present invention, this part uses two Dense layers to fit the trend component. The first Dense layer has 24 neurons, and the second Dense has 1 neuron, which is used to output the prediction result of the trend component; the other part is used to predict the remainder. This part uses 1 Dense layer to fit the remainder, and there is only one neuron, which is used to output the prediction result of the remainder.

[0044] Input format: [None, 60, 6], where 6 represents 6 types of input data: the original sequence, the first-order difference sequence, the second-order difference sequence, the periodic component sequence, the trend component sequence, and the remainder sequence. 60 refers to the first 60 data in each sliding window. None represents the number of groups input each time.

[0045] Output format: [None, 2], where 2 represents the prediction of the 61st trend component and remainder in the sliding window by the neural network, and None represents the number of groups output each time.

[0046] The second neural network model: As Figure 2 shown in the ANN network in, the embodiment of the present invention sets 3 layers, all of which are Dense layers, and the number of neurons is: 50, 24, 1 respectively. The last layer is the output layer for outputting the prediction result.

[0047] Input format: [None, 3], where 3 represents 3 types of input data: the two outputs of the first neural network model and the 61st periodic component in each sliding window. None represents the number of groups input each time.

[0048] Output format: [None, 1], where 1 represents the prediction of the 61st original time series data in the sliding window by the second neural network, and None represents the number of groups output each time.

[0049] The prediction model is composed of two neural network models, namely the first neural network model for predicting the trend component and the remainder, and the second neural network model for fitting the trend component, the periodic component, and the remainder into the corresponding time series data. When processing experimental data, two sets of training sets and test sets are required, namely the first training set and the first test set for training and testing the first neural network model, and the second training set and the second test set for training and testing the second neural network model.

[0050] Since the input of the second neural network model needs to use the output of the first neural network model, the second training set and the second test set can only be obtained after the first neural network model is trained. Specifically, it includes:

[0051] Step 1: Collect time series data, preprocess it, obtain a time series sample set that meets the data requirements for establishing the prediction model, and divide it into a training set and a test set;

[0052] Among them, the experimental data is sourced from the text in the covid19_twitter dataset, which contains tweets related to COVID-19 on Twitter. The time range is 31 days starting from August 1, 2021. Considering that there are many countries and regions using English, in order to enrich the data sample, the experimental data uses English text data. There are approximately 150,000 to 200,000 pieces of data per day. When obtaining specific tweets, a developer account on Twitter needs to be applied for, and during the application process, multiple emails need to be sent to indicate the purpose of the data obtained and other information. The COVID_19_dataset_Tutorial in this GitHub project (https: / / github.com / thepanacealab / covid19_twitter) can be used to quickly download the specific data.

[0053] The original data includes the number of likes, platform, text, comments, retweet information, number, and creation time of the tweets. In the present invention, as a preference, the original data is the creation time.

[0054] Process the original data, sort the original data by time, calculate the number of tweets per minute, and obtain the original time series data;

[0055] Perform first-order difference and second-order difference on the original time series data;

[0056] Use the additive model of the classical decomposition method to decompose the original time series data, obtaining the periodic component, trend component, and remainder;

[0057] Determine the input and output of the first neural network through the sliding window method. Let the window size be n, and the window slides by 1 each time. The input is: the first n - 1 of six types of data before the window, and the output is: the nth trend component of the window and the nth remainder of the window, where n > 1;

[0058] Among them, the additive model is: f(t) = S(t) + T(t) + R(t);

[0059] In the formula, S(t) is the periodic component, T(t) is the trend component, and R(t) is the remainder;

[0060] When decomposing the time series, a period of 60 is selected, that is, one hour as a period;

[0061] When decomposing the trend component, the moving average is used. Since unknown data needs to be predicted using known data, when decomposing the time series, the data to be predicted cannot be used as the input for decomposition, and only past values are used for calculating the moving average;

[0062] When calculating the moving average, the mean can be calculated using the current data and the k values before and after it. If decomposed in this way, the three components decomposed will contain information on future time series data, where k ≥ 1.

[0063] The results of time series decomposition for the first 500 data are as Figure 3 shown. Since only past values are used when calculating the moving average, the trend component and remainder for the first 60 data are null values and are filled with 0;

[0064] The input and output of the first neural network are obtained by using the sliding window method for the six types of data: the original sequence, the first-order difference sequence, the second-order difference sequence, the periodic component sequence, the trend component sequence, and the remainder sequence. The window size is 61, and it slides by 1 each time. The first 60 of the six types of data in the window are used as the input of the first neural network, and the trend component and remainder of the 61st in the window are used as the output of the first neural network;

[0065] The input and output data samples of the first neural network are divided according to 0.95:0.05 to obtain the first training set and the first test set;

[0066] Delete the abnormal data of 0 value and the 60 data before and after it, and use the Min-Max Normalization method to normalize the training set and the test set to obtain the first training set and the first test set;

[0067] Step 2: Based on LSTM, establish the first neural network for predicting the trend component and the remainder. The model structure is as Figure 1 shown by the LSTM in it. The first layer is the LSTM layer, the second layer is the Dense layer, and the third layer is divided into two parts. One part is used to predict the trend component. In the embodiments of the present invention, this part uses two Dense layers to fit the trend component; the other part is used to predict the remainder, and this part uses one Dense layer to fit the remainder. Train and adjust the parameters through the first training set, and use the trained first neural network model to predict the first training set to obtain the prediction results of the trend component and the remainder of the first training set;

[0068] Use the first training set to train and adjust the parameters of the first neural network model, set the learning rate to 0.001, and the number of training times to 20 times;

[0069] Use the trend component and the remainder in the first training set and the first test set predicted by the first neural network model and the corresponding periodic component as the input of the second neural network model, and the corresponding original time series data as the output of the second neural network membrane to obtain the second training set;

[0070] Step 3: Based on ANN, establish the second neural network. The model structure is as Figure 1As shown in the ANN network, since it is relatively simple to fit the original time series data with three decomposed components, an ANN network is connected by three Dense layers and trained and parameter-tuned using the second training set;

[0071] The second neural network model has 3 inputs, namely the trend component, the remainder term, and the periodic component of the 61st data in the sliding window. Among them, the trend component and the remainder term are the prediction results of the first neural network, and the periodic component has periodicity and does not require prediction. It has 1 output, which is the 61st original time series data in the sliding window, that is, the data to be predicted;

[0072] The second neural network model is trained and parameter-tuned using the second training set, with the learning rate set to 0.001 and the number of training times set to 20;

[0073] Step 4: Jointly predict the first test set using the trained first neural network and the second neural network model to obtain the predicted result of the fitted time series data;

[0074] Use the first neural network to process the first test set, take the predicted trend component, remainder term, and the corresponding periodic component as the inputs of the second neural network, and the corresponding original time series data as the output of the second neural network to obtain the second test set;

[0075] Use the second neural network model to predict the second test set, and compare the predicted result after anti-normalization with the true result.

[0076] Among them, the test set is divided into the first test set and the second test set. The first test set is used to test the first neural network. The second test requires the results of the first neural network. After the first test set is input into the first neural network, the output result and part of the data in step 1 are processed to obtain the second test set.

[0077] As Figure 4 shown, for the predicted results of the first 500 data in the test set, it can be seen that there is a good prediction for the position of the wave crest, and the problem of prediction lag in traditional neural networks is better solved.

[0078] As Figure 5 shown, a comparison is made with the traditional model that only uses LSTM:

[0079] Both prediction methods are measured in terms of four metrics: mean squared error (MSE), root mean squared error (RMSE), mean absolute error (MAE), and coefficient of determination (R2).

[0080] Evaluation results of the traditional LSTM model: MSE: 343.57, RMSE: 18.53, MAE: 13.41, R2: 0.717. Evaluation index values of the model of the present invention: MSE: 192.19, RMSE: 13.86, MAE: 10.54, R2: 0.842. Among the above four measurement indicators, the smaller the values of the first three, the smaller the error between the obtained predicted value and the observed value. The closer the final R2 is to 1, the better the prediction result. Through the comparison data of the four groups of measurement indicators obtained above, it is further verified that the time series data prediction method based on time series decomposition and LSTM of the present invention can effectively improve the accuracy of the final prediction result.

[0081] In summary, when predicting time series data, the present invention first decomposes the time series data for processing, predicts the decomposed amounts, and finally synthesizes the predicted decomposed amounts to obtain the prediction of the time series data, and proposes a time series data prediction method based on time series decomposition and LSTM. Compared with the existing LSTM, this method has better performance in predicting streaming data. In a streaming computing platform, application-oriented proactive elastic resource scheduling is based on the prediction of input stream data. Accurate prediction is beneficial to improving the performance of resource scheduling, can save resource usage, and thus save system energy consumption.

[0082] Although the embodiments of the present invention have been disclosed as above, it is not limited to the applications listed in the specification and embodiments. It can be fully applied to various fields suitable for the present invention. For those familiar with the field, additional modifications can be easily made. Therefore, without departing from the general concept defined by the claims and the equivalent scope, the present invention is not limited to the specific details and the illustrations shown and described herein.

Claims

1. A time series data prediction method based on time series decomposition and LSTM, characterized in that, it includes: Step 1: Collect time series data, establish a time series sample set required for the prediction model, and divide it into a training set and a test set to obtain a first training set and a first test set, including: Perform first-order difference and second-order difference on the original time series data, and decompose the original time series data into a periodic component, a trend component, and a remainder; Determine the input and output of the first neural network by the sliding window method. Let the window size be n, and the window slides 1 each time. The input is: the first n - 1 of six kinds of data before the window, and the output is: the nth trend component of the window and the nth remainder of the window; Among them, the six kinds of data include: the original sequence, the first-order difference sequence, the second-order difference sequence, the periodic component sequence, the trend component sequence, and the remainder sequence; Divide the input and output data samples of the first neural network according to 0.95:0.05, and normalize them respectively to obtain a first training set and a first test set; The time series data is obtained through the following method: Process the original data, sort the original data by time, calculate the number of tweets per minute, and obtain the time series data; The original data includes the like count, platform, text, evaluation and creation, repost information, number, and creation time of the tweet; Step 2: Build a first neural network for predicting the trend component and the remainder based on LSTM, train and tune parameters through the first training set, use the trained first neural network model to predict the first training set, obtain the prediction results of the trend component and the remainder of the first training set, and further process them into a second training set; Use the first training set to train and tune parameters of the first neural network model to obtain a trained first neural network model; Predict the trend component and the remainder of the first training set and the first test set through the trained first neural network; Step 3: Build a second neural network based on ANN, train and tune parameters through the second training set to obtain a trained second neural network; Step 4: Jointly predict the first test set using the trained first neural network and the second neural network model to obtain the predicted result of the fitted time series data; Use the first neural network to process the first test set, take the predicted trend component, remainder, and the corresponding periodic component as the input of the second neural network, and the corresponding original time series data as the output of the second neural network to obtain a second test set; Predict the second test set through the trained second neural network, and denormalize the prediction result to obtain the predicted result of the finally fitted time series data.

2. The time series data prediction method based on time series decomposition and LSTM according to claim 1, characterized in that, the step 3 includes: Take the trend component, remainder, and the corresponding periodic component predicted by the first neural network as the input of the second neural network, and the corresponding original time series data as the output of the second neural network, and build the second neural network based on ANN; Train and tune parameters of the second neural network through the second training set to obtain a trained second neural network.

3. The time series data prediction method based on time series decomposition and LSTM according to claim 2, characterized in that, in the first step, the original time series data is decomposed by the additive model of the decomposition method, and the additive model is: f(t) = S(t) + T(t) + R(t); wherein, S(t) is the periodic component, T(t) is the trend component, and R(t) is the remainder term.

4. The time series data prediction method based on time series decomposition and LSTM according to claim 3, characterized in that, in the first step, when decomposing the time series, a period of 60 is selected, and when decomposing the time trend, the moving average is used.

Images