Traffic sequence prediction method based on multivariate time series data analysis

Abstract

The application discloses a traffic sequence prediction method based on multivariate time series data analysis, adopts a random graph diffusion attention mechanism to extract global and local spatial features of a traffic sequence, uses time attention to extract time features, improves prediction accuracy, reduces memory usage of a model, and improves the effect of the model on long-term prediction. The traffic sequence prediction method based on multivariate time series data analysis solves the problems of insufficient short-term prediction accuracy, high calculation complexity, large memory occupation, and insufficient lightweight of the existing model while maintaining the accuracy of long-term prediction.

Description

Technical Field

[0001] This invention relates to the field of traffic forecasting technology, and in particular to a traffic sequence forecasting method based on multivariate time series data analysis. Background Technology

[0002] Traffic forecasting is a crucial service within Intelligent Transportation Systems (ITS), predicting future traffic conditions based on historical traffic data (such as traffic flow and speed) observed by sensors. This capability facilitates a wide range of services related to traffic management, urban computing, and public safety. Traffic forecasting models predict future traffic sequences by learning the characteristic patterns of past traffic sequences. Unlike other univariate time series forecasting problems, traffic forecasting is highly challenging. As an inherent phenomenon in the transportation field, the current traffic conditions at an intersection are influenced by the traffic conditions at nearby and distant intersections, implying a spatial dependency between them. Furthermore, the trend of traffic sequences is influenced by historical trends, making it crucial to fully consider the temporal dependencies of traffic sequences. The complex spatiotemporal correlations of traffic networks severely limit the effectiveness of traffic flow forecasting.

[0003] In recent years, an increasing number of new methods have been proposed, which have significantly improved the accuracy of short-term traffic prediction by modeling spatiotemporal features, and to some extent, also improved the accuracy of long-term traffic prediction. While many methods have demonstrated excellent performance in short-term traffic prediction over time, they have not significantly improved the performance of long-term traffic prediction. Existing methods typically use graph convolutional neural networks (CNNs) to model local spatial information and convolutional neural networks (CNNs) or recurrent neural networks (RNNs) to model temporal information. These methods focus on extracting local information but lack utilization of global information. Until the introduction of attention mechanisms (AT), deep learning had a general method to capture global features. In the latest models, GMAN uses attention mechanisms to build its model and has achieved state-of-the-art results in long-term traffic prediction.

[0004] GMAN employs an Encoder-Decoder architecture, using an attention mechanism to extract features from both spatial and temporal dimensions, and a transformed attention mechanism to mitigate errors between monthly traffic predictions based on historical traffic sequences. GMAN effectively improves the accuracy of long-term traffic forecasts, but it has some drawbacks:

[0005] 1. Spatial attention has high computational complexity, resulting in large memory usage;

[0006] 2. It lacks the ability to capture local features in the spatial dimension, which leads to a decrease in the accuracy of short-term predictions;

[0007] 3. The forecasting time is high and it is not lightweight enough. Summary of the Invention

[0008] The purpose of this invention is to provide a traffic sequence prediction method based on multivariate time series data analysis, which enables existing models to maintain long-term prediction accuracy while solving the problems of insufficient short-term prediction accuracy, excessive computational complexity leading to large memory consumption, and lack of lightweight design.

[0009] To achieve the above objectives, this invention provides a traffic sequence prediction method based on multivariate time series data analysis, comprising the following steps:

[0010] Step 1: First, preprocess the public transportation sequence dataset to obtain the training dataset and the validation dataset;

[0011] Step 2: Using the training dataset obtained in Step 1, 16 sets of training data are randomly selected each time and input into the random graph diffusion attention model. The historical time information, future time information and predefined graph structure information in each set of data are input into the spatiotemporal embedding generator in the graph diffusion attention model to generate historical spatiotemporal embedding and future spatiotemporal embedding.

[0012] Step 3: Input historical traffic data and historical spatiotemporal embedding into the encoder, extract spatial and temporal features using the graph diffusion attention module and the temporal attention module respectively, and fuse the two features using the gate fusion module to generate historical spatiotemporal features;

[0013] Step 4: Feed the historical spatiotemporal features, historical spatiotemporal embedding, and future spatiotemporal embedding into the transformation attention module to generate the corresponding output;

[0014] Step 5: Feed the future spatiotemporal embedding and the output of Step 4 into the decoder, extract spatial and temporal features using the graph diffusion attention module and the temporal attention module respectively, and fuse the two features using the gate fusion module to generate future spatiotemporal features. Finally, use the future spatiotemporal features to generate the final prediction sequence.

[0015] Step 6: Calculate the mean absolute error, root mean square error, and mean absolute percentage error between the final generated and predicted sequences obtained in Step 5. Then, backpropagate through the Adam optimizer to update the network parameters and finally obtain the trained random graph diffusion attention model.

[0016] Step 7: Using the model with updated network parameters from Step 6 and the test dataset obtained in Step 1, select 16 sets of test data as input and execute Steps 2 to 6, where the training data in Step 2 is replaced with the selected 16 sets of test data, and finally the prediction sequence generated based on the test data is obtained.

[0017] Step 8: Using the predicted sequence generated from the test data obtained in Step 7, calculate the mean absolute error between the predicted sequence and the true sequence. After obtaining the mean absolute error of all groups of data, calculate the average to obtain the final mean absolute error representing the model performance.

[0018] Step 9: Repeat steps 2 to 8 until the mean absolute error obtained in step 8 no longer decreases, indicating that the model has reached its optimal performance. Then the network parameters have been updated and the model training has ended.

[0019] Preferably, the specific method of step 1 is as follows: Select a suitable public transportation sequence dataset, set the length of the historical sequence and the length of the predicted sequence according to the requirements, divide the input data into segments, each set of data contains a historical sequence, a predicted sequence and the time information corresponding to the two sequences; divide all the historical sequences, predicted sequences and time information into training datasets and test datasets according to the groups, and complete the data preprocessing; after completing the data grouping, select 70% of the data as the training dataset and 30% of the data as the validation dataset.

[0020] Preferably, the graph diffusion attention model in step 2 includes a spatiotemporal embedding generator, an encoder that requires historical data and historical spatiotemporal embeddings from each set of input data for encoding, a transformation attention module that uses historical spatiotemporal embeddings and future spatiotemporal embeddings to transform the input data, and a decoder that requires future spatiotemporal embeddings and the output of the transformation attention module to decode and output a predicted sequence.

[0021] Preferably, the spatiotemporal embedding generator includes a temporal embedding generator for receiving time information and a spatial embedding generator for receiving graph structure information. The overall formula for the spatiotemporal embedding generator is shown below:

[0022]

[0023] in For time embedding, , These represent the time information within a day and the date information within a week, respectively. For spatial embedding, The adjacency matrix of the predefined road network is Node2vec(), which is the node2vec algorithm used to encode the road network information; E is the spatiotemporal embedding, which is divided into historical time spatiotemporal embedding. and history embedding .

[0024] Preferably, the encoder and its core module are the sum of the graph diffusion attention module and the temporal attention module, with the formula for the temporal attention module as follows:

[0025]

[0026] Let be the input sequence of the l-th layer encoder, B represent the result of concatenating the input sequence with the historical spatiotemporal embedding; W represent the learnable parameters of the model, and Q, K and V represent the problem matrix, key matrix and numerical matrix obtained after the block is linearly transformed, respectively.

[0027] Preferably, the graph diffusion attention module includes random graph attention and adaptive graph convolution, and the formula for random graph attention is as follows:

[0028]

[0029] in It is a randomly generated matrix after a predefined graph result masking operation. It is the output of single-layer random graph attention;

[0030] The formula for adaptive graph convolution is as follows:

[0031]

[0032] It consists of two independent node embeddings. It is based on , The generated adaptive matrix, It is the output of the convolution;

[0033] The formula for the graph diffusion attention module is expressed as follows:

[0034]

[0035] This is the output of the graph diffusion attention module.

[0036] Preferably, the formula for the conversion attention module in step 4 is as follows:

[0037]

[0038] This is the output of the Encoder. It is the output of the attention module.

[0039] Preferably, the formulas for the Mean Absolute Error (MAE), Root Mean Square Error (RMSE), and Mean Absolute Percentage Error (MAPE) described in step 8 are as follows:

[0040]

[0041] in, For predicted values, is the actual value; n represents the length of the sequence.

[0042] Therefore, the traffic sequence prediction method based on multivariate time-series data analysis described above has the following beneficial effects: In traffic prediction, the accuracy of long-term and short-term predictions, as well as the ability to extract global and local spatial features, are all important factors affecting the prediction effect. This invention uses a graph diffusion attention module to extract global and local features of the traffic sequence, improving prediction accuracy, reducing model memory usage, and utilizing local features to improve the fitting ability to subtle local fluctuations in the traffic sequence, thereby improving the model's short-term prediction performance.

[0043] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0044] Figure 1 This is a schematic diagram of the system flow of the present invention;

[0045] Figure 2 This is a detailed structural diagram of the present invention;

[0046] Figure 3 This is a schematic diagram of the spatiotemporal embedding generator of the present invention;

[0047] Figure 4 This is a schematic diagram of the graph diffusion attention module of the present invention;

[0048] Figure 5 This is a comparison chart of the mean absolute error of the present invention with 18 existing methods on 3 public datasets;

[0049] Figure 6 This is a comparison chart of GPU usage between the graph diffusion attention model and Informer under the same conditions. Detailed Implementation

[0050] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0051] Unless otherwise defined, the technical or scientific terms used in this invention shall have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. The terms "set," "install," and "connect" should be interpreted broadly; for example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal connection of two elements. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0052] Example

[0053] Figure 1 This is a schematic diagram of the system flow of the present invention; Figure 2 This is a detailed structural diagram of the present invention; Figure 3 This is a schematic diagram of the spatiotemporal embedding generator of the present invention; Figure 4 This is a schematic diagram of the graph diffusion attention module of the present invention; Figure 5 This is a comparison chart of the mean absolute error of the present invention with 18 existing methods on 3 public datasets; Figure 6 This is a comparison chart of GPU usage between the graph diffusion attention model and Informer under the same conditions.

[0054] like Figure 1 The diagram illustrates the overall structure of the invention. The data processing and segmentation section, located at the entry point of the invention's structure, is responsible for initially processing the raw data to form the data structure required for the prediction model.

[0055] The traffic sequence prediction method based on multivariate time series data analysis described in this invention includes the following steps:

[0056] Step 1: Select a suitable public transportation sequence dataset, set the length of the historical sequence and the length of the predicted sequence according to the requirements, and split the input data. Each data set contains a historical sequence, a predicted sequence, and the time information corresponding to the two sequences. Divide all the historical sequences, predicted sequences, and time information into training datasets and test datasets according to the groups and complete the data preprocessing. After completing the data grouping, select 70% of the data as the training dataset and 30% of the data as the validation dataset.

[0057] Step 2: Using the training dataset obtained in Step 1, 16 sets of training data are randomly selected each time and input into the random graph diffusion attention model. The historical time information, future time information and predefined graph structure information in each set of data are input into the spatiotemporal embedding generator in the graph diffusion attention model to generate historical spatiotemporal embeddings and future spatiotemporal embeddings.

[0058] like Figure 2 The diagram illustrates the overall structure of the Graph Diffusion Attention Model (RGDAN) of this invention. The RGDAN includes a spatiotemporal embedding generator, an encoder, a decoder, and a Transform Attention Module. Spatiotemporal embedding generation requires historical time information, future time information, and predefined graph structure information from each data set as input. The encoder requires historical data and historical spatiotemporal embeddings from each data set for encoding. The Transform Attention Module uses the historical and future spatiotemporal embeddings to transform the input data. The decoder requires the future spatiotemporal embeddings and the output of the Transform Attention Module to decode and output a predicted sequence.

[0059] like Figure 3 As shown, the spatiotemporal embedding generator consists of a temporal embedding generator and a spatial embedding generator. They receive temporal information and graph structure information, respectively, to generate corresponding embeddings, and then add them together to obtain the spatiotemporal embedding. The overall formula for the spatiotemporal embedding generator is shown below:

[0060]

[0061] in For time embedding, , These represent the time information within a day and the date information within a week, respectively. For spatial embedding, The adjacency matrix of the predefined road network is Node2vec(), which is the node2vec algorithm used to encode the road network information; E is the spatiotemporal embedding, which is divided into historical time spatiotemporal embedding. and history embedding .

[0062] Step 3: Using the spatiotemporal embedding obtained in Step 2, historical traffic data and historical spatiotemporal embedding are fed into the encoder for concatenation. Spatial and temporal features are extracted using the Graph DiffusionAttention Module and the Temporal Attention Module, respectively. The two features are then fused using the gate fusion module to generate historical spatiotemporal features.

[0063] The encoder's core modules are the sum of the GraphDiffusion Attention Module and the Temporal Attention Module. The formula for the Temporal Attention Module is shown below:

[0064]

[0065] Let B be the input sequence of the l-th layer encoder, and let W be the result of concatenating the input sequence with the historical spatiotemporal embedding. Let Q, K and V be the query matrix, key matrix and value matrix obtained after the blocks are linearly transformed, respectively.

[0066] like Figure 4 As shown, the graph diffusion attention module consists of Random Graph Attention and Adaptive Graph Convolutional. The formula for Random Graph Attention is as follows:

[0067]

[0068] in It is a randomly generated matrix after a predefined graph result masking operation. It is the output of single-layer random graph attention.

[0069] The formula for Adaptive Graph Convolutional is as follows:

[0070]

[0071] It consists of two independent node embeddings. It is based on , The generated adaptive matrix, It is the output of the convolution.

[0072] The formula for the graph diffusion attention module is expressed as follows:

[0073]

[0074] This is the output of the graph diffusion attention module.

[0075] Step 4: Feed the historical spatiotemporal features, historical spatiotemporal embedding, and future spatiotemporal embedding into the transformation attention module to generate the corresponding output. For example... Figure 2 As shown, by utilizing historical and future spatiotemporal embeddings, the encoder's output is transformed by a transformation attention module to obtain an output more adapted to the future spatiotemporal context. The formula for the transformation attention module is as follows:

[0076]

[0077] This is the output of the Encoder. It is the output of the attention module.

[0078] Step 5: Feed the future spatiotemporal embedding and the output of Step 4 into the decoder. Use the graph diffusion attention module and the temporal attention module to extract spatial and temporal features respectively. Use the gate fusion module to fuse the two features to generate future spatiotemporal features. Finally, use the future spatiotemporal features to generate the final prediction sequence.

[0079] Step 6: Calculate the mean absolute error (MAE), root mean square error (RMSE), and mean absolute percentage error (MAPE) between the final generated and predicted sequences obtained in Step 5. Then, perform backpropagation through the Adam optimizer to update the network parameters, ultimately obtaining the trained random graph diffusion attention model. The formulas for mean absolute error (MAE), root mean square error (RMSE), and mean absolute percentage error (MAPE) are shown below:

[0080]

[0081] in, For predicted values, is the actual value; n represents the length of the sequence.

[0082] Step 7: Using the model with updated network parameters from Step 6 and the test dataset obtained in Step 1, select 16 sets of test data as input and execute Steps 2 to 6, where the training data in Step 2 is replaced with the selected 16 sets of test data, and finally the prediction sequence generated based on the test data is obtained.

[0083] Step 8: Using the predicted sequence generated from the test data obtained in Step 7, calculate the mean absolute error (MAE) between the predicted sequence and the true sequence. After obtaining the mean absolute error (MAE) of all groups of data, calculate the average to obtain the final mean absolute error (MAE) representing the model performance.

[0084] Step 9: Repeat steps 2 to 8 until the mean absolute error (MAE) obtained in step 8 no longer decreases, indicating that the model performance has reached its optimal level. Then the network parameters have been updated and the model training has ended.

[0085] Figure 5 This table displays the experimental results of 18 methods on three real-world datasets under identical experimental conditions, measured by mean absolute error (MAE), root mean square error (RMSE), and mean absolute percentage error (MAPE). For each prediction outcome, the best-performing model's results are shown in bold in the table, while the model with the best overall performance is indicated by a wireframe. Figure 5 As shown in the table, the Graph Diffusion Attention Model (RGDAN) significantly improves long-term predictions compared to the other 17 methods. Compared to the state-of-the-art methods, RGDAN reduces the mean absolute error (MAE) by 2.33%, 2.15%, and 2.30%, the root mean square error (RMSE) by 2.36%, 2.78%, and 3.44%, and the mean absolute percentage error (MAPE) by 1.95%, 2.06%, and 5.23% on the three datasets, respectively.

[0086] Figure 6 The results show a comparison of the single-round training time and memory usage between the Graph Diffusion Attention Model (RGDAN) and GAMN under the same experimental conditions. It can be seen that compared with GMAN, the Graph Diffusion Attention Model reduces the single-round training time by 59.3% and memory usage by 52.2%.

[0087] Therefore, the traffic sequence prediction method based on multivariate time series data analysis described in this invention enables existing models to maintain long-term prediction accuracy while solving the problems of insufficient short-term prediction accuracy, excessive computational complexity leading to large memory consumption, and insufficient lightweight design.

[0088] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A traffic sequence prediction method based on multivariate time series data analysis, characterized in that, Includes the following steps: Step 1: First, preprocess the public transportation sequence dataset to obtain the training dataset and the test dataset. The specific method is as follows: Select the public transportation sequence dataset, set the length of the historical sequence and the length of the predicted sequence according to the requirements, and divide the input data into segments. Each set of data contains a historical sequence, a predicted sequence, and the time information corresponding to the two sequences. Divide all the historical sequences, predicted sequences, and time information into training datasets and test datasets according to the segments to complete the data preprocessing. After completing the data grouping, 70% of the data was used as the training dataset and 30% of the data was used as the test dataset. Step 2: Using the training dataset obtained in Step 1, 16 sets of training data are randomly selected each time and input into the random graph diffusion attention model. The historical time information, future time information and predefined graph structure information in each set of data are input into the spatiotemporal embedding generator in the random graph diffusion attention model to generate historical spatiotemporal embedding and future spatiotemporal embedding. The random graph diffusion attention model includes a spatiotemporal embedding generator, an encoder that requires historical data and historical spatiotemporal embeddings from each set of data for encoding, a transformation attention module that uses historical spatiotemporal embeddings and future spatiotemporal embeddings to transform the input data, and a decoder that decodes the future spatiotemporal embeddings and the output of the transformation attention module to output a predicted sequence. The spatiotemporal embedding generator includes a temporal embedding generator for receiving time information and a spatial embedding generator for receiving graph structure information. The core modules of both the encoder and decoder consist of a graph diffusion attention module and a temporal attention module; Step 3: Input historical traffic data and historical spatiotemporal embedding into the encoder, extract spatial and temporal features using the graph diffusion attention module and the temporal attention module respectively, and fuse the two features using the gate fusion module to generate historical spatiotemporal features; Step 4: Feed the historical spatiotemporal features, historical spatiotemporal embedding, and future spatiotemporal embedding into the transformation attention module to generate the corresponding output; Step 5: Feed the future spatiotemporal embedding and the output of Step 4 into the decoder, extract spatial and temporal features using the graph diffusion attention module and the temporal attention module respectively, and fuse the two features using the gate fusion module to generate future spatiotemporal features. Finally, use the future spatiotemporal features to generate the final prediction sequence. Step 6: Calculate the mean absolute error, root mean square error, and mean absolute percentage error between the final predicted sequence obtained in Step 5 and the predicted sequence in the training dataset. Then, backpropagate through the Adam optimizer to update the network parameters and finally obtain the trained random graph diffusion attention model. Step 7: Using the model with updated network parameters from Step 6 and the test dataset obtained in Step 1, select 16 sets of test data as input and execute Steps 2 to 6, where the training data in Step 2 is replaced with the selected 16 sets of test data, and finally the prediction sequence generated based on the test data is obtained. Step 8: Using the prediction sequence generated based on the test data obtained in Step 7, calculate the mean absolute error between the prediction sequence and the prediction sequence in the test dataset. After obtaining the mean absolute error of all groups of data, calculate the average to obtain the final mean absolute error representing the model performance. Step 9: Repeat steps 2 to 8 until the mean absolute error obtained in step 8 no longer decreases, indicating that the model has reached its optimal performance. Then the network parameters have been updated and the model training has ended. The overall formula for the spatiotemporal embedding generator is as follows: in For time embedding, , These represent the time information within a day and the date information within a week, respectively. For spatial embedding, The adjacency matrix of the predefined road network is Node2vec(), which is the node2vec algorithm used to encode the road network information; E is the spatiotemporal embedding, which is divided into historical spatiotemporal embedding. and future spacetime embedding .

2. The traffic sequence prediction method based on multivariate time series data analysis according to claim 1, characterized in that, The formula for the time attention module is as follows: Let be the input sequence of the l-th layer encoder, B represent the result of concatenating the input sequence with the historical spatiotemporal embedding; W represent the learnable parameters of the model, and Q, K and V represent the problem matrix, key matrix and numerical matrix obtained after the block is linearly transformed, respectively.

3. The traffic sequence prediction method based on multivariate time series data analysis according to claim 2, characterized in that, The graph diffusion attention module includes random graph attention and adaptive graph convolution. The formula for random graph attention is as follows: in It is a randomly generated matrix after a predefined graph result masking operation. It is the output of single-layer random graph attention; The formula for adaptive graph convolution is as follows: It consists of two independent node embeddings. It is based on , The generated adaptive matrix, It is the output of the convolution; The formula for the graph diffusion attention module is expressed as follows: This is the output of the graph diffusion attention module.

4. The traffic sequence prediction method based on multivariate time series data analysis according to claim 3, characterized in that, The formula for the attention conversion module mentioned in step 4 is as follows: This is the output of the Encoder. It is the output of the attention module.

5. The traffic sequence prediction method based on multivariate time series data analysis according to claim 4, characterized in that, The formulas for the Mean Absolute Error (MAE), Root Mean Square Error (RMSE), and Mean Absolute Percentage Error (MAPE) mentioned in step 6 are as follows: in, For predicted values, The actual value; Indicates the length of the sequence.

Images