Traffic flow prediction system based on multi-head self-attention mechanism and space-time fusion graph

Abstract

The invention discloses a traffic flow prediction system based on a multi-head self-attention mechanism and a space-time fusion graph. The system comprises a display layer, a business layer, a data layer and an operation environment. The display layer is used for realizing user interaction and data visualization, the business layer is composed of a Spring Boot-based data management module and a Flask-based deep learning module, the data layer is composed of a MySQL database and a Redis cache, and a running environment adopts Linux and is subjected to containerization deployment through Docker. The system introduces a feature transmission module to realize feature fusion between an aggregation graph and an original graph, constructs a space-time fusion graph to model time delay and spatial dependence of a traffic state, and extracts space-time features of traffic data in combination with a multi-head time self-attention mechanism and gated expansion convolution to realize minute-level high-precision traffic flow prediction. The system is suitable for scenes of urban traffic management, signal scheduling optimization, intelligent traffic control and the like.

Description

Technical Field

[0001] The present invention relates to traffic flow prediction technology in intelligent transportation systems, and in particular to a traffic flow prediction system based on a multi-head self-attention mechanism and a spatio-temporal fusion graph. Background Art

[0002] In recent years, the accelerating development of cities has made traffic problems more and more serious globally. Especially in large cities with dense populations, problems such as traffic congestion and travel delays have become increasingly prominent. Traditional traffic management methods rely on historical data and experience and are difficult to adapt to the increasingly complex urban traffic environment. ITS is one of the important solutions proposed in recent years and is developed to analyze, manage, and improve traffic conditions (e.g., reduce traffic congestion). ITS relies on technologies such as sensor networks, big data analysis, and artificial intelligence to monitor, analyze, and optimize traffic information in real time to improve road usage efficiency, reduce traffic congestion, and safety accidents. As one of the core functions of ITS, traffic flow prediction analyzes historical observation data to infer the vehicle flow, pedestrian flow, or the movement of other traffic participants in the traffic system during a specific future time period, providing a scientific basis for traffic management and scheduling. High-precision traffic flow prediction can not only optimize signal control, improve road capacity, but also effectively reduce carbon emissions and promote the sustainable development of cities.

[0003] The rapid development of deep learning technology has provided new research ideas for traffic flow prediction. Time series prediction methods based on models such as RNN, LSTM, and GCN have been widely applied in the field of traffic flow prediction and can effectively model the time-dependent relationships in traffic data. For example, the Diffusion Convolutional Recurrent Neural Network model (DCRNN) proposed by Li et al. models and predicts traffic data in urban road networks through diffusion convolution combined with recurrent neural networks, improving the model's spatial feature extraction ability (Reference: Li et al., DCRNN, 2018). Another example is the Greyhaven model proposed by Wu et al., which introduces an adaptive graph learning mechanism combined with causal convolution, further enhancing the model's expressive ability and computational efficiency.

[0004] The above methods generally have two main problems: First, most existing models use a fixed adjacency matrix to model the traffic network structure, making it difficult to dynamically adapt to the real-time changes of traffic flow, thus limiting the prediction accuracy of the model in complex traffic environments; Second, the propagation of traffic states between road nodes is not instantaneous, but there is a certain time delay. However, most GNN-based models do not explicitly model this spatio-temporal delay characteristic, resulting in the prediction results being unable to accurately reflect the dynamic propagation process of traffic flow. In addition, although some studies have attempted to introduce attention mechanisms to enhance spatio-temporal modeling capabilities, such as the ASTGCN model that combines spatio-temporal attention and graph convolutional structures, it still has deficiencies in capturing the dynamic interaction patterns between complex heterogeneous traffic regions and is difficult to handle regions with significant differences in traffic patterns such as the city center and suburbs, resulting in performance fluctuations of the model in fine-grained prediction tasks.

[0005] Therefore, the existing technology still has problems such as weak spatial dependence expression ability, inability to capture spatio-temporal propagation delay, and lack of application system support. There is an urgent need to propose a new traffic flow prediction system that can fuse multi-scale spatio-temporal features and adapt to traffic dynamic changes to improve prediction accuracy and system practicability. Summary of the Invention

[0006] In order to overcome the problems in the prior art such as poor adaptability of traffic flow prediction to traffic network topology changes, failure to explicitly model spatio-temporal propagation delay, and lack of available prediction systems, the present invention provides a traffic flow prediction system based on a multi-head self-attention mechanism and a spatio-temporal fusion graph. This system combines a graph neural network and an attention mechanism to construct a dynamic spatio-temporal fusion graph, model the propagation delay of traffic flow between road nodes, and at the same time provide a complete front-end and back-end application architecture to improve prediction accuracy and system practicability.

[0007] The technical problems to be solved by the present invention are:

[0008] (1) Construct a graph structure modeling method that can dynamically capture the spatio-temporal delay relationship of traffic flow propagation;

[0009] (2) Design a spatio-temporal convolution module combined with a multi-head time self-attention mechanism to enhance the modeling ability of heterogeneous dependence relationships between road nodes;

[0010] (3) Construct a visual, modular, and deployable traffic flow prediction system to enable the deep learning prediction model to have practical application capabilities.

[0011] The technical solutions adopted by the present invention to solve the above technical problems include: a traffic flow prediction algorithm based on a multi-head self-attention mechanism and a spatio-temporal fusion graph, and a traffic flow prediction system based on a multi-head self-attention mechanism and a spatio-temporal fusion graph.

[0012] 1. Algorithm Framework Structure

[0013] The processing flow of this algorithm is as follows: The input spatio-temporal data set is grouped by spectral clustering based on spatial characteristics to form clusters, obtaining an aggregated graph based on the original graph. The feature transmission block processes the features extracted from the aggregated graph and splices them onto the original graph features, promoting the interaction of the spatial-temporal relationship between the aggregated graph and the original graph. The spatio-temporal fusion graph convolution aims to capture dynamic spatial-temporal correlations and extract spatio-temporal features of the input time-series graph data by using a novel multi-head temporal self-attention module. After each spatio-temporal fusion graph convolution layer, skip connections are used to transform the output into the skip dimension. By convolving the sum of the outputs of each skip connection layer, the final hidden state is obtained. To make predictions, a ReLu layer with a linear transformation is used to convert the final hidden state into the dimension of the predicted result.

[0014] 1.1 Feature Transmission Module

[0015] To achieve the interaction between the spatio-temporal relationships of the aggregated graph and the original graph, this patent constructs a dynamic information transmission block that combines regional node features and link node features. This interaction enhances feature integration by combining regional information with local details, dynamically adapts to changing traffic patterns, and improves prediction accuracy by leveraging local and global contexts. The transformation matrix is defined as follows:

[0016]

[0017] For road segment i belonging to cluster j, the features of cluster j are fused with the features of road segment i. Due to the spatio-temporal dynamic characteristics of traffic data, the relationship between the road segment and cluster j is also dynamic. Referring to work based on a mechanism similar to attention, a spatial dynamic relationship transfer matrix is further proposed, and its calculation process is shown in formulas (2) to (4):

[0018] F = tanh((X T U1)U2(X C U3) T +b) (2)

[0019] F′ = F - mean(F) (3)

[0020]

[0021] where U1, U2, U3, and b are trainable parameters. X is the original graph feature, and X C is the aggregated graph feature. Using this matrix, the output of the transmission block is calculated as shown in formulas (5)(6):

[0022]

[0023] 1.2 Spatiotemporal Fusion Graph Convolution Module

[0024] The purpose of generating the spatiotemporal fusion graph is to obtain more accurate spatiotemporal dependencies than general spatial graphs and eliminate the time delay error in information propagation between nodes. To explore the correlation of road node pairs, previous traffic flow prediction work proposed the fast DTW algorithm, which improves efficiency by restricting the search length of DTW. We propose a similarity algorithm for the time delay of information propagation between adjacent nodes based on DTW, and calculate A tg (time similarity matrix) at different time step intervals. A stfg (spatiotemporal fusion graph) consists of A tg , A sg (road network adjacency matrix), and A tc (autocorrelation matrix) (as shown in Figure 3-1 (c)). Each node of this fusion graph carries its relevant information, including: (1) the correlation information of spatial neighbors, (2) the time correlation information for eliminating time delay errors, and (3) the autocorrelation information along the time axis.

[0025] In the spatiotemporal fusion graph convolution module, the topological spatial correlation of data is obtained by performing matrix multiplication on the feature input. Then the result is input into the spatiotemporal graph convolution layer to eliminate the influence of time delay errors, as shown in the following equations (7) and (8):

[0026]

[0027] H 2 =ReLu(A stfg H 1 W2 + b2) (8)

[0028] where W1, W2, b1, and b2 are trainable parameters, is the Laplacian matrix representation corresponding to the graph, defined as where D and A are the degree matrix and adjacency matrix of the subgraph, respectively, and Dii = 1.

[0029] This module uses gated convolution to parallelly extract dynamic local time features, which has high efficiency for long time series data. We use dilated convolution to expand the receptive field of the gated convolution, with a dilation coefficient dil = 2. The calculation process is shown in equations (9) and (10):

[0030] [β1, β2]=split(Conv dil＝2 (H 2 )) (9)

[0031] H 3 =tanh(β1)⊙sigmoid(β2) (10)

[0032] Among them, Conv dil＝2 represents a dilated convolution operator, and split represents an equal division operator. kc is the size of the convolution kernel, and T1 is the time step length.

[0033] To further extract the global time relationship, this module uses the attention mechanism to capture the time correlation of traffic data, as shown in the following formulas (11) to (13):

[0034]

[0035] H 4 = H 4 + H 3 + b3 (12)

[0036]

[0037] 1.3 Prediction result output module

[0038] After each spatio-temporal encoder layer, the model uses skip connections composed of 1×1 convolutions to convert the output into the skip dimension. By summing the outputs of each skip connection layer and then performing convolution, the final hidden state is obtained. To make predictions, the final hidden state is converted into the required dimension output using a linear transformation of ReLu:

[0039]

[0040] 2. System framework structure

[0041] 2.1 System hierarchy

[0042] (1) Presentation layer

[0043] The presentation layer is the interface for users to interact with the system. The presentation layer of this system can display information and interact with users. The presentation layer is developed using the Vue3 framework, leveraging its reactive features and efficient data binding capabilities to improve the efficiency and performance of front-end development. To enhance the user experience and the interaction ability of the interface, the presentation layer introduces element-ui.js as a UI component library to provide a beautiful and powerful user interface (such as forms, tables, buttons, etc.).

[0044] The presentation layer uses the Vis.js library to support the visual display of traffic flow data. Vis.js is a powerful visualization graph library that can render complex traffic network data in the browser and display it in the form of interactive charts.

[0045] (2) Business layer

[0046] The business layer is the core of the system, responsible for processing requests from the presentation layer and performing data processing. The business layer can be divided into two parts:

[0047] Basic business system: This part is developed using the Spring Boot framework and mainly processes the basic functions of the system, including user management, historical traffic dataset management, traffic flow prediction, etc. It integrates the MyBatis persistence layer framework, supports efficient database operations, and simplifies the complexity of database interaction through annotations and XML configurations. This part of the system provides functions such as user registration, login, and permission management to ensure the security of the system and multi-role access control. The system manages the uploaded traffic datasets, including operations such as data storage, query, and deletion, to ensure the integrity and security of the data. The system provides a traffic flow prediction function based on historical data to ensure that effective decision-making support is provided to traffic management personnel in a timely manner.

[0048] Deep learning business system: This part is developed using the Flask framework and specializes in processing the business logic related to traffic flow prediction algorithms. Flask is a lightweight Web framework that is very suitable for integration with machine learning and deep learning models. The core function of the deep learning business system is the training and inference of the traffic flow prediction model. The model based on spatio-temporal delay perception achieves more accurate traffic flow prediction. The system uses the trained deep learning model for traffic flow prediction. The system uses the machine learning framework PyTorch for model training and optimization to optimize the prediction results.

[0049] (3) Data layer

[0050] The data layer is responsible for the storage and management of the core data of the system to ensure the reliability, persistence, and security of the data. The design of the data layer includes the following parts:

[0051] MySQL database: As the relational database of the system, MySQL is responsible for storing important information such as user data, historical traffic datasets, and prediction results. MySQL provides high reliability and performance and has strong advantages in the process of storing and querying massive amounts of data. Through the Structured Query Language (SQL), MySQL supports efficient data retrieval and management.

[0052] Redis cache: Redis, as an in-memory cache database, is used to cache commonly used data and query results, reduce the burden on the database, and improve the system response speed. In the traffic flow prediction system, the Redis cache plays an important role in historical traffic data and commonly used prediction models.

[0053] MyBatis Persistence Framework: As a Java persistence framework, MyBatis simplifies the complexity of database operations. MyBatis maps SQL statements to Java methods to achieve efficient database operations.

[0054] (4) Operating Environment

[0055] Linux: As the main operating system environment for business layer system deployment, it provides a stable operating platform.

[0056] Docker: To improve the system deployment efficiency and portability, the system uses Docker technology for containerized deployment. Docker containers can encapsulate the application and all its dependencies in an independent environment, ensuring that the system can be quickly deployed on any platform that supports Docker.

[0057] 2.2 System Function Modules

[0058] (1) User Management Module

[0059] The user management module is one of the basic function modules of the system. Its main functions include: user registration, login, logout, and personal information editing. This module is used to manage user account information, record user status changes, and determine their permission scope based on the user's status and information, improving the system's security and personalized services.

[0060] (2) Historical Traffic Dataset Management Module

[0061] The historical traffic dataset management module is responsible for managing datasets related to traffic flow prediction, including dataset viewing, uploading, editing, and deletion. This module ensures the efficient and orderly organization and storage of data, supports efficient data retrieval and processing, and is the data foundation for the system to run.

[0062] (3) Traffic Flow Prediction Module

[0063] The traffic flow prediction module is the core function module of the system, covering the entire process of traffic flow prediction, including data uploading, prediction model configuration, and traffic flow prediction execution. This module uses deep learning and spatio-temporal data analysis technologies to provide accurate traffic flow predictions, helping traffic managers make data-driven decisions.

[0064] (4) Traffic Data Visualization and Analysis Module

[0065] The traffic data visualization and analysis module allows users to perform visualization analysis on historical datasets and prediction results, including data chart display, prediction result comparison, etc. This module helps users better understand the data and model performance through an intuitive graphical interface and is a key tool for improving the quality of user decisions.

[0066] (5) Model Training / Retraining Module

[0067] The model training / retraining module is responsible for the training and optimization of the system prediction model. The functional points include the processing of new data, automated model training, model performance evaluation, and automated deployment. By continuously learning the latest traffic data, this module ensures that the prediction model continuously adapts to the changing traffic conditions and maintains prediction accuracy.

[0068] 3. Advantages of the Present Invention

[0069] (1) By constructing a spatio-temporal fusion graph and explicitly introducing a time-delay modeling mechanism, the system can more realistically reflect the traffic flow diffusion characteristics between road nodes and improve prediction accuracy.

[0070] (2) Combining the multi-head self-attention mechanism with the graph neural network enables the model to have the dynamic modeling ability between heterogeneous regions and adapt to complex and changing traffic environments.

[0071] (3) The system integrates functions of model training, prediction, visualization, and user management, realizes the engineering implementation of the traffic prediction model, and has high availability and scalability.

[0072] (4) It can achieve real-time prediction updates at the minute level, support traffic management departments for dynamic signal optimization and emergency response, and promote the practical application of intelligent transportation systems. Brief Description of the Drawings

[0073] The present invention will be further described below in conjunction with the drawings and embodiments:

[0074] Figure 1 It is a schematic diagram of the overall framework of the traffic flow prediction algorithm based on the multi-head self-attention mechanism and spatio-temporal fusion graph. This framework is mainly composed of a transmission block, spatio-temporal fusion convolutional blocks (STF-Blocks), and a prediction result output block;

[0075] Figure 2 It is a hierarchical structure diagram of the traffic flow prediction system of the present invention, which can be divided into four main levels, namely the display layer, business layer, data layer, and operating environment;

[0076] Figure 3 It is a schematic diagram of the functional modules of the traffic flow prediction system of the present invention, which can be divided into five main functional modules, including: user management module, historical traffic data set management module, traffic flow prediction module, traffic data visualization and analysis module, and model training / retraining module;

[0077] Figure 4 It is a flow chart of the time-delay graph generation algorithm in the spatio-temporal fusion graph;

[0078] Figure 5This is the overall flowchart of traffic flow prediction, which shows the entire process from data collection, spatio-temporal feature extraction to prediction output. Detailed implementation

[0079]

Example 1

[0080] As Figures 1 to 5 shown, the present invention provides a traffic flow prediction system based on the multi-head self-attention mechanism and spatio-temporal fusion graph. The overall system consists of four main levels: the display layer, the business layer, the data layer, and the operating environment ( Figure 2 ).

[0081] First, the user uploads historical traffic flow data through the front-end interface ( Figure 5 data collection link in). The uploaded file will be parsed and stored, and the relevant metadata (such as dataset name, path, description) will be saved to the database. All data is transmitted by the system to the server, and the data collection frequency is defaulted to once every 5 minutes. The server operating environment is Ubuntu 20.04, equipped with an Nvidia RTX 3090 with 24G of memory. The development environment uses Python 3.11 and is based on the PyTorch 1.11 framework to implement data preprocessing, storage, and subsequent calculations. The preprocessing steps include data cleaning, missing value filling, normalization processing, and time series segmentation. Each dataset is divided into 60% for training, 20% for validation, and 20% for testing in chronological order.

[0082] The optimal model selected is determined based on evaluation metrics. The model trained using Adam optimization. The original learning rate is 0.0001. In the training stage, 80 epochs of training are carried out, and the size of each batch of data is 32. The search length in the time delay graph generation algorithm is 12, which is the maximum prediction step in the traffic flow prediction task.

[0083] As Figure 1 shown, the data will first enter the feature transfer module, which divides the nodes into different clusters according to the spatial structure based on spectral clustering, and realizes the fusion of multi-granularity spatial features through the feature transfer mechanism between the original graph and the aggregated graph. This module significantly enhances the system's ability to capture spatial dependence relationships.

[0084] Subsequently, the system enters the spatio-temporal fusion graph convolution module stage. This module combines three subgraphs: the adjacency graph formed by the road connection structure, the autocorrelation graph, and the time delay graph. Among them, the generation process of the time delay graph is as Figure 4As shown in the figure, an improved dynamic time warping (DTW) algorithm is adopted to generate a time delay matrix based on the similarity of historical states between road nodes, which is used to simulate the time delay generated by the propagation of traffic states between nodes. The convolutional module also combines a multi-head self-attention mechanism to extract features from different time dimensions in parallel, so as to better simulate the changing rules of urban traffic flow in different time periods. The gating mechanism and dilated convolution are fused during the convolution process to effectively improve the long sequence modeling ability and control the problem of gradient dissipation.

[0085] The finally output traffic flow prediction results will be mapped into a numerical sequence of a specified time step through the output module and visualized by the front-end display module.

[0086]

Example 2

[0087] The system function module structure is as Figure 3 shown, including a user management module, a historical traffic data set management module, a traffic flow prediction module, a traffic data visualization analysis module, and a model training / retraining module.

[0088] The user management module is used for system account registration, login, permission control, etc. to ensure usage security. The data set management module provides functions for data uploading, browsing, editing, and deletion, and supports the unified management of multi-source and multi-format data sets. The prediction module is responsible for task scheduling, model selection, and parameter configuration, and controls the output of prediction results.

[0089] The traffic data visualization analysis module supports graph rendering based on Vis.js and displays various data analysis results including line charts. The model training module is based on the PyTorch framework and supports batch training and dynamic retraining of historical data to improve the generalization ability of the model.

[0090]

Example 3

[0091] In terms of system deployment, the business logic layer is jointly composed of Spring Boot and Flask. Spring Boot is mainly used to handle user interactions and data logic requests, and Flask is used to load and run deep learning models. The system backend stores structured data through a MySQL database and caches common models and prediction intermediate results through Redis to improve prediction efficiency.

[0092] The overall system is deployed in a Linux environment and encapsulated through Docker containerization, which facilitates the rapid deployment and migration of the system on different platforms and improves the running stability and maintainability.

[0093] The system supports real-time traffic prediction updates at the minute level, can combine data from urban sensor networks or traffic control centers for dynamic data input, achieve high-precision prediction of urban road traffic flow, and serve practical scenarios such as traffic light control, route recommendation, and traffic congestion emergency response.

Claims

1. A traffic flow prediction system based on the multi-head self-attention mechanism and spatio-temporal fusion graph, comprising a display layer, a business layer, a data layer and a running environment, characterized in that: The display layer is used for interacting with users to achieve the visual display of traffic data; The business layer includes a basic function subsystem for managing user permissions and data sets, and a deep learning subsystem for performing traffic flow prediction. The former is built based on Spring Boot, and the latter is implemented based on Flask and PyTorch; The data layer includes a MySQL database and a Redis cache system for storing user information, traffic data and prediction results; The running environment is based on the Linux system and realizes the containerized deployment of the system through Docker; The system fuses the node features of the aggregated graph and the original graph through a feature transmission module, and constructs a spatio-temporal fusion graph. The fusion graph consists of an adjacency matrix, an autocorrelation matrix and a time-delay similarity matrix; The system extracts spatio-temporal features by combining the multi-head temporal self-attention mechanism and gated dilated convolution, and outputs prediction results; The prediction results are graphically displayed through the display layer and support user download.

2. The traffic flow prediction system according to claim 1, wherein The feature transmission module divides the nodes of the original graph based on the spectral clustering method, aggregates adjacent nodes into spatial clusters, and completes the feature transfer between the original graph and the aggregated graph by constructing a spatial dynamic relationship transmission matrix.

3. The traffic flow prediction system according to claim 1 or 2, characterized in that, The time-delay similarity matrix of the spatio-temporal fusion graph is constructed by an improved dynamic time warping algorithm, and the similarity of node pairs between different time steps is calculated by limiting the search window length.

4. The traffic flow prediction system according to any one of claims 1 to 3, characterized in that, The gated convolution uses a dilated structure to expand the receptive field and captures local and global temporal dependencies through parallel combination with the multi-head temporal self-attention mechanism.

5. The traffic flow prediction system according to any one of claims 1 to 4, characterized in that, The display layer of the system is developed using Vue3 and integrates the element-ui component library and the Vis.js graph library to achieve interactive visualization of data upload, model operation and prediction results.

6. The traffic flow prediction system according to any one of claims 1 to 5, characterized in that, The system supports the training and retraining of traffic prediction models. Parameters such as the learning rate, number of training epochs, batch size, etc. during the training process can be configured by users, and support dynamic updates based on historical data to adapt to changes in traffic patterns.

Images