Multi-node traffic flow prediction method based on simplified attention mechanism
By introducing a simplified mechanism and new attention module in traffic flow forecasts, the existing model has solved the lack of existing models in capturing space dynamic dependence and delaying communication, and achieved more efficient traffic flow forecasting and model training.
Patent Information
- Application Number
- CN202411482079.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-23
- Publication Date
- 2025-01-24
AI Technical Summary
The existing graph neural network model and attention mechanism-based model are difficult to effectively capture the spatial dynamic dependence and delayed propagation between nodes in traffic flow prediction, and the calculation complexity is high.
This paper proposes a multi-node traffic flow prediction method based on a simplified attention mechanism, and designs a new attention module that can simultaneously obtain short-range and long-range spatial dependencies and explicitly model the time delay of spatial information propagation. This method simplifies attention calculation and improves model training speed by introducing TSA, SSA and GSA modules.
It improves the accuracy of traffic flow prediction, simplifies attention calculation, significantly speeds up model training, and can more effectively extract geographical and semantic similarity dependencies between nodes.
Smart Images
Figure CN119360617A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of traffic flow prediction, and specifically is a multi-node traffic flow prediction method based on a simplified attention mechanism, which is suitable for predicting multi-node traffic flow in a city. Background Art
[0002] In recent years, the rapid pace of urbanization has brought severe challenges to contemporary urban traffic management. As a crucial component of modern smart cities, intelligent transportation systems have been developed to analyze, control and optimize traffic conditions (for example, to alleviate traffic congestion). Traffic flow prediction, as one of the key technologies of intelligent transportation systems, has been widely explored, with the goal of predicting the future flow of the transportation system based on historical data. Related studies have shown that accurate traffic flow prediction can be applied to various traffic-related fields, covering route planning, vehicle scheduling, and congestion relief. In the field of traffic prediction, some mainstream methods are through graph neural network models and models based on attention mechanisms. The main disadvantages of graph neural networks are that they are difficult to capture the dynamic spatial dependencies between nodes and that they ignore the possibility of delayed propagation. The disadvantage of models based on attention mechanisms is that they have high computational complexity. Summary of the invention
[0003] In response to the problems in traffic prediction using graph neural network models and models based on attention mechanisms, the present invention proposes a multi-node traffic flow prediction method based on a simplified attention mechanism. The present invention designs a new attention module to simultaneously obtain short-range and long-range spatial dependencies, and explicitly models the time delay of spatial information propagation to improve the overall prediction accuracy of the model. Different from the traditional attention mechanism, the simplified attention mechanism introduced in the present invention speeds up the calculation of attention and the overall training speed of the model.
[0004] Technical solution: A multi-node traffic flow prediction method based on a simplified attention mechanism, characterized in that a multi-node traffic flow prediction model is used to predict the traffic flow data of nodes in the future T period with the node traffic flow data of the previous T period; the multi-node traffic flow prediction model includes a data embedding layer, a simplified attention layer and an output layer; the simplified attention layer includes a heterogeneous attention separation layer, a simplified attention module and a heterogeneous attention aggregation layer; the simplified attention module includes a TSA module, an SSA module and a GSA module; after the traffic node flow data set is input into the embedding layer, the input of the heterogeneous attention separation layer is obtained; the heterogeneous attention separation layer transforms the input through linear projection to obtain the input of the three modules in the simplified attention module; the heterogeneous attention aggregation layer splices the outputs of the three modules through concat; then, the output layer transforms the data dimension output by the heterogeneous attention aggregation layer into the data dimension when the data embedding layer is input through linear transformation.
[0005] Furthermore, the mathematical formula of the multi-node traffic flow prediction model is as follows:
[0006]
[0007] X t =(x 1，t , x 2，t , x 3，t , x 4，t ,...x n，t )
[0008] Among them, X t represents the traffic flow of all nodes at time t, x n，t represents the specific traffic flow of node n at time t, G represents the adjacency matrix of the node's geographical location; f represents the mapping relationship between existing data and predicted data.
[0009] Furthermore, the data embedding layer includes a position encoding embedding X tpe , each weekly cycle is embedded in X w , each day's cycle is embedded in X d , Laplace embedding of the graph structure of the node position X spe , then the traffic node flow dataset X data After inputting into the embedding layer, the input vector X of the simplified attention mechanism layer is obtained. emb ;
[0010] X emb The calculation formula is as follows:
[0011] X emb =X data +X spe +X w +Xd +X tpe .
[0012] Furthermore, X spe , the degree matrix D and the adjacency matrix A are obtained from the graph structure, and the normalized Laplace matrix Δ is calculated. The eigenvalue decomposition is performed to obtain the eigenvalue matrix Λ and the eigenvector matrix U. Finally, the linear projection on the k smallest non-trivial eigenvectors is used to obtain X spe The specific operation method is:
[0013]
[0014]
[0015] X tpe Transformer position embedding is used, specifically:
[0016]
[0017]
[0018] Among them, pos refers to the position, i refers to the dimension, and d model Refers to the total dimension of the input data mapped to the vector space; PE (pos，2i) Indicates the position encoding of the even dimension of the pos position; PE (pos，2i+1) Indicates the position encoding of the odd dimension of the pos position;
[0019] X d , with minute precision, establish a mapping space, and obtain the periodic cycle embedding vector X of each day by inputting the time of each day d ;
[0020] X w , with day as precision, establish a mapping space, and obtain the cycle embedding vector X of each week by inputting the week information of the current moment w ;
[0021] Furthermore, the heterogeneous attention separation layer transforms the input through linear projection as follows:
[0022] Simplify the input X of the attention mechanism layer emb The dimension d model Transformed into d model / 3, get X input , as the input of the three modules in the simplified attention module.
[0023] Furthermore, in the simplified attention module, the simplified attention mechanism calculation formula is as follows:
[0024]
[0025]
[0026]
[0027]
[0028] Among them, X t:: Refers to the simplified input X of the three modules in the attention module input ; A query vector representing the currently processed input element, used to find related information in other elements; is a key vector for each element in the input sequence, used to compare Make a match; It is the value vector of each element in the input sequence, containing the actual information content; SimpleAtten represents the result of simplifying the attention mechanism calculation.
[0029] Furthermore, the calculation formula of the TSA module is as follows:
[0030]
[0031] Among them, TSA represents the output of the TSA module.
[0032] Furthermore, the calculation formula of the SSA module is as follows:
[0033]
[0034] Among them, SemSSA represents the output of the SSA module, M sem is the binary semantic masking matrix;
[0035] M sem The construction method is as follows: using dynamic time warping to calculate the similarity of historical traffic flows between nodes, selecting the K nodes with the highest similarity for each node as its semantic neighbors, and then constructing the binary semantic masking matrix M by setting the weight between the current node and its semantic neighbors to 1. sem ;
[0036] The similarity calculation formula is as follows:
[0037]
[0038] Among them, the traffic flow sequence of node X is X: (X1, X2, ...X t ), the traffic flow sequence of node Y is Y: (Y1, Y2, ...Y t) ; construct a t×t matrix D, where D[t, t] represents the Euclidean distance between X and Y; R represents the path from (1, 1) to (t, t), W * Represents the Euclidean distance, the smallest W * is the similarity of the traffic sequences of two nodes; X x represents the traffic flow sequence of node X, Y y represents the traffic flow sequence of node Y; R x，y It represents the point (x, y) in the matrix D. When it is equal to 1, it means that the path passes through the point. Otherwise, when it is equal to 0, it means that the path does not pass through the point.
[0039] Furthermore, the calculation formula of the GSA module is as follows:
[0040]
[0041] Among them, GeoSSA represents the output of the GSA module; M geo Denotes the geographic masking matrix M geo ;
[0042] In the GSA module, with one node as the center, the remaining nodes are divided into far area and near area according to the distance. Then, the full amount of simplified attention is calculated for the nodes in the near area, while the simplified attention of the representative nodes in the far area is calculated for the nodes in the far area.
[0043] Geographic masking matrix M geo The construction method is as follows: the nodes in the far area and the representative nodes in the near area are in the geographic masking matrix M geo The median value is set to 1 and the remaining nodes are set to 0.
[0044] Compared with the prior art, the present invention has the following beneficial effects: the method of the present invention can realize the simultaneous prediction of traffic flow of multiple nodes. The simplified attention mechanism proposed in the present invention can speed up the calculation speed of attention, thereby effectively improving the training speed of the model; the SSA module and GSA module proposed in the present invention can better extract the geographical similarity dependency between nodes, as well as the similarity dependency in the change mode, thereby effectively improving the accuracy of model prediction. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] Figure 1 It is a flowchart of the multi-node traffic flow prediction method based on the simplified attention mechanism in the present invention;
[0046] Figure 2 This is the calculation process of the simplified attention of TSA, SSA, and GSA in the simplified attention module of the present invention. DETAILED DESCRIPTION
[0047] The technical solution of the present invention is described in detail below with reference to the accompanying drawings, but the protection scope of the present invention is not limited to the embodiments.
[0048] like Figure 1 As shown, a multi-node traffic flow prediction method based on a simplified attention mechanism is provided. The present invention adopts a multi-node traffic flow prediction model to predict the traffic flow data of nodes in the future T period based on the node traffic flow data of the previous T period. The specific steps are as follows:
[0049] Step 1: Build a multi-node traffic flow prediction model:
[0050] The multi-node traffic flow prediction model predicts the node traffic flow data of the next T period based on the node traffic flow data of the previous T period. The formula of the multi-node traffic flow prediction model is as follows:
[0051]
[0052] X t =(x 1，t , x 2，t , x 3，t , x 4，t ,...x n，t )
[0053] Among them, X t represents the traffic flow of all nodes at time t, x n，t represents the specific traffic flow of node n at time t, G represents the adjacency matrix of the node’s geographical location; f represents the mapping relationship between existing data and predicted data, and f can also refer to the model proposed in this paper.
[0054] The multi-node traffic flow prediction model includes a data embedding layer, a simplified attention layer and an output layer; the simplified attention layer includes a heterogeneous attention separation layer, a simplified attention module and a heterogeneous attention aggregation layer.
[0055] Step 2: Build the data embedding layer;
[0056] The data embedding layer includes the classic position encoding embedding X tpe , each week's cycle is embedded in X w , each day's cycle is embedded in X d , the Laplacian vector embedding X of the graph structure of the node position spe Then the traffic node flow data set X data After inputting into the embedding layer, we get the simplified input X of the attention mechanism layer. emb , X emb The calculation formula is as follows:
[0057] Xemb =X data +X spe +X w +X d +X tpe
[0058] 1) Laplacian eigenvector embedding X of graph structure spe , the degree matrix D and the adjacency matrix A are obtained from the graph structure, and the normalized Laplace matrix Δ is calculated. The eigenvalue decomposition is performed to obtain the eigenvalue matrix Λ and the eigenvector matrix U. Finally, the Laplace embedding vector X of the graph is obtained by linear projection on the k smallest non-trivial eigenvectors. spe The specific operation method is:
[0059]
[0060]
[0061] 2) Positional encoding embedding X tpe The classic Transformer position embedding is used; the specific method of position encoding embedding operation is:
[0062]
[0063]
[0064] Among them, pos refers to the position, i refers to the dimension, and d model Refers to the total dimension of the input data mapped to the vector space; PE (pos，2i) Indicates the position encoding of the even dimension of the pos position; PE (pos，2i+1) Represents the position encoding of the odd dimension at position pos.
[0065] 3) Every day's cycle is embedded in X d :
[0066] With minute precision, i.e. 1440 minutes a day, a mapping space is established to obtain the periodic cyclic embedding vector X of each day by inputting the time of each day. d .
[0067] 4) In step 2, each week's cycle is embedded in X w :
[0068] With the precision of day, that is, 7 days a week, a mapping space is established. By inputting the week information of the current moment, the cycle embedding vector X of each week can be obtained. w .
[0069] Step 3: Construct a heterogeneous attention separation layer;
[0070] X emb The vector is transformed by linear projection, changing its original dimension d model Transformed into d model / 3, get X input , which serves as the input for the subsequent three attention modules.
[0071] Step 4: Build a simplified attention module based on the simplified attention mechanism:
[0072] The simplified attention module includes a TSA module, an SSA module, and a GSA module. Figure 2 As shown, the simplified attention mechanism calculation (SimpleAtten) is as follows:
[0073]
[0074]
[0075]
[0076]
[0077] X t:: Refers to X input , here A query vector representing the currently processed input element, used to find related information in other elements; is a key vector for each element in the input sequence, used to compare To match, is a value vector for each element in the input sequence, containing the actual information content.
[0078] 1) The TSA (Temporal Self-Attention) module:
[0079]
[0080] X input The query vector Q, key vector K, and value vector V of the attention mechanism are directly used to simplify the calculation of the attention mechanism. At the same time, the MLP is calculated as the residual connection. Finally, the calculation result of the simplified attention mechanism is added to the output of the MLP, that is, the values of each corresponding dimension of the two vectors are added. The MLP here refers to the position feedforward network, which can perform nonlinear transformations at each position, so that the model has a stronger ability to express and fit complex patterns.
[0081] 2) The SSA (Semantic Self-Attention) module:
[0082]
[0083] Dynamic time warping is used to calculate the similarity of historical traffic flows between nodes, and the K nodes with the highest similarity for each node are selected as its semantic neighbors. Then, the binary semantic masking matrix M is constructed by setting the weight between the current node and its semantic neighbors to 1 and vice versa. sem The calculation of the remaining simplified attention is consistent with the TSA (Temporal Self-Attention) module.
[0084] The formula for calculating node traffic similarity in dynamic time is as follows:
[0085]
[0086] Here we take the calculation of nodes X and Y as an example. The traffic flow sequence X of node X is: (X1, X2, ...X t ), the traffic flow sequence Y of node Y: (Y1, Y2, ...Y t ). First, we construct a t×t matrix D, where D[t, t] represents the Euclidean distance between X and Y. Dynamic time warping is to find a path R from (1, 1) to (t, t) such that the overall cumulative Euclidean distance of this path is W * Minimum, the minimum cumulative Euclidean distance here is the similarity of the traffic sequences of the two nodes. x represents the traffic flow sequence of node X, Y y represents the traffic flow sequence of node Y; R x，y It represents the point (x, y) in the matrix D. When it is equal to 1, it means that the path passes through the point. Otherwise, when it is equal to 0, it means that the path does not pass through the point.
[0087] 3) The GSA (geographic self-attention) module:
[0088]
[0089] First, taking a node as the center, the remaining nodes are divided into far area and near area according to the distance; then, the full simplified attention is calculated for the nodes in the near area, and the simplified attention of the representative nodes in the far area is calculated for the nodes.
[0090] The division of the near and far areas is based on 0.618 of the distance from the farthest node to the central node. The area within the range is the near area, and the area beyond the range is the far area. The far area is then further divided into a circular area with the farthest distance as the radius and the near area removed, which is divided into 8 equal parts of the same arc.
[0091] The representative node of the far area is the geometric center of all nodes in the corresponding area. It is necessary to ensure that the sum of the distances from the center to all nodes in the area is minimized. The calculation formula for the geometric center is as follows:
[0092]
[0093] Where (x i ,y i ) are the coordinates of all nodes in the region, (x c ,y c )The coordinates of the center node.
[0094] Based on this, the geographic masking matrix M is obtained geo , all nodes in the near area and the representative nodes in the far area need to calculate the simplified attention mechanism, so the values of these nodes in the geographic masking matrix are set to 1, and the remaining nodes are set to 0. The calculation of the remaining simplified attention is consistent with the TSA (temporal self-attention) module.
[0095] Step 4: Construct a heterogeneous attention aggregation layer:
[0096] The outputs of the three attention modules are concatenated through concat, and finally the dimension of the vector is transformed into d through a linear transformation. model .
[0097] Step 5: Construct the output layer:
[0098] The output of the heterogeneous attention aggregation layer is transformed into the dimension of the input embedding layer through linear transformation, and finally the prediction output of the model is obtained, that is, the multi-node traffic flow prediction data.
[0099] Step 6: Compare the prediction effects of the multi-node traffic flow prediction model proposed in the present invention and the conventional traffic flow prediction model.
[0100] We obtain two grid-based urban traffic datasets (NYCTaxi and CHIBike), which are derived from LibCity, a traffic prediction community established and maintained by Beihang University. We construct traffic node flow datasets in time period T for these two datasets, and divide the training set and test set into 8:2 for the overall time period, and train and test the models respectively; the results are shown in Table 1:
[0101] Table 1
[0102]
[0103]
[0104] Table 1 is a comparison of the prediction effects of the model proposed by the present invention and the conventional traffic flow prediction model on these two data sets. The test data sets and the division of the training set and the test set are consistent. The model in the first part of Table 1 is mainly based on the graph neural network (GNN), and the model in the second part is mainly based on the attention mechanism. The model is trained by the training set, and then tested by the test set. In the two indicators of MAE and RMSE in Table 1, the model proposed by the present invention performs best.
[0105] As described above, although the present invention has been shown and described with reference to specific preferred embodiments, it should not be construed as limiting the present invention itself. Various changes in form and details may be made without departing from the spirit and scope of the present invention.
Claims
1. A multi-node traffic flow prediction method based on a simplified attention mechanism, characterized in that: A multi-node traffic flow prediction model is used to predict the traffic flow data of nodes in the next T period based on the node traffic flow data of the previous T period; The multi-node traffic flow prediction model includes a data embedding layer, a simplified attention layer and an output layer; the simplified attention layer includes a heterogeneous attention separation layer, a simplified attention module and a heterogeneous attention aggregation layer; the simplified attention module includes a TSA module, an SSA module and a GSA module; After the traffic node flow data set is input into the embedding layer, the input of the heterogeneous attention separation layer is obtained; the heterogeneous attention separation layer transforms the input through linear projection to obtain the input of the three modules in the simplified attention module; The heterogeneous attention aggregation layer concatenates the outputs of the three modules through concat; then, the output layer transforms the data dimension of the output of the heterogeneous attention aggregation layer into the data dimension of the input of the data embedding layer through linear transformation.
2. The prediction method according to claim 1, characterized in that: The mathematical formula of the multi-node traffic flow prediction model is as follows: X t =(x 1,t ,x 2,t ,x 3,t ,x 4,t ,...x n,t ) Among them, X t represents the traffic flow of all nodes at time t, x n,t represents the specific traffic flow of node n at time t, G represents the adjacency matrix of the node's geographical location; f represents the mapping relationship between existing data and predicted data.
3. The prediction method according to claim 1, characterized in that: The data embedding layer includes a position encoding embedding X tpe , each weekly cycle is embedded in X w , each day's cycle is embedded in X d , Laplace embedding of the graph structure of the node position X spe , then the traffic node flow dataset X data After inputting into the embedding layer, the input vector X of the simplified attention mechanism layer is obtained. emb ; X emb The calculation formula is as follows: X emb =X data +X spe +X w +X d +X tpe 。 4. The prediction method according to claim 2, characterized in that: X spe , the degree matrix D and the adjacency matrix A are obtained from the graph structure, and the normalized Laplace matrix Δ is calculated. The eigenvalue decomposition is performed to obtain the eigenvalue matrix Λ and the eigenvector matrix U. Finally, the linear projection on the k smallest non-trivial eigenvectors is used to obtain X spe The specific operation method is: X tpe Transformer position embedding is used, specifically: Among them, pos refers to the position, i refers to the dimension, and d model Refers to the total dimension of the input data mapped to the vector space; PE (pos,2i) Indicates the position encoding of the even dimension of the pos position; PE (pos,2i+1) Indicates the position encoding of the odd dimension of the pos position; X d , with minute precision, establish a mapping space, and obtain the periodic cycle embedding vector X of each day by inputting the time of each day d ; X w , with day as precision, establish a mapping space, and obtain the cycle embedding vector X of each week by inputting the week information of the current moment w .
5. The prediction method according to claim 1, characterized in that: The method by which the heterogeneous attention separation layer transforms the input through linear projection is: Simplify the input X of the attention mechanism layer emb The dimension d model Transformed into d model / 3, get X input , as the input of the three modules in the simplified attention module.
6. The prediction method according to claim 1, characterized in that: In the simplified attention module, the simplified attention mechanism calculation formula is as follows: Among them, X t:: Refers to the simplified input X of the three modules in the attention module input ; A query vector representing the currently processed input element, used to find related information in other elements; is a key vector for each element in the input sequence, used to compare Make a match; It is the value vector of each element in the input sequence, containing the actual information content; SimpleAtten represents the result of simplifying the attention mechanism calculation.
7. The prediction method according to claim 6, characterized in that: The calculation formula of the TSA module is as follows: Among them, TSA represents the output of the TSA module.
8. The prediction method according to claim 6, characterized in that: The calculation formula of the SSA module is as follows: Among them, SemSSA represents the output of the SSA module, M sem is the binary semantic masking matrix; M sem The construction method is as follows: using dynamic time warping to calculate the similarity of historical traffic flows between nodes, selecting the K nodes with the highest similarity for each node as its semantic neighbors, and then constructing the binary semantic masking matrix M by setting the weight between the current node and its semantic neighbors to 1. sem ; The similarity calculation formula is as follows: Among them, the traffic flow sequence of node X is X:(X1,X2,...X t ), the traffic flow sequence of node Y is Y:(Y1,Y2,...Y t ) ; construct a t×t matrix D, where D[t,t] represents the Euclidean distance between X and Y; R represents the path from (1,1) to (t,t), W * Represents the Euclidean distance, the smallest W * is the similarity of the traffic sequences of two nodes; X x represents the traffic flow sequence of node X, Y y represents the traffic flow sequence of node Y; R x,y It represents the point (x, y) in the matrix D. When it is equal to 1, it means that the path passes through this point. Otherwise, when it is equal to 0, it means that the path does not pass through this point.
9. The prediction method according to claim 7, characterized in that: The calculation formula of the GSA module is as follows: Among them, GeoSSA represents the output of the GSA module; M geo Denotes the geographic masking matrix M geo ; In the GSA module, with one node as the center, the remaining nodes are divided into far area and near area according to the distance. Then, the full amount of simplified attention is calculated for the nodes in the near area, while the simplified attention of the representative nodes in the far area is calculated for the nodes in the far area. Geographic masking matrix M geo The construction method is as follows: the nodes in the far area and the representative nodes in the near area are in the geographic masking matrix M geo The median value is set to 1 and the remaining nodes are set to 0.