A traffic flow prediction method and system based on selective attention mechanism

Abstract

The present invention discloses a traffic flow prediction method and system based on a selective attention mechanism, which belongs to the field of intelligent transportation and machine learning. The traffic flow prediction method and system based on the selective attention mechanism include: first, obtaining historical traffic data, road topology data and road attribute data; then, based on the KGCN-GSAM model, a traffic prediction model is established according to the road topology data and the road attribute data; then, based on the obtained traffic prediction model, according to the historical traffic data, the traffic prediction value in the next time period is predicted. By using the KGCN-GSAM model traffic prediction, the present invention effectively improves the accuracy of traffic prediction.

Description

Technical Field

[0001] The present invention belongs to the field of intelligent transportation technology, and in particular relates to a traffic flow prediction method and system based on a selective attention mechanism. Background Art

[0002] Traffic flow forecasting is a process that uses data analysis and machine learning techniques to predict future traffic flow in a specific area or road section. This task usually uses historical traffic data, road network data, weather conditions and other factors for analysis and modeling. With the development of the Internet of Things, these data sources can now be collected in various ways, including sensor devices, government data disclosure, third-party data providers and public databases, providing strong data support for the field of traffic flow forecasting.

[0003] Traditional traffic flow prediction methods are mainly based on statistical models and time series analysis. These methods use historical traffic data and some mathematical models for modeling and prediction. Although traditional methods can meet the needs of traffic flow prediction to a certain extent, they often cannot handle complex nonlinear problems. Compared with traditional methods, deep learning can automatically learn feature representations from data and perform data modeling and prediction in an end-to-end manner. This end-to-end learning method eliminates the cumbersome feature engineering process in traditional methods and can better capture the potential patterns in the data. However, most models ignore the integrity of the traffic network and the global correlation of roads. Some models use attention mechanisms to solve this problem, but they face the problem of over-averaging of outputs.

[0004] Based on the above problems, it is urgent to design a new model to comprehensively model the roads in order to improve the accuracy of traffic flow prediction. Summary of the invention

[0005] The purpose of the present invention is to provide a traffic flow prediction method and system based on the selective attention mechanism, which improves the prediction accuracy by improving the attention mechanism and establishing the KGCN-GSAM traffic flow prediction model.

[0006] In order to achieve the purpose of the present invention, on the one hand, the present invention provides a traffic flow prediction method based on a selective attention mechanism, the steps are as follows:

[0007] Step 1: Obtain the road topology network structure and number it through maps and field surveys; define the set of observation points, obtain road knowledge attribute information, and obtain the traffic speed matrix of the observation points at the time point;

[0008] Step 2: Use the KR-EAR method and road knowledge attribute information to construct a road knowledge graph and complete the generation of knowledge vectors; input the knowledge vectors and traffic speed matrix into KF-Cell to obtain the traffic speed matrix embedded with the knowledge vectors;

[0009] Step 3: The traffic speed matrix and road topology network structure embedded with the knowledge vector are used as the input of the stacked two-layer graph convolutional network model to obtain the traffic speed matrix with spatial information;

[0010] Step 4: Use the traffic speed matrix with spatial information as the input of the stacked two-layer gated recurrent unit model to obtain the hidden state of the gated recurrent unit with temporal information;

[0011] Step 5: Use the Gumbel-Max trick to improve the attention mechanism, input the traffic speed matrix with spatial information into the selective attention mechanism, and the selective attention mechanism automatically generates a weight matrix to obtain the hidden matrix with the selective attention mechanism;

[0012] Step 6: Obtain the traffic flow prediction result through the fully connected layer; obtain more accurate prediction results by training the KGCN-GSAM model.

[0013] On the other hand, the present invention provides a traffic flow prediction system based on a selective attention mechanism, the system comprising:

[0014] Road knowledge attribute information acquisition module: used to obtain the road topology network structure and number it, define the observation point set, and obtain the traffic speed matrix of the observation point at the time point;

[0015] Knowledge vector generation module: used to construct the road knowledge graph, complete the generation of knowledge vectors, and obtain the traffic speed matrix embedded in the knowledge vectors;

[0016] Graph convolutional network module: used to calculate the traffic speed matrix and road topology network structure embedded in the knowledge vector to obtain the traffic speed matrix with spatial information;

[0017] Gated recurrent unit hidden module: used to calculate the traffic speed matrix with spatial information and obtain the gated recurrent unit hidden state with temporal information;

[0018] Attention mechanism matrix module: used to calculate the traffic speed matrix with spatial information, generate the weight matrix, and obtain the hidden matrix with selective attention mechanism;

[0019] Prediction result generation module: used to calculate the fully connected layer to obtain traffic flow prediction results, and also used to train the KGCN-GSAM model to obtain more accurate prediction results.

[0020] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the traffic flow prediction method using the selective attention mechanism is implemented.

[0021] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the above-mentioned traffic flow prediction method using a selective attention mechanism.

[0022] Compared with the prior art, the significant improvements of the present invention are: (1) the present invention uses the Gumbel-Max trick to improve the attention mechanism, which makes up for the defect that the output of the attention mechanism is too average; (2) the present invention reduces the prediction error and improves the prediction accuracy by establishing the KGCN-GSAM traffic flow prediction model.

[0023] In order to more clearly illustrate the functional characteristics and structural parameters of the present invention, further description is given below in conjunction with the accompanying drawings and specific implementation methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0025] Figure 1 Flow chart of the traffic prediction method based on KGCN-GSAM of the present invention

[0026] Figure 2 It is a system model diagram of traffic flow prediction based on the selective attention mechanism of the present invention.

[0027] Figure 3 This is the KF-Cell implementation flow chart of the present invention.

[0028] Figure 4 (a), (b), and (c) are simulation experiments of the present invention for determining that the hidden unit of the gated recurrent unit is 64.

[0029] Figure 5 (a), (b), and (c) are simulation experiments of the present invention in which the value of the temperature parameter τ is determined to be 10.

[0030] Figure 6 (a), (b), and (c) are visualizations of the evaluation index results of the simulation experiments of the KGCN-GSAM model of the present invention and the baseline model.

[0031] Figure 7 (a) and (b) are visualizations of the output results of the simulation experiments of KGCN-GSAM and the baseline model of the present invention.

[0032] Figure 8 (a) and (b) are visualizations of the output results of the simulation experiments of the GCN-aided GSAM of the present invention and the baseline model. DETAILED DESCRIPTION

[0033] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0034] The present invention designs a traffic flow prediction method and system model based on the selective attention mechanism. The system model diagram is shown in Figures 1 and 2, which can comprehensively model the roads to reduce traffic flow prediction errors and improve traffic flow prediction accuracy.

[0035] Step 1: We define a set of observation points V, such as sensors on a road segment, represented by V = {v1, v2, ..., v m}, where m is the total number. Each observation point on a road segment is represented as a node. The observed characteristics, such as traffic speed, are considered to be a node attribute. In addition, the time series is the sequence of node attributes collected at m observation points within a time interval, denoted by T = {t1, t2, …, t p Assuming that the observation time length of T is P, the time series can be expressed as the traffic speed matrix Given by:

[0036]

[0037] in, is the traffic speed of the ith observation point at the jth moment. Then, we represent the topological structure of the road network as a graph, denoted as G = (V, E), where V is the node set and E is the edge set. Suppose an adjacency matrix A∈R m×m , R m×m It is represented as an m×m matrix. It is a two-dimensional array representing the connectivity of the corresponding road segment. If two roads are connected, the corresponding element is set to 1, otherwise it is set to 0.

[0038] Step 2: Use the KR-EAR method and the collected road knowledge attribute information to complete the construction of the road knowledge graph and generate the knowledge embedding vector X E , the KF-Cell method is used to embed the knowledge vector into the traffic speed. The input of KF-Cell is the embedded knowledge X E The road section feature observed at a certain observation point at time t is X t, which is also referred to as the traffic speed matrix in this invention KF-Cell divides the embedded knowledge into two parts, dynamic and static, and embeds them separately. The output is the updated road segment features fused with external knowledge at time t, denoted as X′ t Specific implementation is as follows Figure 3 As shown, KF-Cell can be given by the following formula:

[0039] X s =ReLU(e s X t w s +b s );

[0040] X d =ReLU(e d X t w d +b d );

[0041] X′ t =[X s , X d ];

[0042] Among them, e s and e d are the static and dynamic external factors of the embedded information, w s and w d are their respective linear transformations, b s and b d are their respective bias constants, X s and X d are the traffic speed matrices embedded with static and dynamic external factors, respectively. ReLU(x) is the activation function, and [·] is the concatenation of the two matrices.

[0043] Step 3: Use graph convolutional network to model the road space and obtain road space dependency. Stack two layers of graph convolutional network models and use X′ t and road topology network as its input, the process is formulated as follows:

[0044]

[0045] Among them, A is the adjacency matrix of the road network graph G, is the normalized matrix of the matrix (A+I), D is the degree matrix of the matrix (A+I), I is the unit matrix of the same size as A, ReLU(x) is the activation function, σ(x) is the Sigmoid activation function. The weight matrix from the input layer to the hidden unit layer is represented by W0, and the weight matrix from the hidden layer to the output layer is represented by W1. The output of the two-layer graph convolutional network is denoted by X GCN .

[0046] Step 4: Use gated recurrent units to model road time and obtain road time dependency. Stack two layers of gated recurrent unit models and use X GCN As its input, its process formulation is as follows:

[0047] z t =σ(W z ·[H t-1 , X GGN ]+b z );

[0048] r t =σ(W r ·[H t-1 , X GCN ]+b r );

[0049]

[0050]

[0051] where z t is the update gate, r t To reset the gate, is the hidden matrix that “remembers the current state”, W z , W r and W h are the weight matrices of the update gate, reset gate and hidden matrix, respectively, and b z 、b r and b h They are z t 、r t and The bias term, H t-1 is the hidden state of the previous time step. σ(x) is the Sigmoid activation function, and Tanh(x) is the hyperbolic tangent activation function. The hidden state of the gated recurrent unit is denoted by H t .

[0052] Step 5: Generate a set of probabilities through the hidden state of the gated recurrent unit, denoted as π = {π1, π2, ..., π n}, is given by:

[0053] π i =sigmoid(W π H t )

[0054] Among them, W π is the weight matrix, H tis the hidden state of the gated recurrent unit at each time step t; in order to obtain a sharper distribution for the result, the Gumbel-Max method used in this paper can generate a categorical distribution with class probability π in a simple and effective way:

[0055] z=argmax(g i +logπ i )

[0056] Among them, g i ={g1, g2, ..., g n = -log(-log(u i )),u i ~U(0,1) is a sample generated by a standard Gumbel distribution.

[0057] Proof: The probability of Gumbel Max outputting category i is π i , in order to maintain generality, here we prove that the probability of outputting category 1 is π1. Outputting category 1 means that logπ1-log(-log(u1)) is the largest, so:

[0058]

[0059] In order to maintain generality, we simplify the first inequality to obtain:

[0060]

[0061] Since u1~U(0,1) and u2~U(0,1), U(0,1) is evenly distributed, The probability is The probability that the inequality holds is Similarly, the probability that all the above inequalities hold is:

[0062]

[0063] Then, we average u1 and get:

[0064]

[0065] By using the Gumbel-Max method, we can obtain z with Gumbel noise i ; Since the Gumbel-Max method has a non-differentiable arg max part, the above proof can be used to use Softmax(x) i The function is a continuous differentiable approximation of argmax, generating an n-dimensional sampling vector, using y i ={y1, y2, ..., y n}, expressed as:

[0066]

[0067] In this way, the approximate y of z can be generated by the above formula i , where n is the n-dimensional sampling vector, j is the constant index. τ is the temperature parameter, π j is the probability of output category j, g j When the output category is j, the samples generated by the standard Gumbel. When the parameter τ approaches 0, the samples generated by the Gumbel-Softmax distribution approximately follow the one_hot distribution, which is ideally very sharp. As τ increases, the samples from the Gumbel-Softmax distribution become average.

[0068] Define the score function s i , is given by:

[0069]

[0070] in, and W s Yes i The external and internal weight matrices, b s is the bias term. Therefore, the attention value with the selector It can be calculated by the following formula:

[0071]

[0072] Where n is an n-dimensional sampling vector.

[0073] Therefore, the output of the attention mechanism-assisted gated recurrent unit can be given as follows:

[0074]

[0075] in, is the hidden state at time step t-(i-1).

[0076] Step 6: Get the corresponding traffic flow prediction result through a fully connected layer, as shown in the following formula:

[0077]

[0078] Among them, f FCL (x) is a fully connected function. Then, the output of GCN is fed into the SAM module to train the KGCN-GSAM model to reduce the difference between the actual speed and the expected speed on the road network. For the traffic characteristics at time t, we denote the speed as Y t , the expected speed is denoted as Y t p;

[0079] The loss function of the KGCN-GSAM model is:

[0080] Loss=||Y t -Y t p ||+λL reg

[0081] Among them, L reg is a normalization term that can prevent overfitting, and λ is a hyperparameter with a value of 0.0015, ||Y t -Y t p ||Used to reduce the difference between expected speed and actual speed;

[0082] KGCN-GSAM model training makes the prediction results more accurate.

[0083] The present invention also provides a traffic flow prediction system based on a selective attention mechanism, comprising:

[0084] Road knowledge attribute information acquisition module: used to obtain the road topology network structure and number it, define the observation point set, and obtain the traffic speed matrix of the observation point at the time point;

[0085] Knowledge vector generation module: used to construct the road knowledge graph, complete the generation of knowledge vectors, and obtain the traffic speed matrix embedded in the knowledge vectors;

[0086] Graph convolutional network module: used to calculate the traffic speed matrix and road topology network structure embedded in the knowledge vector to obtain the traffic speed matrix with spatial information;

[0087] Gated recurrent unit hidden module: used to calculate the traffic speed matrix with spatial information and obtain the gated recurrent unit hidden state with temporal information;

[0088] Attention mechanism matrix module: used to calculate the traffic speed matrix with spatial information, generate the weight matrix, and obtain the hidden matrix with selective attention mechanism;

[0089] Prediction result generation module: used to calculate the fully connected layer to obtain traffic flow prediction results, and also used to train the KGCN-GSAM model to obtain more accurate prediction results.

[0090] The specific implementation method of each module of the above system is the same as the aforementioned traffic flow prediction method of selecting the attention mechanism, which will not be repeated here.

[0091] Example

[0092] The experimental simulations in this chapter are based on real-world data. Although there are several open traffic flow datasets, it is not easy to collect road network data, traffic data, and other knowledge information, including POI data and weather data for the same period. Due to the difficulty of data collection and knowledge graph construction, the experiments in this chapter are all based on a dataset in Luohu District, Shenzhen.

[0093] The Shenzhen dataset contains taxi track data, road network data, weather data, and POI data for each street from January 1 to January 31, 2015. The study area includes 156 road segments and 9 POI types: food services, enterprises, shopping services, transportation services, education services, life services, medical services, accommodation services, etc. In addition, the weather temperature, weather conditions, wind speed, humidity, air pressure, visibility, etc. at 15-min intervals in the study area in January are used as auxiliary data. We divide weather conditions into five categories: sunny, cloudy, foggy, light rain, and heavy rain.

[0094] To evaluate the prediction performance, we also use the root mean square error (RMSE) and mean absolute error (MAE) as metrics to measure the prediction error, and use accuracy to measure the prediction accuracy. The specific formula is shown below.

[0095]

[0096] In the above formula, and are the real traffic information and predicted traffic information of time samples on road i, respectively. NN is the number of nodes on the road, and M is the number of time samples. Y and P are and Y and P are the real traffic flow dataset and the predicted traffic flow dataset respectively.

[0097] like Figure 4 As shown in (a), (b), and (c), after simulation control experiments, we determined that the hidden unit of the gated recurrent unit is 64, which has a better effect while maintaining training efficiency.

[0098] like Figure 5 As shown in (a), (b), and (c), after simulation control experiments, we determined that the value of the temperature parameter τ is 10, which has a better effect while taking into account the training efficiency.

[0099] like Figure 6 (a), (b), (c) and Figure 7 As shown in (a) and (b), the figure shows the prediction ability of KGCN-GSAM. The KGCN-GSAM model has certain advantages over the current mainstream traffic flow prediction models, and its advantages are more obvious in short-term traffic flow prediction.

[0100] Without loss of generality, we conduct simulation experiments on another dataset, the Los_loop dataset, which is collected in real time on the highways of Los Angeles County by loop detectors. The entire Los_loop dataset has a total of 207 sensors placed at different intersections, recording traffic speed information from March 1, 2012 to May 7, 2012.

[0101] like Figure 8 As shown in (a) and (b), the GCN-aided GSAM (GCN+GRU+SAM) module performs better than T-GCN (GCN+GRU) or A3T-GCN (GCN+GRU+Attention) and is more capable of making accurate traffic predictions, which shows that the GSAM module plays a positive role in traffic prediction.

[0102] It should be noted that, in this article, the terms "comprises", "includes" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or apparatus that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or apparatus.

[0103] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A traffic flow prediction method based on selective attention mechanism, characterized in that: The following steps are involved: Step 1: Obtain the road topology network structure and number it through maps and field surveys; define the set of observation points, obtain road knowledge attribute information, and obtain the traffic speed matrix of the observation points at the time point; Step 2: Use the KR-EAR method and road knowledge attribute information to construct a road knowledge graph and complete the generation of knowledge vectors; input the knowledge vectors and traffic speed matrix into KF-Cell to obtain the traffic speed matrix embedded with the knowledge vectors; Step 3: The traffic speed matrix and road topology network structure embedded with the knowledge vector are used as the input of the stacked two-layer graph convolutional network model to obtain the traffic speed matrix with spatial information; Step 4: Use the traffic speed matrix with spatial information as the input of the stacked two-layer gated recurrent unit model to obtain the hidden state of the gated recurrent unit with temporal information; Step 5: Use the Gumbel-Max trick to improve the attention mechanism, input the traffic speed matrix with spatial information into the selective attention mechanism, and the selective attention mechanism automatically generates a weight matrix to obtain the hidden matrix with the selective attention mechanism; Step 5-1: Pass the hidden state H of the gated recurrent unit with time information t Generate probability π i : p i =sigmoid(W π H t );π={π1,π2,…,π n }; Among them, W π is the probability π i The weight matrix of Step 5-2: Generate a categorical distribution z with class probability π using the Gumbel-Max method: z=argmax(g i +logπ i ) Among them, g i is a sample generated by standard Gumbel; Step 5-3: Prove that the probability of outputting category 1 is π1, from which the probability of category i is π i ; Step 5-4: Use the Gumbel-Max method to obtain y that approximates the classification distribution z of probability π i : Among them, n is the n-dimensional sampling vector, τ is the temperature parameter, π j is the probability of output category j, g j is the sample generated by standard Gumbel when outputting category j; When the parameter τ approaches 0, the samples generated by Gumbel-Softmax are approximately one hot distributed; As τ increases, the generated samples of Gumbel-Softmax become average; Step 5-5: Calculate the score function s i : in, and W s are the external and internal weight matrices, respectively, and b s is the bias term; Step 5-6: Using the score function s i Calculate the attention value of the selector Step 5-7: Select the output of the attention mechanism-assisted gated recurrent unit as the hidden matrix with the selected attention mechanism in, is the hidden state at time step t-(i-1); Step 6: Obtain traffic flow prediction results through the fully connected layer; obtain more accurate prediction results by training the KGCN-GSAM model; The fully connected function of step 6 is f FCL (x): For the traffic characteristics at time t, the expected speed is expressed as Y t p ; Training KGCN-GSAM model: The loss function of KGCN-GSAM is defined as: Loss=||Y t -Y t p ||+λL reg Among them, L reg is the normalization term, λ is a hyperparameter with a value of 0.0015, ||Y t -Y t p ||To reduce the difference between expected speed and actual speed.

2. The traffic flow prediction method based on the selective attention mechanism according to claim 1 is characterized in that: The step 1 comprises: Step 1-1: Define the set of observation points V, V = {v1, v2, ..., v m } means that v i is the observation point, m is the total number of observation points; Step 1-2: The observation time length of the time series T is P, T = {t1, t2, ..., t P }; Traffic speed matrix of time series T for: is the traffic speed of the i-th observation point at the j-th moment; Step 1-3: Represent the road topology network structure as a graph G = (V, E), where E is an edge set; set the adjacency matrix A to be a two-dimensional array representing the connectivity of corresponding road sections. If two roads are connected, set the corresponding element to 1, otherwise set it to 0.

3. The traffic flow prediction method based on the selective attention mechanism according to claim 2 is characterized in that: In step 2, the knowledge vector and the traffic speed matrix are input into KF-Cell to obtain an embedded knowledge vector X E The traffic speed matrix X′ t : X′ t =[X s ,X d ]; Among them, e s and e d are the static and dynamic external factors of the embedded information, w s and w d are static linear transformation and dynamic linear transformation respectively, b s and b d are the static bias constant and the dynamic bias constant, respectively, X s and X d are the traffic speed matrices embedded with static and dynamic external factors respectively.

4. The traffic flow prediction method based on selective attention mechanism according to claim 3 is characterized in that: The traffic speed matrix X with spatial information described in step 3 GCN for: Among them, A is the adjacency matrix of the road network graph G, is the normalized matrix of the matrix (A+I), I is the unit matrix of the same size as A; W0 is the weight matrix from the input layer to the hidden unit layer, and W1 is the weight matrix from the hidden layer to the output layer.

5. The traffic flow prediction method based on selective attention mechanism according to claim 4 is characterized in that: The hidden state H of the gated recurrent unit with temporal information described in step 4 t Calculated as: z t =σ(W z ·[H t-1 ,X GCN ]+b z ); r t =σ(W r ·[H t-1 ,X GCN ]+b r ); Among them, z t is the update gate, r t To reset the gate, is the hidden matrix that remembers the current state, W z , W r and W h are the weight matrices of the update gate, reset gate and hidden matrix, respectively, z 、b r and b h They are z t 、r t and The bias term, H t-1 is the hidden state at the previous time step.

6. A system for traffic flow prediction method based on selective attention mechanism according to any one of claims 1 to 5, characterized in that: include: Road knowledge attribute information acquisition module: used to obtain the road topology network structure and number it, define the observation point set, and obtain the traffic speed matrix of the observation point at the time point; Knowledge vector generation module: used to construct the road knowledge graph, complete the generation of knowledge vectors, and obtain the traffic speed matrix embedded in the knowledge vectors; Graph convolutional network module: used to calculate the traffic speed matrix and road topology network structure embedded in the knowledge vector to obtain the traffic speed matrix with spatial information; Gated recurrent unit hidden module: used to calculate the traffic speed matrix with spatial information and obtain the gated recurrent unit hidden state with temporal information; Attention mechanism matrix module: used to calculate the traffic speed matrix with spatial information, generate the weight matrix, and obtain the hidden matrix with selective attention mechanism; Prediction result generation module: used to calculate the fully connected layer to obtain traffic flow prediction results, and also used to train the KGCN-GSAM model to obtain more accurate prediction results.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the method according to any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method described in any one of claims 1 to 5 are implemented.

Images