Traffic flow repairing method and system based on space-time fusion and self-attention mechanism

Abstract

The application discloses a traffic flow repairing method and system based on space-time fusion and self-attention mechanism. The application proposes a traffic flow repairing method combining space-time and space-time elimination. The space-time combination enables the model to add spatial information, time information and numerical values and regard them as a whole when using the self-attention mechanism, fully utilizes the spatial information of the road network and the before-and-after relationship of time in the road, and improves the utilization rate of road map information. The application proposes a space-time fusion and self-attention mechanism for traffic flow repairing. The space-time fusion and self-attention mechanism combines spatial and time information, enables the model to extract the implicit relationship of traffic flow between different roads, and learns the historical flow characteristics of the road map. The application proposes a method of repairing and fusing two kinds of sequence data in forward and reverse sequences. Two sequences are obtained through the traversal method in the forward and reverse sequences, various information of different sequences is extracted, and the repairing accuracy is improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of data mining and deep learning, and relates to a traffic flow repair method and system based on space-time fusion and self-attention mechanism. BACKGROUND

[0002] Since the 21st century, with the stable growth of the national economy, the number of private cars has been increasing. Due to the increase in the frequency of people's travel and the rapid growth of the number of cars, the pressure on urban traffic is increasing. The toll equipment can obtain the traffic flow, speed and other information of the road. The massive toll data has important research value, and the research with the aid of these data can provide effective support for the planning and recommendation of urban traffic construction and travel software. For example, the analysis of road traffic flow can mine the congestion degree of each road at each time. Urban construction can widen the road that is congested for a long time, and travel service providers can provide users with a route plan with lower congestion at that time. However, the missing data caused by network fluctuations, air quality and equipment failure will have a great impact on data analysis. Therefore, the repair of traffic data flow and speed is of great significance to the construction of smart city traffic.

[0003] At present, the solution to traffic data repair is mainly based on traditional repair models. The traditional repair method commonly used is the multiple linear interpolation method, which has a simple model, fast calculation speed, but poor robustness, and is only suitable for stable road sections. People often use deep learning prediction methods in repair, such as deep learning prediction models based on the Transformer architecture, which use learning on a large amount of historical data to predict the value of missing data. However, this method does not use information after the missing data section in the repair, and the data information extraction is insufficient in traffic data repair. SUMMARY

[0004] An object of the present application is to solve the above problems by providing a traffic flow repair method based on deep learning, a traffic flow repair model STMAR based on space-time fusion and self-attention mechanism. The model can learn the historical information of the same road and extract the mutual influence of traffic flow between different roads.

[0005] The technical solutions adopted by the present application are as follows:

[0006] Step 1: space-time data fusion;

[0007] 1-1 Generate a space embedding vector:

[0008] Substitute the traffic map information into the node2vec algorithm to calculate the space embedding vector E S of each sensor in the traffic map.

[0009] The traffic map information Including the road connection relationship between sensors and the distance, the road connection relationship includes two states of passable and impassable.

[0010] 1-2 Generate time embedding vector:

[0011] Assume that a day is divided into T time steps, and use one-hot to encode the day of the week and the time of the day of each time step into R 7 and R T , and connect R 7 , R T into a vector R T+7 , that is, the time feature; use a fully connected neural network to convert the time feature R T+7 into a time embedding vector E T .

[0012] E T = W d Concat(R T ,R 7 )+b d

[0013] Where W d and b d are the weights and biases of the fully connected layer, respectively;

[0014] 1-3 Fusion of space-time embedding vector:

[0015] According to the following formula, the space embedding vector E S and the time embedding vector E T are fused into a space-time embedding vector.

[0016] E ST = add(tan(E S ),tan(E T ))

[0017] Where add(·,·) is the matrix addition operation.

[0018] 1-4 Splitting space-time embedding vector

[0019] Split the time dimension of E ST , and split E ST into

[0020] Where represent t1-t p , t p+1 -t 2p , t 2p+1 -t3p the time period corresponding to the spatio-temporal embedding vector.

[0021] Step 2: Obtain the traffic data to be repaired by the sensor Centering on the missing data, the first t p data in X are arranged in a forward order The last t p data in X are arranged in a reverse order The traffic data includes one or both of traffic flow and vehicle speed.

[0022] Step 3: Use the built STMAR traffic flow repair model based on spatio-temporal fusion and self-attention mechanism to realize traffic flow repair.

[0023] The STMAR traffic flow repair model based on spatio-temporal fusion and self-attention mechanism includes two parallel networks and a fusion layer.

[0024] The inputs of the two parallel networks are forward sequence data X f and reverse sequence data X r , which have the same structure and include a spatio-temporal fusion block, a spatio-temporal self-attention block, and a spatio-temporal pruning block.

[0025] 1) The spatio-temporal fusion block is used to integrate the spatio-temporal embedding vector into the traffic data, specifically:

[0026] The X f and X r are encoded and converted into embedding vector representations by a fully connected layer:

[0027] E τ = ReLU(W τ X τ + b τ )

[0028] Where ReLU is an activation function, W τ and b τ are the weights and biases of the fully connected layer, and τ = f or r.

[0029] The spatio-temporal embedding vector E ST and the numerical embedding vector E τ are fused into a spatio-temporal numerical embedding vector E τ’ :

[0030]

[0031] 2) The spatio-temporal self-attention block includes time self-attention, spatial self-attention, and gated fusion; its input is E τ’ , which uses time self-attention mechanism and spatial self-attention mechanism to obtain outputs H τ,s, H τ,t , H τ,s , H τ,t After fusion, the output H τ is the output of the spatio-temporal self-attention block. Specifically:

[0032] ①By dimension exchange on the matrix E τ’ , we get and respectively. and extract important features through self-attention mechanism. The formula of self-attention mechanism is as follows:

[0033]

[0034]

[0035]

[0036] H τ,θ = softmax(query·key T )·value

[0037] where θ = s or t.

[0038] ②The H τ,s and H τ,t obtained above are fused using a fully connected layer to generate the output H τ of the spatio-temporal self-attention block. The specific formula is as follows:

[0039] H τ = W h (H τ,s ||H τ,t )+b h

[0040] where W h and b h are the weights and biases of the fully connected layer, and τ = f or r.

[0041] 3) The spatio-temporal pruning block is used to exclude the spatio-temporal information contained in the H τ output by the spatio-temporal self-attention block to obtain the repaired numerical data. Specifically:

[0042] For the input E τ’ of the spatio-temporal self-attention block, it represents a set of spatio-temporal data fusion vectors of all sensors at T time points. It not only stores the embedding vectors of the values recorded by all sensors at T time points, but also contains the spatio-temporal embedding vectors of all sensors at T time points. The output H τ, represents the set of spatiotemporal data fusion vectors from all sensors at time T. Compared to the data that needs repair, it includes spatiotemporal embedding vectors. Therefore, the spatiotemporal embedding vectors of all sensors at time T are removed during the spatiotemporal removal block repair period, as shown in the formula:

[0043]

[0044] Where Sub(·,·) represents subtraction. The repair time period (t) obtained in step 1 p+1 -t 2p The spatiotemporal embedding vector of ).

[0045] The fusion layer is used to combine the outputs of the two parallel networks. The final output Y is obtained by fusing the components according to the following formula.

[0046] For ascending and descending data X f and X r After their respective training The accuracy of the two repair results differs at different times; it may be greater or less than the accuracy of the two repair results. To merge the two reconstructed values ​​into a single, more accurate repair sequence, this method uses a fully connected layer to extract the importance of the two sequences.

[0047] The calculation formula is as follows:

[0048]

[0049] Where || represents the concatenation operation between vectors, W y and b y These are the weights and biases of the fully connected layer.

[0050] Another objective of this invention is to provide a traffic flow repair system based on spatiotemporal fusion and self-attention mechanism for implementing the above-mentioned method, including a traffic flow repair model STMAR based on spatiotemporal fusion and self-attention mechanism and a spatiotemporal data fusion module.

[0051] Another object of the present invention is to provide an electronic device including a processor and a memory, the memory storing machine-executable instructions executable by the processor, the processor executing the machine-executable instructions to implement the method.

[0052] Another object of the present invention is to provide a machine-readable storage medium storing machine-executable instructions that, when invoked and executed by a processor, cause the processor to implement the method described herein.

[0053] The specific beneficial effects of this invention are as follows:

[0054] (1) The application proposes a traffic flow repair method combining space and time and removing space and time. The combination of space and time enables the model to add spatial information, temporal information and numerical values when using self-attention mechanism and regard them as a whole, fully utilizes the spatial information of the road network and the temporal relationship before and after, and improves the utilization rate of road graph information.

[0055] (2) The application proposes a space-time fusion and self-attention mechanism for traffic flow repair. The space-time fusion and self-attention mechanism can extract the implicit relationship between traffic flows of different roads by combining spatial and temporal information, and learn the historical flow characteristics of the road graph. The accuracy of the repair is improved in the network road flow repair.

[0056] (3) The application proposes a method of repairing fusion of two sequence data of forward and reverse sequences. Two sequences are obtained by forward and reverse traversal methods, and various information of different sequences is extracted to improve the repair accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0057] Figure 1 is the space-time fusion graph of the application;

[0058] Figure 2 is the space-time self-attention graph of the application;

[0059] Figure 3 is the space-time removal graph of the application;

[0060] Figure 4 is the fusion layer graph of the application;

[0061] Figure 5 is the model structure diagram proposed by the application. DETAILED DESCRIPTION

[0062] The specific embodiments of the application will be further described in detail below with reference to the accompanying drawings.

[0063] Traffic flow repair method based on space-time fusion and self-attention mechanism, Figure 5 comprising the following steps:

[0064] Step 1: Space-time data fusion:

[0065] 1-1 Generate spatial embedding vector:

[0066] As shown in Figure 1 , the graph information is composed of the route relationship of each sensor on the map, the sensor representation is learned using the node2vec method, and the spatial embedding vector / coding E S of the sensor is generated, and the vector length is 8.

[0067] 1-2 Generating time embedding vector:

[0068] The time information input mainly extracts the week, hour, and minute information in the time information for encoding. For example, the information of (2017-4-11 12:20:00, Saturday), the 12 o'clock, 20 minutes, and Saturday information of the information are extracted. The time of each day is divided into 5-minute segments and numbered, and the number of 12:20 is (12*60+20) / 5, that is, 1444, which is converted into a one-bit effective code (one-hot encoding) R T is [0 0 0 … 1 0 …], where the subscript of 1 is 1444. The Saturday is converted into a one-bit effective code R 7 [0 00 0 0 1 0].

[0069] The two vectors are mapped into a vector E T according to the following formula: D , which is a space-time vector. Where W T is a learnable fully connected layer parameter. The length of the time embedding vector / encoding E T is 8.

[0070] E d = W T Concat(R 7 ,R d )+b d

[0071] where W d and b S are the weights and biases of the fully connected layer, respectively;

[0072] 1-3 Fusion of space-time embedding vector:

[0073] For the above-mentioned time embedding vector E T and the space embedding vector E S , the method first uses the tanh activation function to E T and E ST , and then adds the vectors to fuse them into a space-time embedding vector E ST (the length of the embedding vector is 8), and the formula is as follows:

[0074] E S = add(tan(E T ), tan(E 3p ))

[0075] where add(·,·) is a matrix addition operation.

[0076] 1-4 Splitting the space-time embedding vector:

[0077] For the space-time embedding vector obtained in 1-3, it covers t1-tThe space-time embedding vectors of the 3p time points in the time period are divided into 2 blocks:

[0078] Wherein respectively represent the space-time embedding vectors corresponding to t1-t p , t p+1 -t 2p , t 2p+1 -t 3p .

[0079] Step 2: Obtain the traffic data to be repaired by the sensor Where the missing part is The front part of X is constructed in a positive order The rear part of X is constructed in a reverse order Wherein the traffic data includes one or both of the traffic flow and the vehicle speed.

[0080] Step 3: Use the built STMAR traffic flow repair model based on space-time fusion and self-attention mechanism to realize traffic flow repair.

[0081] The STMAR traffic flow repair model based on space-time fusion and self-attention mechanism includes two parallel networks and a fusion layer.

[0082] The inputs of the two parallel networks are respectively the positive order sequence data X f and the reverse order sequence data X r , which have the same structure and include a space-time fusion block, a space-time self-attention block and a space-time pruning block.

[0083] 1) The space-time fusion block is used to fuse the space-time embedding vector into the traffic data, specifically:

[0084] X f and X r are converted into an embedding vector E τ with a length of 8 through full connection layer coding, and the addition formula is as follows.

[0085] E τ =ReLU(W τ X τ +b τ )

[0086] Where ReLU is an activation function, W τ and b τ are the weights and biases of the full connection layer, and τ=f or r;

[0087] The space-time embedding vector E ST (matrix size [1, 325, 8]) and the numerical embedding vector E τ(The matrix size is [12, 1, 8]) is fused into a spatiotemporal numerical embedding vector E. τ’ :

[0088]

[0089] The obtained E τ’ The size of the matrix is ​​[12, 325, 8].

[0090] 2) The spatiotemporal self-attention block includes temporal self-attention, spatial self-attention, and gated fusion; its input is E. τ’ By utilizing temporal and spatial self-attention mechanisms, the output H is obtained respectively. τ,s H τ,t Then H τ,s H τ,t The output H after fusion τ This is the output of the spatiotemporal self-attention block. Figure 2 Specifically:

[0091] 2-1 By examining matrix E τ’ (The matrix of size [12, 325, 8]) is obtained by performing dimension swapping. The matrix size is [12, 325, 8] and The matrix size is [325, 12, 8].

[0092] Where 12, 325, and 8 represent the length of the time dimension, the length of the spatial dimension, and the length of the embedding vector, respectively.

[0093] And on and Important features are extracted using a self-attention mechanism. The calculation formula for the self-attention mechanism is as follows:

[0094]

[0095]

[0096]

[0097] H τ,θ =softmax(query·key) T )·value

[0098] Where θ = s or t.

[0099] 2-2 Using a fully connected layer to fuse the H obtained above τ,s and H τ,t The output H of the generated spatiotemporal self-attention block τ The specific formula is as follows:

[0100] Hτ = W h τ, s||H τ,t + b h

[0101] where W h and b h are the weights and bias of the fully connected layer, τ = f or r;

[0102] 3) the spatiotemporal culling block, configured to exclude the spatiotemporal information contained in the H τ output by the spatiotemporal self-attention block, to obtain repaired numerical data. Figure 3 Specifically,

[0103] For the input E τ’ of the spatiotemporal self-attention block, it represents a set of spatiotemporal data fusion vectors of all sensors at T time points. It not only stores the embedding vectors of the values recorded by all sensors at T time points, but also contains the spatiotemporal embedding vectors of all sensors at T time points. The output H τ of the spatiotemporal self-attention block represents a set of spatiotemporal data fusion vectors of all sensors at T time points. Compared with the data to be repaired, it has more spatiotemporal embedding vectors, so the spatiotemporal culling block excludes the spatiotemporal embedding vectors of all sensors at T time points, as shown in the formula:

[0104]

[0105] where Sub(·,·) represents subtraction, the spatiotemporal embedding vectors of the repair time period (t p+1 -t 2p ) obtained in step 1.

[0106] The fusion layer is configured to fuse the H output by the two parallel networks according to the following formula to obtain the final output Y, as shown in Figure 4 .

[0107] For the normal order data X f and the reverse order data X r , the repair results obtained through respective training to have different accuracies at different time points, which may be greater or less. In order to fuse the two reconstruction values and make them become a repaired sequence data with higher accuracy, the method uses a fully connected layer to extract the importance of the two sequences.

[0108] The calculation formula is as follows:

[0109]

[0110] ​where || is the concatenation operation between vectors, W y and b y are the weights and bias of the fully connected layer.

[0111] The experiments of the present application were performed on two datasets, which were the Bay Area Freeway Dataset (PEMS) and the Los Angeles Freeway Dataset (METR). The Bay Area Freeway Dataset contained 6 months of data recorded by 325 traffic sensors in the Bay Area from January 1, 2017 to June 30, 2017, with a total of 52116 data. The Los Angeles Freeway Dataset contained traffic information collected from loop detectors on the freeways of Los Angeles County. We selected 207 sensors and collected 4 months of data from March 1, 2012 to June 30, 2012, with a total of 37272 data.

[0112]

[0113] The effect indicators used in the present application are MAE, RMSE and MAPE. The calculation formulas are as follows

[0114]

[0115]

[0116]

[0117] And we repaired the data set missing 15 minutes, 30 minutes and 60 minutes respectively, and compared with other three methods. The repair results are as follows:

[0118] Table 1: Experimental results of the Bay Area Freeway Dataset (PEMS)

[0119]

[0120] Table 2: Experimental results of the Los Angeles Freeway Dataset (METR)

[0121]

[0122]

[0123] The model proposed in the present application is the STMAR model, compared with the Linear model, the CPI model and the GMAN model. Linear uses the data near the missing data for interpolation repair. CPI replicates data blocks with similar characteristics and pastes them into the gaps of the time series, and GMAN uses graph self-attention for data completion. It can be seen that the model proposed in the present application has an improvement compared with them on the two datasets.

Claims

1. A traffic flow repairing method based on spatio-temporal fusion and self-attention mechanism, characterized in that The method comprises the following steps: Step (1): spatio-temporal data fusion; Step (2): obtaining the traffic data to be repaired by a sensor With the missing data as the center, the first t p data in X are arranged in a forward order The last t p data in X are arranged in a reverse order Wherein the traffic data includes one or both of traffic flow and vehicle speed Step (3): traffic flow repair is realized by using the built traffic flow repair model STMAR based on spatio-temporal fusion and self-attention mechanism; The traffic flow repair model STMAR based on spatio-temporal fusion and self-attention mechanism comprises two parallel networks and a fusion layer; The two parallel networks have the same structure, and each includes a space-time fusion block, a space-time self-attention block and a space-time pruning block f , and the input of each is a forward sequence data X r and a reverse sequence data X The spatio-temporal fusion block is used for fusing the spatio-temporal embedding vector into the traffic data, and specifically is: X through a fully connected layer f X r Encode and convert to an embedded vector representation: E τ = ReLU(W τ X τ + b τ ) where ReLU is an activation function, W τ and b τ are weights, bias of the fully connected layer, τ = f or r; spatiotemporal embedding vector E ST and a numerical embedding vector E τ fused into a spatiotemporal numerical embedding vector E τ’ : The spatio-temporal self-attention block comprises a time self-attention, a space self-attention and a gated fusion; an input of the spatio-temporal self-attention block is E τ’ , the time self-attention mechanism and the space self-attention mechanism are utilized to obtain outputs H τ，s , H τ，t , respectively, and the outputs H τ，s , H τ，t after fusion are H τ ; The spatiotemporal culling block is configured to perform spatiotemporal culling on H τ The spatiotemporal information contained therein is excluded to obtain repaired numerical data; specifically: Input E of the spatio-temporal self-attention block τ’ , represents a set of spatio-temporal data fusion vectors of all sensors at T time points; it not only stores the embedding vectors of the values recorded by all sensors at T time points, but also contains the spatio-temporal embedding vectors of all sensors at T time points; the output H of the spatio-temporal self-attention τ , represents a set of spatio-temporal data fusion vectors of all sensors at T time points, which has more spatio-temporal embedding vectors than the data to be repaired; therefore, the spatio-temporal embedding vectors of all sensors at T time points are removed in the spatio-temporal removal block, as shown in the formula: where Sub(·, ·) denotes subtraction, a repair time period (t p+1 -t 2p ) for the traffic data to be repaired according to the repair time period (t ) obtained in step (1); The fusion layer is configured to fuse the outputs of the two parallel networks The fusion is performed according to the following formula to obtain the final output Y; where || is the concatenation operation between vectors, W y and b y are the weights and bias of the fully connected layer.

2. The method of claim 1, wherein Step (1) is specifically: 1-1 generating a spatial embedding vector: Traffic map information is obtained The node2vec algorithm is applied to calculate the spatial embedding vectors E of each sensor in the traffic map S ; The traffic map information The road connection relationship between the sensors includes passable and impassable states. 1-2 generating a time embedding vector: Assume a day is divided into T time steps, encode the day of the week and the time of the day of each time step into R 7 and R T , and concatenate R 7 , R T into a vector R T+7 , which is the time feature; convert the time feature R T+7 into a time embedding vector E T using a fully connected neural network; E T = W d Concat(R T , R 7 ) + b d where W d and b d are weights, biases of the fully connected neural network, respectively. 1-3 fusing the spatio-temporal embedding vector: The spatial embedding vector E S and the temporal embedding vector E T are fused into a spatio-temporal embedding vector; E ST = add(tan(E S ), tan(E T )) Wherein add(·,·) is a matrix addition operation; 1-4 splitting the spatio-temporal embedding vector According to the time dimension, E ST is split into wherein represents the spatio-temporal embedding vector corresponding to the time period t p , t p+1 , t 2p , t 2p+1 , and t 3p , respectively.

3. The method of claim 1, wherein Step (3) the spatio-temporal self-attention block is specifically: ①By dimension exchange on matrix E τ’ and respectively on and important features are extracted by self-attention mechanism; the calculation formula of self-attention mechanism is as follows:​ H τ，θ = softmax(query · key T ) · value Wherein θ=s or t; ii. using a fully connected layer to fuse the H τ，s and H τ，t to generate the output H τ of the spatio-temporal white attention block, in particular as follows: H τ = W h (H τ， || H τ，t ) + b h where W h and b h are weights, biases of the fully connected layer, τ = f or r.

4. A spatio-temporal fusion and self-attention mechanism based traffic flow inpainting system implementing the method of any one of claims 1-3, characterized in that The method comprises a traffic flow repair model STMAR based on spatio-temporal fusion and self-attention mechanism, and a spatio-temporal data fusion module.

5. An electronic device, comprising: The processor and the memory are included, the memory has machine executable instructions capable of being executed by the processor, and the processor executes the machine executable instructions to realize the method in any one of claims 1-3.

6. A machine-readable storage medium, characterized in that, The machine readable storage medium stores machine executable instructions, and when the machine executable instructions are called and executed by the processor, the machine executable instructions cause the processor to realize the method in any one of claims 1-3.

Images