Traffic flow prediction method for traffic flow spatio-temporal data information
By introducing attention mechanism and space-time cross attention mechanism into the traffic flow prediction model, and integrating time and space characteristics, the problem of existing models ignoring space-time correlation is solved, which significantly improves the accuracy and generalization ability of traffic flow prediction.
Patent Information
- Application Number
- CN202411115452.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-14
- Publication Date
- 2024-12-24
AI Technical Summary
When processing space-time information, the existing traffic flow prediction model ignores the intrinsic correlation between space-time information, resulting in insufficient prediction accuracy.
Using the traffic flow prediction method based on attention mechanism, the time and space characteristics are integrated through the time self-attention layer and the space-time cross-attention mechanism, combined with the feature extraction block and the space-time self-attention layer, to achieve full fusion of space-time information, and to utilize the The regression layer generates prediction results.
It improves the accuracy and generalization ability of traffic flow prediction, significantly improves the prediction accuracy, and provides new ideas for smart transportation.
Smart Images
Figure CN119181236A_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of traffic flow prediction, and in particular relates to a traffic flow prediction method oriented to traffic flow spatiotemporal data information. Background Art
[0002] Traffic flow prediction is of great significance in the fields of traffic management, urban computing and public safety. With the advancement of urbanization and the increase in car ownership, traffic flow prediction has received increasing attention. The core of this problem is to use historical data obtained by sensors to predict indicators such as traffic flow, vehicle speed and road occupancy rate in the future in order to arrange transportation tools reasonably. Traffic flow prediction is essentially a multivariate time series prediction problem, but due to the large number of urban road nodes and complex structures, especially in busy sections, the strong interaction between nodes, traffic fluctuations and the uncertainty of emergencies increase the difficulty of prediction.
[0003] How to effectively utilize historical data is a key issue in traffic flow prediction. In recent years, traffic flow prediction models based on deep learning have achieved remarkable success, mainly because they can capture the inherent spatiotemporal dependencies in traffic systems. Among them, spatiotemporal graph neural networks and attention-based models have become very popular due to their excellent performance. In addition, researchers have developed many sophisticated models for traffic flow prediction, including graph convolution models, learning graph structure models, efficient attention mechanism models, and other models. However, these methods all have obvious shortcomings, that is, they extract temporal information and spatial information independently, ignoring the inherent correlation between temporal and spatial information, which is precisely the key to traffic flow prediction. Summary of the invention
[0004] In view of the deficiencies in the prior art, the present invention proposes a traffic flow prediction method oriented to traffic flow spatiotemporal data information.
[0005] The specific implementation scheme of the present invention is as follows:
[0006] A traffic flow prediction method for traffic flow spatiotemporal data information includes the following steps:
[0007] S1: Obtaining the original data of traffic flow and interpolating and standardizing the missing data to obtain pre-processed data; and segmenting the pre-processed traffic flow data;
[0008] S2: The data processed by S2 is passed to the embedding layer for processing to obtain the embedded result.
[0009] S3: Using the attention mechanism, the embedded result in S2 is processed along the time dimension to obtain the output of the temporal self-attention layer.
[0010] S4: Using the spatiotemporal cross attention mechanism and feature extraction block, the output of the temporal self-attention layer is processed along the spatial dimension to obtain an output that integrates spatiotemporal attention.
[0011] S5: Using the spatial attention mechanism, the output of the fused spatiotemporal attention is processed along the spatial dimension to obtain the output result of the spatial self-attention layer that integrates spatiotemporal features.
[0012] S6. Use the regression layer to process the output results of the spatial self-attention layer that integrates the spatiotemporal features to obtain the final prediction result.
[0013] S7. Repeat the above steps and train the model using the data in the training set.
[0014] S8. Use the data in the test set and validation set to test the performance of the trained model.
[0015] When making predictions in real time, the data from the previous hour of the time period to be tested (one hour in length) in the validation and test sets is directly used as input to obtain the predicted value.
[0016] Furthermore, the specific method of step S1 is as follows:
[0017] First, the data obtained from the sensor are sorted in chronological order. The data obtained from the sensor at least includes the flow, speed and lane occupancy information of the road section. These data are divided into training, verification and test sets according to a certain ratio.
[0018] For the missing data, the mean value method is used for interpolation, that is, the mean value of the sum of the data at the moment before the missing data and the data at the moment after the missing data is used as a substitute for the missing data.
[0019] After completing the data interpolation, the data is normalized so that all data values are scaled to between [-1, 1]. The process of normalizing the data includes:
[0020]
[0021] Where X is the value of the original data, mean(X) is the average value of all X, σ represents the standard deviation of the data, and X' is the normalized scaled data.
[0022] Furthermore, the scaled data is divided into several batches in the order of a group of 12 timestamps (the interval between each timestamp is 5 minutes).
[0023] Furthermore, the specific method of step S2 is as follows:
[0024] The embedding layer is divided into three main parts: the spatiotemporal embedding layer, the adaptive embedding layer, and the periodic embedding layer.
[0025] The spatiotemporal embedding layer consists of two parts: the spatial embedding block T s and the temporal embedding block T t . where d f For the spatial embedding block T s The output dimension of where d t is the time embedding block T t The output dimension of the spatiotemporal embedding layer is represented as
[0026] The period embedding layer consists of two parts: timestamp embedding block T d and weekday information embedded in block T w . Where N d =288 is the number of timestamps per day. Where N w =7 is the number of days in a week. Periodic embedding
[0027] Adaptive Embedding Layer Used to adaptively capture features in space and time, where d a Adaptive Embedding Layer E A The output dimension of .
[0028] By concatenating the above embeddings, we get the total representation of the embedding layer where d h =3d f +d t +d a , which is expressed as follows:
[0029] Z∈E P ||E A ||E TS ;
[0030] Through the embedding layer, the input data H∈R B×T×N×D The output H' of the embedding layer is obtained after processing, where B represents the number of batches of input data, T represents the number of timestamps contained in a batch of data (the interval between timestamps is 5 minutes, and a batch contains 12 timestamps), N refers to the total number of sensors, and D is the number of dimensions of the information obtained by the sensor.
[0031] Furthermore, the specific method of step S3 is as follows:
[0032] Apply the basic attention mechanism on the time axis to capture complex traffic relationships. The input of the temporal self-attention layer is the output H' of the embedding layer. The query matrix Q is obtained through the temporal self-attention layer. (te) , key matrix K (te) Sum value matrix V (te) .
[0033]
[0034] in, is a learnable parameter. t , K t 、V t The value is H', the self-attention weight A (te) It is expressed as:
[0035]
[0036] Among them, A (te) ∈R N×T×T The temporal relationship between different nodes is captured. Finally, the output of the temporal self-attention layer is obtained:
[0037] Z (te) =BN(F(A (te) V (te) ));
[0038] Among them, F represents the feed-forward neural layer and BN represents the batch processing layer.
[0039] Furthermore, the specific method of step S4 is as follows:
[0040] The feature extraction block consists of three key components, namely key embedding, value embedding and attention embedding. Key embedding consists of a convolution kernel Cov 3 ∈R 3×3 , a batch normalization layer RelU activation layer. The value embedding consists of a convolution kernel Cov 1 ∈R 1×1 And a batch normalization layer. The attention embedding consists of two Cov 1 ∈R 1×1 The convolution kernel, a batch normalization layer and a RelU activation layer;
[0041] First, the output Z of the temporal self-attention layer is (te) Assign it to Q as input f , K f and V f , where Q f , K f and V fis an intermediate variable. Using key embedding and K f To generate the key feature matrix K 1 , using value embedding and V f Generate the value feature matrix V' for subsequent attention weighting, the formula is as follows:
[0042] K 1 =KeyEmbedding(K f );
[0043] V'=ValueEmbedding(V f );
[0044] Afterwards, the key feature matrix K 1 With Q f The key values and Y' are concatenated in the feature dimension, and the attention distribution Att is obtained after attention embedding. Next, the calculated attention distribution Att and the value feature matrix V' are dot-producted to obtain the weighted feature matrix K 2 .
[0045] Y'=Concat(K 1 ,Q f );
[0046] Att = ValueEmbedding(Y′);
[0047] K 2 =DotProduct(Att,V′);
[0048] Finally, the key feature matrix K 1 With the weighted feature matrix K 2 Add the values of the corresponding positions to get the final output features of the feature extraction block
[0049]
[0050] Furthermore, the specific method of the spatiotemporal cross attention mechanism is as follows:
[0051] The output of the feature extraction block The cross attention calculation is performed along the spatial axis with the output H' of the embedding layer in order to pre-learn the temporal features before extracting the spatial features. The learning process is as follows:
[0052]
[0053] Among them, n represents the number of spatiotemporal cross attention layers. Only when n is 1, The value of is H'. CrossAttentino represents spatiotemporal cross attention.
[0054] Furthermore, the specific method of step S5 is as follows:
[0055] The output of the fusion of spatiotemporal attention Input to the spatial self-attention layer, continue to perform self-attention calculation along the spatial dimension, and obtain the output result Z' of the spatial self-attention layer that integrates the spatiotemporal features. The process is shown as follows:
[0056]
[0057] Among them, SelfAttention represents spatial self-attention.
[0058] Furthermore, the specific method of step S6 is as follows:
[0059] The output of the spatial self-attention layer Generate prediction values through regression layer. The detailed process can be expressed as:
[0060]
[0061] in, is the predicted value, T' is the predicted time span, d tar is the target dimension, which is fixed to 1. Therefore, the regression layer maps the tensor dimension from T×D in Z' to T'×(d tar =1).
[0062] Furthermore, the specific method of step S7 is as follows:
[0063] The data in the training set is used to train the parameters within the overall model.
[0064] The loss function selected is the mean absolute error (MAE). The training rounds are set to 100 rounds, Adam is selected as the optimizer, and the learning rate is decayed from 0.001.
[0065] The beneficial effects of the present invention are as follows:
[0066] The present invention proposes a prediction model based on the attention mechanism for traffic flow prediction tasks, which improves the prediction accuracy. The innovation of the present invention is that the cross-attention mechanism of Transformer and the original feature extraction block are used to pre-learn the time features, which intersects with the previous model to more fully integrate the time features and spatial features in the traffic flow task; the results of the present invention are effective, showing strong performance and generalization ability on multiple data sets, further improving the accuracy of traffic flow prediction, and providing a new idea for smart transportation. BRIEF DESCRIPTION OF THE DRAWINGS
[0067] Figure 1 Schematic diagram of a model framework of an embodiment of the present invention.
[0068] Figure 2 Schematic diagram of the structure of the feature extraction block of an embodiment of the present invention.
[0069] Figure 3 Schematic diagram of components of a feature extraction block according to an embodiment of the present invention.
[0070] Figure 4 The performance of the present invention (TSformer) is compared with other models with the same task on the METR-LA and PEMS-BAY datasets.
[0071] Figure 5 The performance of the present invention (TSformer) is compared with other models of the same task on the PEMS03, 04, 07 and 08 datasets. DETAILED DESCRIPTION
[0072] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0073] The present invention proposes a traffic flow prediction method oriented to traffic flow spatiotemporal data information, such as Figure 1 As shown, the specific steps include:
[0074] S1: Obtain the original data of traffic flow and interpolate and standardize the missing data to obtain preprocessed data; and segment the preprocessed traffic flow data.
[0075] First, the data obtained from the sensor are sorted in chronological order. The data obtained from the sensor at least includes the flow, speed and lane occupancy information of the road section. These data are divided into training, verification and test sets according to a certain ratio.
[0076] For the missing data, the mean value method is used for interpolation, that is, the mean value of the sum of the data at the moment before the missing data and the data at the moment after the missing data is used as a substitute for the missing data.
[0077] After completing the data interpolation, the data is normalized so that all data values are scaled to between [-1, 1]. The process of normalizing the data includes:
[0078]
[0079] Where X is the value of the original data, mean(X) is the average value of all X, σ represents the standard deviation of the data, and X' is the normalized scaled data.
[0080] Furthermore, the scaled data is divided into several batches in the order of a group of 12 timestamps (the interval between each timestamp is 5 minutes).
[0081] S2: The data processed by S2 is passed to the embedding layer for processing to obtain the embedded result.
[0082] The embedding layer is divided into three main parts: the spatiotemporal embedding layer, the adaptive embedding layer, and the periodic embedding layer.
[0083] The spatiotemporal embedding layer consists of two parts: the spatial embedding block T s and the temporal embedding block T t . where d f For the spatial embedding block T s The output dimension of where d t is the time embedding block T t The output dimension of the spatiotemporal embedding layer is represented as
[0084] The period embedding layer consists of two parts: timestamp embedding block T d and weekday information embedded in block T w . Where N d =288 is the number of timestamps per day. Where N w =7 is the number of days in a week. Periodic embedding
[0085] Adaptive Embedding Layer Used to adaptively capture features in space and time, where d a Adaptive Embedding Layer E A The output dimension of .
[0086] By concatenating the above embeddings, we get the total representation of the embedding layer where d h =3d f +d t +d a , which is expressed as follows:
[0087] Z∈E P ||E A ||E TS ;
[0088] Through the embedding layer, the input data H∈R B×T×N×D The output H' of the embedding layer is obtained after processing, where B represents the number of batches of input data, T represents the number of timestamps contained in a batch of data (the interval between timestamps is 5 minutes, and a batch contains 12 timestamps), N refers to the total number of sensors, and D is the number of dimensions of the information obtained by the sensor.
[0089] S3: Using the attention mechanism, the embedded result in S2 is processed along the time dimension to obtain the output of the temporal self-attention layer.
[0090] Apply the basic attention mechanism on the time axis to capture complex traffic relationships. The input of the temporal self-attention layer is the output H' of the embedding layer. The query matrix Q is obtained through the temporal self-attention layer. (te) , key matrix K (te) Sum value matrix V (te) .
[0091]
[0092] in, is a learnable parameter. t , K t 、V t The value is H', the self-attention weight A (te) It is expressed as:
[0093]
[0094] Among them, A (te) ∈R N×T×T The temporal relationship between different nodes is captured. Finally, the output of the temporal self-attention layer is obtained:
[0095] Z (te) =BN(F(A (te) V (te) ));
[0096] Among them, F represents the feed-forward neural layer and BN represents the batch processing layer.
[0097] S4: Using the spatiotemporal cross attention mechanism and feature extraction block, the output of the temporal self-attention layer is processed along the spatial dimension to obtain an output that integrates spatiotemporal attention.
[0098] The feature extraction block consists of three key components, namely key embedding, value embedding and attention embedding. Key embedding consists of a convolution kernel Cov 3 ∈R3×3 , a batch normalization layer RelU activation layer. The value embedding consists of a convolution kernel Cov 1 ∈R 1×1 And a batch normalization layer. The attention embedding consists of two Cov 1 ∈R 1×1 The convolution kernel, a batch normalization layer and a RelU activation layer.
[0099] First, the output Z of the temporal self-attention layer is (te) Assign it to Q as input f , K f and V f , where Q f , K f and V f is an intermediate variable. Using key embedding and K f To generate the key feature matrix K 1 , using value embedding and V f Generate the value feature matrix V' for subsequent attention weighting, the formula is as follows:
[0100] K 1 =KeyEmbedding(K f );
[0101] V'=ValueEmbedding(V f );
[0102] Afterwards, the key feature matrix K 1 With Q f The key values and Y' are concatenated in the feature dimension, and the attention distribution Att is obtained after attention embedding. Next, the calculated attention distribution Att and the value feature matrix V' are dot-producted to obtain the weighted feature matrix K 2 .
[0103] Y'=Concat(K 1 ,Q f );
[0104] Att = ValueEmbedding(Y′);
[0105] K 2 =DotProduct(Att,V′);
[0106] Finally, the key feature matrix K 1 With the weighted feature matrix K 2 Add the values of the corresponding positions to get the final output features of the feature extraction block
[0107]
[0108] The specific method of the spatiotemporal cross attention mechanism is as follows:
[0109] The output of the feature extraction block The cross attention calculation is performed along the spatial axis with the output H' of the embedding layer in order to pre-learn the temporal features before extracting the spatial features. The learning process is as follows:
[0110]
[0111] Among them, n represents the number of spatiotemporal cross attention layers. Only when n is 1, The value of is H'. The CrossAttention method here has the same principle as the attention mechanism, but the input is different. For details, see S3.
[0112] S5: Using the spatial attention mechanism, the output of the fused spatiotemporal attention is processed along the spatial dimension to obtain the output result of the spatial self-attention layer that integrates spatiotemporal features.
[0113] The output of the fusion of spatiotemporal attention Input to the spatial self-attention layer, continue to perform self-attention calculation along the spatial dimension, and obtain the output result Z' of the spatial self-attention layer that integrates the spatiotemporal features. The process is shown as follows:
[0114]
[0115] The principle of the self-attention method here is the same as the attention mechanism, except that the input is different. For details, see S3.
[0116] S6. Use the regression layer to process the output results of the spatial self-attention layer that integrates the spatiotemporal features to obtain the final prediction result.
[0117] The output of the spatial self-attention layer Generate prediction values through regression layer. The detailed process can be expressed as:
[0118]
[0119] in, is the predicted value, T' is the predicted time span, d tar is the target dimension, which is fixed to 1 in this invention. Therefore, the regression layer maps the tensor dimension from T×D in Z' to T'×(d tar =1).
[0120] S7. Repeat the above steps and train the model using the data in the training set.
[0121] We use the data in the training set to train the parameters within the model.
[0122] The loss function we choose is the mean absolute error MAE. The calculation formula of MAE is as follows:
[0123]
[0124] Where n is the number of samples, Y i is the true value, Y' i is the predicted value.
[0125] We set the training rounds to 100 rounds, we choose Adam as the optimizer, and the learning rate decays from 0.001.
[0126] S8. Use the data in the test set and validation set to test the performance of the trained model.
[0127] When making predictions in real time, we directly use the data of the previous hour of the test period (one hour in length) in the validation and test sets as input to get the predicted value. We take the set of model parameters with the best performance in the validation set during 100 rounds of training as the final parameters of the model, and then record the test set performance at this time as the optimal performance of the model.
[0128] We implemented the model using the PyTorch toolkit on a Windows server with a GeForce RTX 4070Ti GPU. We split METR-LA and PEMS-BAY into training, validation, and test sets in a ratio of 7:1:2, and PEMS03, PEMS04, PEMS07, and PEMS08 in a ratio of 6:2:2. In more detail, d a 84,d f 24,d t is 28. The number of temporal and spatial self-attention layers is 3, the number of temporal and spatial cross-attention layers is 2, and the number of attention mechanism heads is 4. We set the input and prediction length to 1 hour, that is, T = T' = 12.
[0129] The present invention has achieved good performance in traffic flow tasks. For specific data, see Figure 4 and Figure 5 . Figure 4 The performance comparison of our model (TSformer) and other models on the METR-LA and PEMS-BAY datasets and other models with the same task is shown. Figure 5The performance of the present invention (TSformer) is compared with other models with the same task on the PEMS03, 04, 07 and 08 datasets. The comparison performance indicators are mean absolute error (Mean Absolute Error), root mean square error (Root MeanSquare Error) and mean absolute percentage error (MeanAbsolutePercentageError).
[0130] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.
Claims
1. A traffic flow prediction method based on spatiotemporal data information of traffic flow, characterized in that: The following steps are involved: S1: Obtaining the original data of traffic flow and interpolating and standardizing the missing data to obtain pre-processed data; and segmenting the pre-processed traffic flow data; S2: The data processed by S2 is passed to the embedding layer for processing to obtain the embedded result; S3: Using the attention mechanism, the embedded result in S2 is processed along the time dimension to obtain the output of the temporal self-attention layer; S4: Using the spatiotemporal cross attention mechanism and feature extraction block, the output of the temporal self-attention layer is processed along the spatial dimension to obtain an output that integrates spatiotemporal attention; S5: Using the spatial attention mechanism, the output of the fused spatiotemporal attention is processed along the spatial dimension to obtain the output result of the spatial self-attention layer that integrates the spatiotemporal features; S6. Use the regression layer to process the output of the spatial self-attention layer that integrates the spatiotemporal features to obtain the final prediction result; S7. Repeat the above steps to train the model using the data in the training set; S8. Use the data in the test set and validation set to perform performance tests on the trained model. When making predictions in real time, the data from the previous hour of the time period to be tested in the validation and test sets is directly used as input to obtain the predicted value.
2. The traffic flow prediction method based on traffic flow spatiotemporal data information according to claim 1, characterized in that: The specific method of step S1 is as follows: First, the data obtained from the sensor are sorted in chronological order; the data obtained from the sensor at least includes the flow, speed and lane occupancy rate information of the road section; these data are divided into training, verification and test sets according to a certain ratio; For the missing data, the mean value method is used for interpolation, that is, the mean value of the sum of the data at the moment before the missing data and the data at the moment after the missing data is used as a substitute for the missing data; After completing the data interpolation, the data is normalized so that all data values are scaled to between [-1, 1]. The process of normalizing the data includes: Where X is the value of the original data, mean(X) is the average value of all X, σ represents the standard deviation of the data, and X' is the scaled data after normalization; The scaled data is divided into batches in groups of 12 timestamps, with an interval of 5 minutes between each timestamp.
3. The traffic flow prediction method based on traffic flow spatiotemporal data information according to claim 1, characterized in that: The specific method of step S2 is as follows: The embedding layer is divided into three main parts: the first is the spatiotemporal embedding layer, the second is the adaptive embedding layer, and the third is the periodic embedding layer; The spatiotemporal embedding layer consists of two parts: the spatial embedding block T s and the temporal embedding block T t ; where d f For the spatial embedding block T s The output dimension of where d t is the time embedding block T t The output dimension of the spatiotemporal embedding layer is The period embedding layer consists of two parts: timestamp embedding block T d and weekday information embedded in block T w ; Where N d =288 is the number of timestamps per day; Where N w =7 is the number of days of the week; periodic embedding Adaptive Embedding Layer Used to adaptively capture features in space and time, where d a Adaptive Embedding Layer E A The output dimension of By concatenating the above embeddings, we get the total representation of the embedding layer where d h =3d f +d t +d a , which is expressed as follows: Z∈E P ||And A ||And TS ; Through the embedding layer, the input data H∈R B×T×N×D Enter the processing to get the output H' of the embedding layer, where B represents the number of batches of input data, T represents the number of timestamps contained in a batch of data, the timestamp interval is 5 minutes, and a batch contains 12 timestamps. N refers to the total number of sensors, and D is the number of dimensions of the information obtained by the sensor.
4. The traffic flow prediction method based on traffic flow spatiotemporal data information according to claim 3 is characterized in that: The specific method of step S3 is as follows: Apply the basic attention mechanism on the time axis to capture complex traffic relationships; the input of the temporal self-attention layer is the output H' of the embedding layer; obtain the query matrix Q through the temporal self-attention layer (te) , key matrix K (te) Sum value matrix V (te) ; in, is a learnable parameter; Q t , K t 、V t The value is H', the self-attention weight A (te) It is expressed as: Among them, A (te) ∈R N×T×T The temporal relationship among different nodes is captured; finally, the output of the temporal self-attention layer is obtained: With (te) =BN(F(A (te) In (te) )); Among them, F represents the feed-forward neural layer and BN represents the batch processing layer.
5. The traffic flow prediction method based on traffic flow spatiotemporal data information according to claim 4, characterized in that: The specific method of step S4 is as follows: The feature extraction block consists of three key components: key embedding KeyEmbedding, value embedding ValueEmbedding and attention embedding ValueEmbedding; the key embedding is composed of a convolution kernel Cov3∈R 3×3 , a batch normalization layer RelU activation layer; the value embedding consists of a convolution kernel Cov1∈R 1×1 and a batch normalization layer; the attention embedding consists of two Cov1∈R 1×1 The convolution kernel, a batch normalization layer and a RelU activation layer; First, the output Z of the temporal self-attention layer is (te) Assign it to Q as input f , K f and V f , where Q f , K f and V f is an intermediate variable; use key embedding and Kf to generate the key feature matrix K1, use value embedding and V f Generate the value feature matrix V' for subsequent attention weighting, the formula is as follows: K1=KeyEmbedding(K f ); V'=ValueEmbedding(V f ); After that, the key feature matrix K1 and Q f The key values and Y' are concatenated in the feature dimension, and the attention distribution Att is obtained after attention embedding. Then, the calculated attention distribution Att and the value feature matrix V' are dot-producted to obtain the weighted feature matrix K2. Y'=Concat(K1,Q f ); Att = ValueEmbedding(Y′); K2 = DotProduct(Att, V′); Finally, add the values of the corresponding positions of the key feature matrix K1 and the weighted feature matrix K2 to obtain the final output feature of the feature extraction block 6. The traffic flow prediction method based on spatiotemporal data information of traffic flow according to claim 5, characterized in that: The specific method of the spatiotemporal cross attention mechanism is as follows: The output of the feature extraction block The cross attention calculation is performed along the spatial axis with the output H' of the embedding layer in order to pre-learn the temporal features before extracting the spatial features. The learning process is as follows: Among them, n represents the number of spatiotemporal cross attention layers. Only when n is 1, The value of is H'; CrossAttention represents spatiotemporal cross attention.
7. The traffic flow prediction method based on traffic flow spatiotemporal data information according to claim 6, characterized in that: The specific method of step S5 is as follows: The output of the fusion of spatiotemporal attention Input to the spatial self-attention layer, continue to perform self-attention calculation along the spatial dimension, and obtain the output result Z' of the spatial self-attention layer that integrates the spatiotemporal features. The process is shown as follows: Among them, SelfAttention represents spatial self-attention.
8. The traffic flow prediction method based on traffic flow spatiotemporal data information according to claim 7, characterized in that: The specific method of step S6 is as follows: The output of the spatial self-attention layer Generate prediction values through the regression layer; the detailed process can be expressed as: in, is the predicted value, T' is the predicted time span, d tar is the target dimension, which is fixed to 1; therefore, the regression layer maps the tensor dimension from T×D in Z' to T'×(d tar =1).
9. The traffic flow prediction method based on spatiotemporal data information of traffic flow according to claim 8, characterized in that: The specific method of step S7 is as follows: The data in the training set is used to train the parameters in the overall model; The loss function selected is the mean absolute error (MAE); the number of training rounds is set to 100, Adam is selected as the optimizer, and the learning rate decays from 0.001.