Adaptive traffic flow prediction method and system based on Transform mechanism
Through the adaptive traffic flow prediction method based on the Transformer mechanism, combined with adaptive graph convolution and Chebyshev graph convolution layer, the spatiotemporal characteristics of the traffic network are dynamically captured, which solves the problem of insufficient spatial and time dependence in traffic flow prediction, and improves the prediction accuracy and generalization ability of the model.
Patent Information
- Application Number
- CN202510528943.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-25
- Publication Date
- 2025-07-11
AI Technical Summary
Existing deep learning models are difficult to dynamically capture spatial dependence and global time dependence in traffic flow prediction, resulting in insufficient prediction accuracy and generalization capabilities.
Adaptive traffic flow prediction method based on Transformer mechanism is adopted, combining adaptive graph convolution and Chebyshev graph convolution layer, and dynamically capture the relationship between nodes through the adaptive graph learning layer, and combining Transformer spatiotemporal convolution layer to extract local and global spatiotemporal features.
It realizes accurate modeling of the dynamic characteristics of the traffic network, improves prediction accuracy and generalization capabilities of the model, and breaks through the limitations of insufficient long-range dependency modeling of traditional methods.
Smart Images
Figure CN120299249A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of deep learning, and particularly relates to an adaptive traffic flow prediction method and system based on the Transformer mechanism. Background Art
[0002] Traffic flow prediction is the core supporting technology for realizing a modern and intelligent transportation system. In recent years, with the development of deep learning and graph neural networks, it has received more extensive research attention. The latest work usually focuses on the modeling ability of deep learning methods for spatio-temporal characteristics. Representative methods include spatio-temporal graph convolutional networks (STGCN), graph attention networks (GAT), and hybrid models that combine convolutional and attention mechanisms. These models have shown superiority in capturing complex spatial and temporal dependencies, but still face challenges in dealing with heterogeneous data, emergencies, and improving computational efficiency.
[0003] Deep learning models have demonstrated excellent capabilities in mining the potential feature representations of data. STGCN combines the advantages of convolutional neural networks and graph convolutional networks, extracts dependencies in the time dimension through temporal convolution, and simultaneously captures the spatial relationships between nodes in the traffic network using graph convolution. GAT introduces a self-attention mechanism to dynamically learn the relationship weights between nodes in the graph. By assigning different attention weights to neighbor nodes, it enhances the modeling ability for important neighbors.
[0004] The models of the above methods have achieved great success, but there are certain defects in modeling spatio-temporal dependencies. Both STGCN and GAT use static graphs based on spatial information and are difficult to dynamically capture spatial dependencies. In addition, both are still limited in the ability to capture global temporal dependencies, and these deficiencies limit the performance of the models in traffic flow prediction tasks. Summary of the Invention
[0005] The present invention provides an adaptive traffic flow prediction method and system based on the Transformer mechanism to solve the technical problems mentioned in the background art.
[0006] An adaptive traffic flow prediction method based on the Transformer mechanism, the method comprising:
[0007] S1. Data collection;
[0008] S2. Preprocessing the collected data;
[0009] S3. Establishing and training a graph convolutional neural network model based on Transformer and adaptive graph convolution;
[0010] S4. Predicting the traffic flow.
[0011] As a further technical solution of the present invention, the step S1 includes the following sub-steps:
[0012] S11. Set the time interval for data collection;
[0013] S12. Collect traffic flow data at the set time interval;
[0014] S13. Aggregate the collected data.
[0015] As a further technical solution of the present invention, the step S2 includes the following sub-steps:
[0016] S21. Fill in the missing values of the collected traffic flow data;
[0017] S22. Perform Z-Score standardization on the traffic flow data;
[0018] S23. Set the size of the sliding window, including the sliding step K, the length history_len of the input historical sequence, and the length pred_len of the output predicted time series;
[0019] S24. Divide the processed data into a training set, a validation set, and a test set according to the ratio of 14:3:3.
[0020] As a further technical solution of the present invention, the graph convolutional neural network model based on Transformer and adaptive graph convolution in the step S3 is mainly divided into the following modules: an adaptive graph learning layer, a spatio-temporal convolutional layer based on Transformer, a Chebyshev graph convolutional layer, and an output layer.
[0021] As a further technical solution of the present invention, the step S3 includes the following sub-steps:
[0022] S31. Take the samples of the training set and obtain the corresponding batch of dynamic graph G through the adaptive graph learning layer;
[0023] S32. Take the same training set samples and convert the input X into a representation Xenhanced_1 that strengthens the global time dependence through the spatio-temporal convolutional layer based on Transformer;
[0024] S33. Take the obtained dynamic graph G and Xenhanced_1 as the inputs of the Chebyshev graph convolutional layer, perform spatio-temporal feature fusion, and obtain Xenhanced_2;
[0025] S34. Take Xenhanced_2 as the input and repeat S32;
[0026] S35. Use the output obtained in S34 as the input for this step, and repeat S32 - S34;
[0027] S36. Input the output of the above steps into the spatio - temporal convolutional layer based on Transformer again, and finally obtain the predicted value Y through the fully - connected module;
[0028] S37. Calculate the difference between the predicted value and the true value and train. Use the validation set to observe whether the model reaches the best effect and prevent overfitting.
[0029] As a further technical solution of the present invention, the input X in step S32 is expressed as: X = {X1, X2, X3,...., Xhistory_len}; the output predicted value Xpred in step S36 is expressed as: Y = {Y1, Y2, Y3,...., Ypred_len}.
[0030] As a further technical solution of the present invention, step S37 includes the following sub - steps: When the model is not overfitting, repeat steps S31 - S36; when the model is overfitting or the number of repeated executions reaches the threshold, end the training and save the current model.
[0031] Another object of the present invention is to provide an adaptive traffic flow prediction system based on the Transformer mechanism. The system includes a traffic flow data acquisition module, a data pre - processing module, a model establishment module, a traffic flow prediction module, and a result display module; the traffic flow data acquisition module is used to collect traffic flow data in real - time; the data pre - processing module is used to pre - process the collected traffic flow data; the model establishment module is used to establish and train a graph convolutional neural network model based on Transformer and adaptive graph convolution; the traffic flow prediction module is used to predict and analyze the traffic flow situation according to the pre - processed data; the result display module is used to output the prediction result and send it to the user.
[0032] The beneficial effects achieved by the present invention:
[0033] By applying a spatio-temporal graph convolutional model that combines fusion adaptive graph learning and the Transformer mechanism to traffic flow prediction, it has significant innovative points compared with previous methods: traditional models mostly rely on static adjacency matrices and cannot adapt to the dynamic characteristics of traffic networks that change over time. In this method, a multi-head weight matrix is introduced in the adaptive graph learning layer to dynamically capture the complex relationships between nodes from multiple feature perspectives, achieving precise modeling of time-varying spatial dependencies dynamically and continuously. In addition, in the Transformer spatio-temporal convolutional layer, the simple sequence modeling method is abandoned, and two-dimensional convolution is combined with the Chebyshev graph convolutional layer to deeply explore spatial correlations while extracting local temporal features. This deep fusion of convolution and the Transformer mechanism enables the model to capture both local and global spatio-temporal features, effectively breaking through the limitation of traditional methods in insufficient long-range dependence modeling and greatly improving the prediction accuracy and generalization ability of the model. Description of the Drawings
[0034] Figure 1 It is a flowchart of an adaptive traffic flow prediction method based on the Transformer mechanism.
[0035] Figure 2 It is a schematic diagram of the model framework provided by an embodiment of the present invention. Detailed Embodiments
[0036] The technical solutions of the present invention will be described in detail below with reference to specific drawings.
[0037] Please refer to Figure 1 , an embodiment of the present invention provides an adaptive traffic flow prediction method based on the Transformer mechanism, and the method includes:
[0038] S1. Data collection;
[0039] S2. Preprocess the collected data;
[0040] S3. Establish and train a graph convolutional neural network model based on Transformer and adaptive graph convolution;
[0041] S4. Predict the traffic flow.
[0042] In this embodiment, the S1 step includes the following sub-steps:
[0043] S11. Set the time interval for data collection;
[0044] S12. Collect traffic flow data at the set time interval;
[0045] S13. Aggregate the collected data.
[0046] In this embodiment, the step S2 includes the following sub-steps:
[0047] S21. Fill in the missing values of the collected traffic flow data;
[0048] S22. Perform Z-Score standardization on the traffic flow data;
[0049] S23. Set the size of the sliding window, including the sliding step K, the length history_len of the input historical sequence, and the length pred_len of the output predicted time series;
[0050] S24. Divide the processed data into a training set, a validation set, and a test set according to the ratio of 14:3:3.
[0051] In this embodiment, the graph convolutional neural network model based on Transformer and adaptive graph convolution in the step S3 is mainly divided into the following modules: an adaptive graph learning layer, a spatio-temporal convolutional layer based on Transformer, a Chebyshev graph convolutional layer, and an output layer.
[0052] In this embodiment, the step S3 includes the following sub-steps:
[0053] S31. Take the samples of the training set and obtain the corresponding batch of dynamic graph G through the adaptive graph learning layer;
[0054] S32. Take the same training set samples and convert the input X into a representation Xenhanced_1 that strengthens the global time dependence through the spatio-temporal convolutional layer based on Transformer;
[0055] S33. Take the obtained dynamic graph G and Xenhanced_1 as the input of the Chebyshev graph convolutional layer, perform spatio-temporal feature fusion, and obtain Xenhanced_2;
[0056] S34. Take Xenhanced_2 as the input and repeat S32;
[0057] S35. Take the output obtained in S34 as the input of this step and repeat S32 - S34;
[0058] S36. Input the output of the above steps into the spatio-temporal convolutional layer based on Transformer again, and finally obtain the predicted value Y through the fully connected module;
[0059] S37. Calculate the difference between the predicted value and the true value and train, and use the validation set to observe whether the model reaches the best effect to prevent overfitting.
[0060] In this embodiment, the input X in step S32 is represented as: X = {X1, X2, X3,...., Xhistory_len}; the output prediction value Xpred in step S36 is represented as: Y = {Y1, Y2, Y3,...., Ypred_len}.
[0061] In this embodiment, step S37 includes the following sub-steps: when the model is not overfitting, repeat steps S31 - S36; when the model is overfitting or the number of repetitions reaches the threshold, end the training and save the current model.
[0062] Another object of the present invention is to provide an adaptive traffic flow prediction system based on the Transformer mechanism. The system includes a traffic flow data collection module, a data preprocessing module, a model establishment module, a traffic flow prediction module, and a result display module; the traffic flow data collection module is used to collect traffic flow data in real time; the data preprocessing module is used to preprocess the collected traffic flow data; the model establishment module is used to establish and train a graph convolutional neural network model based on Transformer and adaptive graph convolution; the traffic flow prediction module is used to predict and analyze the traffic flow situation according to the preprocessed data; the result display module is used to output the prediction result and send it to the user.
[0063] To facilitate those skilled in the art to better understand the technical solution of the present invention, the following specific embodiments of the present invention are given:
[0064] An adaptive traffic flow prediction method based on the Transformer mechanism includes the following steps:
[0065] Step 1, data collection: Collect traffic flow data in real time through a traffic measurement system.
[0066] Using a traffic performance measurement system, obtain traffic flow data every 5 minutes, and aggregate the collected data at 60 - minute time intervals.
[0067] Step 2, data preprocessing: Standardize the collected traffic flow data and divide it into training, validation, and test sets.
[0068] First, use the z - score standardization method to normalize the traffic flow data. Its standardization formula is: . Where x is the original traffic flow data, is the average value of the samples, is the standard deviation of the samples.
[0069] Subsequently, the standardized data is processed using the sliding window method. Set the input sequence length (seq_len), prediction distance M, and prediction sequence length (pred_len) according to the total time steps of the data. Use the true data of length pred_len after M time steps of the window as the label, and set the window step size to 1.
[0070] The preprocessed data is divided into a training set, a validation set, and a test set in a ratio of 14:3:3 for model training, parameter adjustment, and performance evaluation respectively.
[0071] Step 3: Model training. The graph convolutional neural network model based on Transformer and adaptive graph convolution of the present invention includes an attention layer, a two-dimensional convolutional layer, a spatio-temporal convolutional layer based on Transformer, a ChebNet graph convolutional layer, a position encoding layer, an adaptive graph learning layer, a multi-head self-attention layer, and a fully connected layer. Input the data into this traffic flow prediction model and train it.
[0072] Take the training samples. Denote the samples input into the model as X = {X1, X2, X3,...., Xseq_len}. By inputting X into the adaptive graph learning layer. First, transpose the input historical traffic flow data X and introduce a weight matrix Perform multi-view feature learning, where P is the number of views, N is the number of traffic nodes to be learned, and H is the historical sequence length, so that each view ppp generates a weighted feature matrix, that is, the formula: , where W is the weight matrix. Subsequently, we calculate the node attention matrix under each view and obtain the global attention matrix A by averaging and fusing: . Next, set a threshold for A for screening and construct the final adaptive adjacency matrix S: , where m is the set default value, is the learnable relationship matrix. During the training process, we perform average pooling on S within the batch to reduce complexity and improve generalization ability. Finally, the weight matrix W is jointly optimized during the training of the Transforemr spatio-temporal convolutional layer and the output layer, enabling the model to dynamically generate an adjacency matrix adapted to different traffic environments.
[0073] Take the same training samples as the adaptive graph learning layer, that is, the model input samples X = {X1, X2, X3,...., Xseq_len}. First, the input data X undergoes multi-channel feature expansion to enhance adaptability to spatio-temporal dynamics, and its input form: . where C_in is the number of input channels and H_in is the historical time step. Then, the input data is subjected to feature extraction through a 2D convolutional layer (Conv2D): ; Subsequently, positional encoding (PE) is added to capture the sequential information of time steps. The positional information is the relative position of the input sample in the input sample sequence, and its formula is: Subsequently, using the Transformer mechanism, the features are mapped to query (Q), key (K), and value (V) vectors, and self-attention is calculated: . Thus, we can summarize the above steps into a formula: . Then, using the relationship matrix S obtained by the adaptive graph learner, the features processed by the Transformer are input into the ChebNet graph convolutional layer for spatial feature learning: . Among them, is the Laplacian matrix. Wk learns the contributions of different orders of Chebyshev polynomials through training. T k represents the Chebyshev polynomial to enhance the modeling ability of spatial dependence. Finally, the features output by ChebNet are processed again by the Transformer mechanism. The processing method here is the same as that of the above Transformer spatio-temporal convolutional layer. The final output of the T-ST module is obtained: , where C our , T OUT represent the output channel dimension and sequence length respectively.
[0074] As Figure 2 , finally, repeat the above Transformer spatio-temporal convolutional layer and adaptive graph learning layer in sequence, and then pass through the output layer, as Figure 2 shown. The final output can be regarded as the prediction result of the model.
[0075] Train the model, compare the prediction result with the true label to train the model, use the validation set data for validation, judge whether the model is overfitting, and repeat the above steps until the specified number of rounds is reached or it is found that the model is overfitting and stop training, and save the current model as the best model.
[0076] Step 4: Traffic flow prediction. Input the traffic flow data to be tested into the trained model for prediction of subsequent time steps.
[0077] The present invention proposes an adaptive traffic flow prediction method based on the Transformer mechanism. The model uses the Transformer spatio-temporal convolutional layer mechanism to capture the long-term time dependence in traffic flow data and uses the adaptive graph learning layer to dynamically obtain spatial dependence. It fully mines the spatio-temporal features of the data and improves the performance of the model.
[0078] It should be noted that in this text, the term "including" or any other variant thereof is intended to cover non-exclusive inclusion, such that a process, method, article, or device that includes a series of elements not only includes those elements but also includes other elements not explicitly listed, or further includes elements inherent to such process, method, article, or device. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of additional identical elements in the process, method, article, or device that includes such element.
[0079] The above are only the preferred embodiments of the present invention, and do not limit the patent scope of the present invention accordingly. Any equivalent structure or equivalent process transformation made by using the content of the specification and drawings of the present invention, or directly or indirectly applied in other related technical fields, shall equally be included in the patent protection scope of the present invention.
Claims
1. An adaptive traffic flow prediction method based on the Transformer mechanism, characterized in that The method includes the following: S1. Data collection; S2. Preprocess the collected data; S3. Establish and train a graph convolutional neural network model based on Transformer and adaptive graph convolution; S4. Predict the traffic flow.
2. The adaptive traffic flow prediction method and system based on the Transformer mechanism according to claim 1, characterized in that The S1 step includes the following sub-steps: S11. Set the time interval for data collection; S12. Collect traffic flow data according to the set time interval; S13. Aggregate the collected data.
3. An adaptive traffic flow prediction method based on the Transformer mechanism according to claim 1, characterized in that, The S2 step includes the following sub-steps: S21. Fill in the missing values of the collected traffic flow data; S22. Perform Z-Score standardization on the traffic flow data; S23. Set the size of the sliding window, including the sliding step K, the length of the input historical sequence history_len, and the length of the output prediction time series pred_len; S24. Divide the processed data into a training set, a validation set, and a test set according to the ratio of 14:3:
3.
4. An adaptive traffic flow prediction method based on the Transformer mechanism according to claim 1, wherein, In the S3 step, the graph convolutional neural network model based on Transformer and adaptive graph convolution is mainly divided into the following modules: an adaptive graph learning layer, a spatio-temporal convolutional layer based on Transformer, a Chebyshev graph convolutional layer, and an output layer.
5. The adaptive traffic flow prediction method based on the Transformer mechanism according to claim 4, characterized in that, The S3 step includes the following sub-steps: S31. Take the samples of the training set and obtain the corresponding batch of dynamic graph G through the adaptive graph learning layer; S32. Take the same training set samples and transform the input X into a representation Xenhanced_1 that strengthens the global time dependence through the spatio-temporal convolutional layer based on Transformer; S33. Use the obtained dynamic graph G and Xenhanced_1 as the input of the Chebyshev graph convolutional layer to perform spatio-temporal feature fusion and obtain Xenhanced_2; S34. Use Xenhanced_2 as the input and repeat S32; S35. Use the output obtained in S34 as the input of this step and repeat S32 - S34; S36. Input the output of the above steps into the spatio-temporal convolutional layer based on Transformer again, and finally obtain the predicted value Y through the fully connected module; S37. Calculate the difference between the predicted value and the true value and train, and use the validation set to observe whether the model reaches the best effect to prevent overfitting.
6. The adaptive traffic flow prediction method based on the Transformer mechanism according to claim 5, characterized in that The input X in the S32 step is expressed as: X = {X1, X2, X3,...., Xhistory_len}; the output predicted value Xpred in the S36 step is expressed as: Y = {Y1, Y2, Y3,...., Ypred_len}.
7. An adaptive traffic flow prediction method based on the Transformer mechanism according to claim 5, characterized in that, The S37 step includes the following sub-steps: When the model is not overfitting, repeat steps S31 - S36; when the model is overfitting or the number of repeated executions reaches the threshold, end the training and save the current model.
8. An adaptive traffic flow prediction system based on the Transformer mechanism, characterized in that, The system includes a traffic flow data collection module, a data preprocessing module, a model establishment module, a traffic flow prediction module, and a result display module; the traffic flow data collection module is used to collect traffic flow data in real time; the data preprocessing module is used to preprocess the collected traffic flow data; the model establishment module is used to establish and train a graph convolutional neural network model based on Transformer and adaptive graph convolution; the traffic flow prediction module is used to predict and analyze the traffic flow situation according to the preprocessed data; the result display module is used to output the prediction results and send them to the user.