Traffic flow time series prediction method and system based on transformer
By introducing a wide local perception module and a heterogeneous asymmetric parallel architecture, combined with reversible instance normalization and dynamic gating mechanisms, the modeling instability problem of the Transformer model in traffic flow prediction is solved, and high-precision and robust prediction of traffic flow data is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- ZHEJIANG SCI-TECH UNIV
- Filing Date
- 2026-04-22
- Publication Date
- 2026-07-31
AI Technical Summary
Existing Transformer models struggle to simultaneously consider causal logic and a global perspective when processing traffic flow data. This leads to instability in modeling short-term fluctuations and long-term periodic patterns, and also results in distribution drift, affecting the model's generalization ability.
By introducing a Wide Local Perception Block and a heterogeneous asymmetric parallel architecture, combined with reversible instance normalization and layer-by-layer downsampling mechanisms, and by coupling causal constraint paths with dynamically gated global paths, the model's ability to model complex traffic conditions is enhanced.
It improves the model's prediction accuracy and temporal stability for multi-dimensional data such as traffic flow, road segment speed, and occupancy. It can accurately capture local causal details and adaptively filter global key features, thereby enhancing the model's prediction stability and anti-interference ability in extreme non-stationary environments.
Smart Images
Figure CN122493671A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of intelligent transportation systems and deep learning processing technology for time series data, specifically relating to a method and system for predicting traffic flow time series based on Transformer. Background Technology
[0002] Traffic flow prediction is a core application of intelligent transportation systems, significantly contributing to alleviating urban traffic congestion, optimizing traffic light control, improving logistics efficiency, and ensuring travel safety. Traditional traffic prediction methods typically rely on shallow statistical learning methods such as historical averaging and autoregressive models, or simple physical modeling, which struggle to fully exploit the complex nonlinear spatial correlations and temporal dependencies within high-dimensional traffic sensor data. In recent years, time series modeling methods based on deep learning, particularly Transformer models, have demonstrated immense potential in traffic prediction; however, existing technologies still face significant challenges in practical applications.
[0003] Traffic flow sequences exhibit strong non-stationarity, displaying complex short-term fluctuations and long-term periodic patterns. For example, sudden reductions in traffic flow due to traffic accidents, temporary traffic control, or extreme weather represent localized, sudden fluctuations, while the alternation of morning and evening rush hours, and weekdays and holidays, represent long-term structured trends. Existing Transformer models often struggle to simultaneously consider causal logic and a global perspective when processing such data. On one hand, standard attention mechanisms, when modeling long-term dependencies, tend to overlook local causal evolution of traffic flow over short timescales, leading to spurious regressions in predictions. On the other hand, while simple block-based techniques can improve computational efficiency, they still suffer from modeling instability when capturing the dynamic interactions between different traffic variables (such as flow rate, speed, and occupancy).
[0004] Furthermore, the distribution drift of traffic flow data also limits the generalization ability of the model. Due to the differences in geographical environment of different monitored road sections and the changes in traffic patterns at different time periods, the mean and variance of the original series often fluctuate dramatically. If these non-stationary features cannot be effectively handled, the model is prone to serious bias accumulation in long-term prediction tasks. Therefore, how to design a traffic flow prediction architecture that can accurately capture local causal details, adaptively select global key features, and possess strong robustness has become a key problem that urgently needs to be solved in the field of intelligent transportation. Based on this, this invention proposes a Transformer-based traffic flow time series prediction method and system. Summary of the Invention
[0005] To address the aforementioned issues, this invention provides a Transformer-based method and system for predicting traffic flow time series, aiming to improve the model's prediction accuracy and temporal stability for multi-dimensional traffic data such as traffic flow, road segment speed, and occupancy. This invention introduces a heterogeneous asymmetric parallel architecture guided by a Wide Local Perception Block (WLP). During the feature extraction stage, it leverages the coupling of causal constraint paths and dynamically gated global paths to enhance the model's ability to model complex traffic conditions. Simultaneously, it combines reversible instance normalization and layer-by-layer downsampling mechanisms to ensure the model's generalization and robustness in long-term prediction tasks.
[0006] The present invention adopts the following technical solution:
[0007] The Transformer-based traffic flow time series prediction method includes the following steps:
[0008] S1. Perform reversible instance normalization on the input traffic flow time series to obtain a standardized sequence. In this step, reversible instance normalization (RevIN) is performed on the input traffic flow time series. The mean and variance of each variable are recorded to obtain a standardized sequence that removes distribution bias. This reduces mean-variance drift between different road segments and time periods and enhances the model's adaptability to dynamic traffic environments.
[0009] S2, Feature Embedding Mapping: The standardized sequence is directly linearly mapped through the linear embedding layer; the original dimensional features are projected into a high-dimensional feature space, and time position encoding and channel position encoding are superimposed to form the initial feature embedding vector;
[0010] S3. Heterogeneous Parallel Composite Unit Feature Extraction: L composite units are stacked sequentially on the feature embedding vector sequence; each composite unit is processed strictly in the order of WLP (Wide Local Perception Block), asymmetric dual-path parallel module, path fusion module, and downsampling.
[0011] S4. Project the top-level features (referring to the features output by the Lth (last) composite unit) into a prediction step size H×C dimensional vector, and then perform inverse normalization to restore the original dimensions to obtain the prediction model; where H represents the length of the time series (which can also be understood as the prediction step size), that is, the number of time steps the model needs to predict, and C represents the number of different traffic variables that need to be predicted at the same time.
[0012] S5. Divide the dataset (data sourced from a publicly available benchmark dataset, preferably the Traffic dataset) proportionally to obtain a training set, validation set, and test set; optimize the parameters of the prediction model from step S4 using an optimizer (preferably the Adam-W optimizer); input the prediction model into the training set for model training to obtain predicted values, compare the predicted values with the true values, and calculate the loss (the training set loss is the model's prediction error on the training set, used to update parameters. The optimizer adjusts the model's internal weights based on the gradient generated by this loss); after each training epoch, calculate the loss on the validation set (the validation set loss is the model's prediction error on the validation set (data the model has not seen before), used to judge model performance. It does not directly participate in parameter updates); if the iteration stopping condition is met, stop the iteration and save the optimal model;
[0013] S6. Use the trained optimal model to make predictions on the test set. Further, compare the predicted results with the actual results using error evaluation metrics.
[0014] As a preferred approach, in step S1, reversible instance normalization (RevIN) is performed on the input traffic flow time series to eliminate the interference of non-stationary statistical features. The calculation formula is as follows: ,in, Input a tensor into the history. For the backtracking window length, The number of variables.
[0015] As a preferred approach, in step S2, the standardized sequence is mapped to a high-dimensional latent space through an embedding layer. .
[0016] As a preferred embodiment, step S3 specifically includes the following processing steps within the composite unit:
[0017] S3.1 The Wide Local Perception Module (WLP) uses dilated convolution and layer normalization to smooth local noise and extract context. The calculation formula is as follows: When in deep networks ( When this occurs, features are reinforced through a cross-layer feedback mechanism, at which point the features enter the dual-path system. for: .
[0018] S3.2. An asymmetric dual-path parallel module is used for processing. The causal constraint path (Local Path) sequentially performs layer normalization (LayerNorm) and autoregressive self-attention. This path utilizes an autoregressive mechanism to model short-range dependencies and introduces a causal mask. (when hour (Otherwise 0) Ensure no future information leakage, where M ij It is a causal mask matrix The specific values in the i-th row and j-th column determine whether the model is allowed to see information from other times when calculating information at the current time. Local path output. for: ,in , , All are obtained through linear projection; the Global Path sequentially executes a Dynamically Gated Feedforward Network (FFN) and a Non-causal self-attention network. It utilizes the relative positions... Generated shape factor Dynamically adjust the nonlinear slope of the gated activation function. Intermediate variable. ,in For learnable parameters, This represents the internal projection dimension of the generator network. To ensure numerical stability and control the effective propagation of gradients, we will... Mapping to bounded intervals To obtain the final , To prevent division by zero of small constants. Output The calculation formula is as follows: , These are the projection matrices for the gated branch and the value branch, respectively. To output the projection matrix, For model dimensions, This is an extension dimension for the feedforward network. Subsequently, As input, it enters the non-causal self-attention module, capturing long-range periodic dependency patterns through a global perspective, and its output... The calculation formula is as follows: ,in, respectively by It is obtained through linear projection. Since this path does not have mask constraints, the model can process all sequence features in parallel to extract global spatiotemporal evolution patterns.
[0019] S3.3 Output the causal constraint path from step S3.2. With global dynamic path output The input path fusion module (Path Fusion) uses the learned "soft routing" mechanism to generate a point-to-point fusion weight matrix. : ,in To fuse the projection matrix, This indicates a splicing operation. The final output is the composite unit feature. .
[0020] S3.4. Heterogeneous parallel composite units are cascaded through hierarchical downsampling. Layer input The output of the previous layer is obtained by downsampling: . This represents the intermediate feature state between the path fusion module and the downsampling module in the (i-1)th layer composite unit; it is the "stage output" of the previous layer, and only after being processed by the downsampling function does it officially become the input of the next layer. .
[0021] As a preferred approach, in step S4, the top-level features are projected and mapped into a prediction step size H×C dimensional vector. The specific process is as follows:
[0022] S4.1 Following the channel independence paradigm, the output characteristics obtained after processing by N composite units are... Flattened into a vector in the time and channel dimensions ,in, Let be the projection matrix. This is a bias term.
[0023] S4.2 Perform inverse normalization, using the mean recorded in step S1. and standard deviation Inverse normalization is used to restore the original physical dimensions of the traffic data, resulting in the final prediction: .
[0024] As a preferred option, in step S5, the ratio of the training set, validation set, and test set is 7:1:2. If there is no improvement or deterioration after k consecutive validations, early stopping is triggered, and the optimal weights are saved.
[0025] As a preferred option, in step S6, the error evaluation indexes are the mean absolute error (MAE) and the mean squared error (MSE).
[0026] This invention also discloses a Transformer-based traffic flow time series prediction system for performing the above method, comprising the following modules:
[0027] Normalization module: Performs instance normalization on the input traffic flow time series to obtain a standardized series;
[0028] Mapping module: The normalized sequence is linearly mapped through the linear embedding layer, the original dimensional features are projected into the high-dimensional feature space, and the temporal position encoding and channel position encoding are superimposed to form the initial feature embedding vector sequence;
[0029] Heterogeneous parallel processing module: L heterogeneous parallel composite units are stacked sequentially on the feature embedding vector sequence. Each composite unit is processed sequentially through a wide local perception module, an asymmetric dual-path parallel module, a path fusion module, and downsampling. The asymmetric dual-path parallel module includes a causal constraint path composed of autoregressive self-attention and a global dynamic path composed of a dynamic gated feedforward network and non-causal self-attention.
[0030] Prediction model acquisition module: The top-level features are projected and mapped into H×C-dimensional vectors through a fully connected layer to establish a mapping relationship between the feature space and the prediction target; then inverse normalization is performed to obtain the prediction model;
[0031] Dataset partitioning and iteration module: The Traffic dataset is partitioned proportionally to obtain training, validation, and test sets; the Adam-W optimizer is used to optimize the parameters of the prediction model; the prediction model is trained by inputting the training set to obtain predicted values, and the predicted values are compared with the true values to calculate the loss; after each training round, the loss is calculated on the validation set; if the iteration stopping condition is met, the iteration is stopped and the optimal model is saved.
[0032] Prediction module: The trained optimal model makes predictions on the test set; further, the prediction results are compared with the actual results through error evaluation metrics.
[0033] Compared with the prior art, the present invention has the following beneficial effects:
[0034] (1) This invention effectively solves the "over-smoothing" problem in time series modeling by introducing a wide local perception module and heterogeneous asymmetric parallel paths. This invention utilizes the autoregressive self-attention of causal constraint paths in conjunction with causal masks to accurately capture the local causal logic of traffic flow evolution over time. Compared with traditional pure attention models, this invention can effectively track high-frequency instantaneous changes in traffic flow, avoiding the smoothing of key local patterns by global trends, thereby improving the perception accuracy of the model under complex traffic conditions.
[0035] (2) This invention designs a dynamic gated feedforward network, realizing time-sensitive information cleanup and feature adaptive routing. This invention introduces a shape factor dynamically generated based on relative position, which can adaptively adjust the nonlinear slope of the gated activation function according to the input context. This design enables the model to automatically shield high-frequency noise under long prediction step lengths, and while ensuring global trend modeling, it significantly enhances the prediction stability and anti-interference ability of the model in extreme non-stationary environments.
[0036] (3) This invention introduces a dynamic path fusion mechanism, achieving efficient coupling between local detailed features and global dependency patterns. This invention utilizes the learned weight matrix to dynamically quantify the importance of different path features at each time step and channel. Compared with the static weighted fusion of existing technologies, this invention enables the model to spontaneously adjust its focus based on whether the current traffic sequence is "trend-driven" or "fluctuation-driven," significantly improving the model's ability to generalize and represent diverse and complex traffic patterns. Attached Figure Description
[0037] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0038] Figure 1 This is a flowchart of a preferred embodiment of the present invention for a traffic flow time series prediction method based on Transformer. The dashed box represents the multi-layer stacked part of the model.
[0039] Figure 2 This is a structural diagram of the Wide Local Perception Module (WLP) proposed in a preferred embodiment of the present invention. In the diagram, DilatedConvolution refers to dilated convolution, and LayerNorm refers to layer normalization.
[0040] Figure 3 This is a structural diagram of the Autoregressive Self-Attention module and the Non-causal Self-Attention module in a preferred embodiment of the present invention; in the diagram, Q, K, V (query, key, value) are well-known mathematical representations in the field of deep learning;
[0041] Figure 4 This is a structural diagram of the Dynamic Gated FFN in a preferred embodiment of the present invention. In the diagram, Dynamic Temporal Sigmoid is a dynamic temporal gated activation function. It is element-wise multiplication; Linear refers to a linear layer.
[0042] Figure 5This is a structural diagram of the network model processing procedure in a preferred embodiment of the present invention. Input / Output refers to input / output, RevIN (Norm / De-norm) refers to reversible instance normalization (normalization / de-normalization), Embedding refers to the embedding layer, Path Fusion refers to path fusion, Down Sampling refers to downsampling, and Projection refers to the projection layer.
[0043] Figure 6 This is a block diagram of a Transformer-based traffic flow time series prediction system according to a preferred embodiment of the present invention. Detailed Implementation
[0044] The following specific embodiments illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that, unless otherwise specified, the following embodiments and features described therein can be combined with each other.
[0045] like Figure 1 As shown in the figure, this embodiment of a traffic flow time series prediction method based on Transformer specifically includes the following steps:
[0046] S1. Perform Reversible Instance Normalization (RevIN) on the input traffic flow time series, record the mean and variance of each variable, and obtain a standardized sequence to remove distribution bias, so as to reduce mean-variance drift between different road segments and different time periods and enhance the model's adaptability to dynamic traffic environment.
[0047] Specifically, in step S1, reversible instance normalization (RevIN) is performed on the input traffic flow time series to eliminate the interference of non-stationary statistical features. The calculation formula is as follows: ,in, Input a tensor into the history. For the backtracking window length, The number of variables.
[0048] S2. Feature embedding mapping: This method directly maps the normalized sequence to a high-dimensional latent space through a linear embedding layer. The original dimensional features are projected into a high-dimensional feature space to form an initial feature embedding vector sequence.
[0049] S3. Heterogeneous parallel composite unit feature extraction: L composite units are stacked sequentially on the feature embedding vector sequence; each composite unit strictly follows the WLP (Wide Local Perception Block) module, an asymmetric dual-path parallel module, and a path fusion module (the asymmetric dual-path parallel module consists of, for example,...). Figure 5 The path fusion module consists of two parallel paths: the causal constraint path (upper path) and the global dynamic path (lower path); Figure 5 The overall architecture is reflected in the adaptive coupling operation of the output features of two parallel paths, which is physically manifested as the confluence connection of the causal constraint path (upper path) and the global dynamic path (lower path) and the sequential processing of downsampling.
[0050] In step S3 of this embodiment, the composite unit specifically includes the following processing steps:
[0051] S3.1 The Wide Local Perception Module (WLP) uses dilated convolution and layer normalization to smooth local noise and extract context. The calculation formula is as follows: When in deep networks ( When this occurs, features are reinforced through a cross-layer feedback mechanism, at which point the features enter the dual-path system. for: .
[0052] S3.2. An asymmetric dual-path parallel module is used for processing. The causal constraint path (Local Path) sequentially performs layer normalization (LayerNorm) and autoregressive self-attention. This path utilizes an autoregressive mechanism to model short-range dependencies and introduces a causal mask. (when hour (Otherwise, the value is 0) Ensure no future information leakage; local path output. for: ,in , , All are obtained through linear projection; the Global Path sequentially executes a Dynamically Gated Feedforward Network (Dynamic GatedFFN) and a Non-causal self-attention network. It utilizes the relative positions... Generated shape factor Dynamically adjust the nonlinear slope of the gated activation function. Intermediate variable. ,in For learnable parameters, This represents the internal projection dimension of the generator network. To ensure numerical stability and control the effective propagation of gradients, we will... Mapping to bounded intervals To obtain the final , To prevent division by zero of small constants. Output The calculation formula is as follows: , These are the projection matrices for the gated branch and the value branch, respectively. To output the projection matrix, For model dimensions, This is an extension dimension for the feedforward network. Subsequently, As input, it enters the non-causal self-attention module, capturing long-range periodic dependency patterns through a global perspective, and its output... The calculation formula is as follows: ,in, respectively by It is obtained through linear projection. Since this path does not have mask constraints, the model can process all sequence features in parallel to extract global spatiotemporal evolution patterns.
[0053] S3.3 Output the causal constraint path from step S3.2. With global dynamic path output The input path fusion module (Path Fusion) uses the learned "soft routing" mechanism to generate a point-to-point fusion weight matrix. : ,in, To fuse the projection matrix, This indicates a splicing operation. The final output is the composite unit feature. .
[0054] S3.4. Heterogeneous parallel composite units are cascaded through hierarchical downsampling. Layer input The output of the previous layer is obtained by downsampling: .
[0055] S4. Project the top-level features into a prediction step size H×C dimensional vector, then perform inverse normalization to restore the original dimensions and obtain the prediction model; H represents the length of the time series (which can also be understood as the prediction step size), that is, the number of time steps the model needs to predict, and C represents the number of different traffic variables that need to be predicted at the same time.
[0056] Step S4 in this embodiment specifically includes the following steps:
[0057] S4.1 Following the channel independence paradigm, the output characteristics obtained after processing by N composite units are... Flattened into a vector in the time and channel dimensions ,in Let be the projection matrix. This is a bias term.
[0058] S4.2 Perform inverse normalization, using the mean recorded in step S1. and standard deviation Inverse normalization is used to restore the original physical dimensions of the traffic data, resulting in the final prediction: .
[0059] S5. Divide the Traffic dataset into a 7:1:2 ratio to obtain a training set, a validation set, and a test set, and generate a test_results.txt file to save the corresponding error result list; optimize the parameters of the prediction model from step S4 using an optimizer (preferably the Adam-W optimizer); input the prediction model into the training set for model training, obtain predicted values, compare the predicted values with the true values, and calculate the loss; after each training epoch, calculate the loss on the validation set. The loss function includes mean squared error (MSE): Mean Absolute Error (MAE): ,in, The number of samples in the training set. and These are the predicted and true values in the validation set, respectively. After each training round, the loss is calculated on the validation set. Similar to the training loss, the same loss function (such as MSE or MAE) is used to calculate the loss on the validation set. If there is no improvement after 10 consecutive validations, the training is stopped early, and the optimal model weights are saved.
[0060] S6. Use the trained optimal model to predict the test set, and compare the predicted results with the actual results using error evaluation metrics. Evaluate the model performance based on the error metrics. If the MSE or MAE is low, it indicates that the model performance is good.
[0061] In steps S5 and S6 of this embodiment, the specific steps for training, validating, and testing the network model are described in detail below:
[0062] A: Configure the network environment, Python version 3.10, deep learning framework PyTorch 2.2, CUDA 11.8 with cuDNN and TensorFloat-32 acceleration enabled;
[0063] B: The Adam-W optimizer is used, with an initial learning rate of 0.001. A linear warm-up is employed for the first 5 epochs (rounds) to increase the learning rate to the maximum, followed by cosine annealing to decay it to 1 × 10⁻. 5 ;
[0064] C: Set the batch_size of each batch of input network to 32, the recall window L to 96, and enable mixed precision training to reduce GPU memory usage;
[0065] D: The network does not use pre-trained weights; during training, the loss is calculated on the validation set after each round. If the metric does not improve for 10 consecutive rounds, early stopping is triggered and the current optimal model weights are automatically saved.
[0066] E: Stop training early or after a maximum of 10 rounds, then infer on the test set and record the loss, single-batch inference latency, and GPU memory usage to comprehensively evaluate model accuracy and real-time performance.
[0067] The performance evaluation of this invention utilizes the Traffic dataset, sourced from the California Department of Transportation (Caltrans) Performance Measurement System. This dataset records hourly traffic flow data collected by multiple sensors distributed along highways in the San Francisco Bay Area between 2018 and 2019. The dataset includes multivariate time-series data on traffic flow, road occupancy, and average speed, making it suitable for scenarios such as urban traffic planning, congestion warning, and road network stress analysis. The dataset is sampled hourly, covering the flow evolution of urban arterial roads and highway networks, exhibiting clear intraday periodicity, weekly fluctuations, and non-stationary instantaneous trends caused by sudden events.
[0068] The prediction performance of the model of this invention on the above dataset is compared with that of other models in Table 1 below.
[0069] Table 1 Comparison of Model Prediction Results
[0070]
[0071] The comparison of the prediction results of this invention with other existing models in the table above shows that the prediction method proposed in this invention has higher accuracy than other models, indicating that the prediction method proposed in this invention can achieve better results in time series prediction.
[0072] like Figure 6 As shown, this embodiment discloses a Transformer-based traffic flow time series prediction system for performing the above method, including the following modules:
[0073] Normalization module: Performs instance normalization on the input traffic flow time series to obtain a standardized series;
[0074] Mapping module: The normalized sequence is linearly mapped through the linear embedding layer, the original dimensional features are projected into the high-dimensional feature space, and the temporal position encoding and channel position encoding are superimposed to form the initial feature embedding vector sequence;
[0075] Heterogeneous parallel processing module: L heterogeneous parallel composite units are stacked sequentially for the feature embedding vector sequence features. Each composite unit is processed sequentially through a wide local perception module, an asymmetric dual-path parallel module, a path fusion module, and downsampling. The asymmetric dual-path parallel module includes a causal constraint path composed of autoregressive self-attention and a global dynamic path composed of a dynamic gated feedforward network and non-causal self-attention.
[0076] Prediction model acquisition module: The top-level features are projected and mapped into H×C-dimensional vectors through a fully connected layer to establish a mapping relationship between the feature space and the prediction target; then inverse normalization is performed to obtain the prediction model;
[0077] Dataset partitioning and iteration module: The Traffic dataset is partitioned proportionally to obtain training, validation, and test sets; the Adam-W optimizer is used to optimize the parameters of the prediction model; the prediction model is trained by inputting the training set to obtain predicted values, and the predicted values are compared with the true values to calculate the loss; after each training round, the loss is calculated on the validation set; if the iteration stopping condition is met, the iteration is stopped and the optimal model is saved.
[0078] Prediction module: The trained optimal model makes predictions on the test set and compares the prediction results with the actual results using error evaluation metrics.
[0079] Other aspects of this embodiment can be found in the above method embodiments.
[0080] In summary, this invention discloses a multivariate long-term traffic flow prediction method and system based on Transformer. The method includes: 1) reversibly normalizing the input traffic flow data to remove mean-variance bias and address the non-stationarity of traffic flow; 2) mapping the standardized data to a high-dimensional space through an embedding layer and superimposing time-location encoding and channel-location encoding, capturing the initial feature representation through linear projection; 3) using a wide local perception (WLP) module to extract multi-scale local semantic information, smoothing local noise and enhancing context awareness; 4) extracting spatiotemporal features through hierarchically stacked heterogeneous parallel composite units, using causal constraint paths and global dynamic paths for parallel processing, and collaboratively learning the instantaneous fluctuations and long-term periodic patterns of traffic flow; 5) adaptively coupling local and global features using a dynamic path fusion mechanism, and restoring the original dimensions through linear projection and inverse normalization to achieve the final prediction; 6) performing traffic prediction using the optimal network obtained through training, and comparing the prediction results with the actual results through an error evaluation index. This invention can effectively process and integrate dynamic information of different traffic factors, significantly improving the accuracy and robustness of traffic flow prediction. At the same time, through an asymmetric dual-path architecture and dynamic gating mechanism, it effectively solves the problems of over-smoothing and missing causality in long-sequence prediction of traditional models, and improves the prediction ability and stability of the model in complex traffic scenarios.
[0081] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made by those skilled in the art to the technical solutions of the present invention without departing from the spirit of the present invention should fall within the protection scope of the present invention.
Claims
1. A traffic flow time series prediction method based on Transformer, characterized in that, Includes the following steps: S1. Perform reversible instance normalization on the input traffic flow time series to obtain a standardized series; S2. The standardized sequence from step S1 is linearly mapped through a linear embedding layer to project the original dimensional features into a high-dimensional feature space, and time position encoding and channel position encoding are superimposed to form an initial feature embedding vector sequence. S3. Stack L composite units sequentially on the feature embedding vector sequence from step S2. Each composite unit is processed in the order of wide local perception module, asymmetric dual-path parallel module, path fusion module, and downsampling. S4. Project the top-level features into a prediction step size H×C dimensional vector, then perform inverse normalization to restore the original dimensions and obtain the prediction model; where H represents the length of the time series and C represents the number of different traffic variables that need to be predicted simultaneously. S5. Divide the traffic dataset proportionally to obtain a training set, a validation set, and a test set; optimize the parameters of the prediction model from step S4 using an optimizer; input the prediction model into the training set to train the model, obtain predicted values, compare the predicted values with the true values, and calculate the loss; calculate the loss on the validation set after each training round; if the iteration stops when the stopping condition is met, stop the iteration and save the optimal model. S6. The optimal model saved in step S5 is used to predict the test set.
2. The traffic flow time series prediction method based on Transformer as described in claim 1, characterized in that, In step S1, reversible instance normalization is performed on the input traffic flow time series, the calculation formula is: wherein, is the historical input tensor, is the backtracking window length, is the number of variables.
3. The traffic flow time series prediction method based on Transformer as described in claim 2, characterized in that, In step S2, the standardized sequence is mapped to a high-dimensional latent space through an embedding layer. .
4. The traffic flow time series prediction method based on Transformer as described in claim 3, characterized in that, In step S3, the composite unit processing steps are as follows: S3.1 The wide local perception module uses dilated convolution and layer normalization to smooth local noise and extract context. The calculation formula is as follows: When in deep networks, features are reinforced through cross-layer feedback mechanisms. The features entering the dual-path are as follows: ; S3.
2. An asymmetric dual-path parallel module is used for processing. The causal constraint path sequentially performs layer normalization and autoregressive self-attention processing. This path uses an autoregressive mechanism to model short-range dependencies and introduces a causal mask. To ensure no future information leakage, output via localized paths. for: ,in, The meaning is the dimension of the input feature vector, which is used to scale the dot product attention. , , All are obtained by linear projection; the global dynamic path sequentially executes a dynamic gated feedforward network and non-causal self-attention processing, utilizing the relative positions Generated shape factor Dynamically adjust the nonlinear slope of the gated activation function; intermediate variables ,in For learnable parameters, Indicates the internal projection dimension of the generator network; Mapping to bounded intervals In order to obtain the final result ,in, These are shape factors The minimum and maximum values; It is a constant; output The calculation formula is as follows: , These are the projection matrices for the gated branch and the value branch, respectively. To output the projection matrix, For model dimensions, For extended dimensions of feedforward networks; As input, it enters the non-causal self-attention module, captures long-range periodic dependency patterns through a global perspective, and outputs... The calculation formula is as follows: ,in, Each by Obtained by linear projection; S3.3 Output the causal constraint path from step S3.
2. With global dynamic path output The input path fusion module generates a point-to-point fusion weight matrix using the learned soft routing mechanism: ,in, This represents the length of the feature sequence corresponding to the i-th layer composite unit. To fuse the projection matrix, This represents the splicing operation; the final output is the composite unit feature. ; S3.4, cascading is performed through hierarchical downsampling, the first... Layer input The output of the previous layer is obtained by downsampling: ,in, This represents the intermediate feature state after the path fusion module and before the downsampling module in the (i-1)th layer composite unit.
5. The traffic flow time series prediction method based on Transformer as described in claim 4, characterized in that, The specific process of step S4 is as follows: S4.1 Following the channel independence paradigm, the output characteristics obtained after processing by N composite units are... Flattened into a vector in the time and channel dimensions ,in, These are the flattened feature vectors. For the projection matrix, For bias terms; S4.2 Perform inverse normalization, using the mean recorded in step S1. and standard deviation Inverse normalization is used to restore the original physical dimensions of the traffic data, resulting in the final prediction: .
6. The traffic flow time series prediction method based on Transformer as described in any one of claims 1-5, characterized in that, In step S5, the ratio of the training set, validation set, and test set is 7:1:
2.
7. The traffic flow time series prediction method based on Transformer as described in claim 6, characterized in that, In step S5, if there is no improvement or deterioration after k consecutive verifications, an early stop is triggered, the iteration is stopped, and the optimal model is saved.
8. The traffic flow time series prediction method based on Transformer as described in any one of claims 1-5, characterized in that, In step S6, the predicted results are compared with the actual results using error evaluation indicators.
9. The traffic flow time series prediction method based on Transformer as described in claim 8, characterized in that, In step S6, the error evaluation indicators are the mean absolute error (MAE) and the mean squared error (MSE).
10. A Transformer-based traffic flow time series prediction system, used to perform the method as described in any one of claims 1-9, characterized in that, Includes the following modules: Normalization module: Performs instance normalization on the input traffic flow time series to obtain a standardized series; Mapping module: The normalized sequence is linearly mapped through the linear embedding layer, the original dimensional features are projected into the high-dimensional feature space, and the temporal position encoding and channel position encoding are superimposed to form the initial feature embedding vector sequence; Heterogeneous parallel processing module: L heterogeneous parallel composite units are stacked sequentially on the feature embedding vector sequence. Each composite unit is processed sequentially through the wide local perception module, the asymmetric dual-path parallel module, the path fusion module, and downsampling. Prediction model acquisition module: The top-level features are projected and mapped into H×C dimensional vectors through a fully connected layer to establish the mapping relationship between the feature space and the prediction target; then inverse normalization is performed to obtain the prediction model; Dataset partitioning and iteration module: The Traffic dataset is partitioned proportionally to obtain training, validation, and test sets; The optimizer is used to optimize the parameters of the prediction model; the prediction model is trained by inputting the training set to obtain the predicted value, and the predicted value is compared with the true value to calculate the loss; after each training round, the loss is calculated on the validation set; if the iteration stopping condition is met, the iteration is stopped and the optimal model is saved. Prediction module: The trained optimal model makes predictions on the test set.