Expressway OD flow prediction method based on multi-scale causal Transform architecture

Abstract

The invention belongs to the technical field of intelligent traffic, and discloses a highway OD flow prediction method based on a multi-scale causal Transform architecture, which constructs an ODFormer model, and integrates a trend-seasonal decomposition module, a time sequence feature encoder, a causal dependent decoder and a fusion and output layer to realize the prediction of the OD flow of a highway. The time-space characteristics and the multi-scale periodic characteristics in the highway OD flow data are fully mined, and the prediction precision and robustness are significantly improved. According to the technical scheme, the problems of high-dimensional sparsity, long-term prediction error accumulation, insufficient heterogeneous flow mode adaptability and the like are effectively solved, the complex topological relation and dynamic change between OD pairs are revealed through visual analysis, and accurate prediction support is provided for real-time traffic management. According to the method, expressway flow scheduling, congestion early warning and resource allocation are optimized, an innovative technical path is provided for future development of an intelligent traffic system, and the wide practical value and the profound application prospect of the method in the traffic management field are shown.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of intelligent transportation, and particularly relates to a highway OD flow prediction method based on a multi-scale causal Transformer architecture. BACKGROUND

[0002] Highway OD (Origin-Destination) flow prediction is a core component of intelligent transportation systems (ITS) and is of great significance for optimizing traffic management, improving resource allocation efficiency, and effectively alleviating traffic congestion. With the rapid growth of urbanization and traffic demand, accurate prediction of highway OD flow can provide key data support for traffic planning, real-time scheduling, and network optimization, thereby improving the overall efficiency and response capability of the transportation system. However, existing highway OD flow prediction techniques face a series of complex technical challenges, limiting their performance and reliability in practical applications. The highway OD matrix has a significant high-dimensional sparsity characteristic, with a large number of OD pairs having zero or near-zero flow. This sparsity makes it extremely difficult to extract features, and traditional prediction models struggle to mine effective spatio-temporal patterns from limited non-zero data, especially during low-flow periods, resulting in a significant decrease in prediction accuracy. In addition, the multi-scale periodicity of highway OD flow further increases the complexity of prediction. OD flow is not only influenced by short-term periodicity during daily commuting peaks, but also exhibits long-term periodic fluctuations based on weeks, months, or even seasons. These multi-scale periodic characteristics are often obscured in a sparse data environment, making it difficult for existing models to capture long-term trends and periodic fluctuations, and unable to adapt to the dynamic characteristics of highway traffic.

[0003] At the same time, the error accumulation problem in long-term multi-step prediction significantly reduces the accuracy and reliability of the prediction. Under dynamic traffic conditions, such as the influence of sudden events or weather changes, the robustness of existing models is insufficient, and prediction errors rapidly accumulate with the increase in the number of prediction steps, leading to long-term prediction results deviating from the actual traffic state. In addition, highway OD flow contains multiple heterogeneous mobile types, such as commuting flow during peak hours, long-distance travel flow during holidays, etc. These flow patterns differ significantly in terms of temporal distribution and spatial characteristics. Existing models often lack effective differentiation and adaptation mechanisms for these heterogeneous flow patterns, making it difficult to meet the diverse needs of traffic management applications in different scenarios, such as real-time scheduling, congestion warning, or resource optimization. The existence of these technical problems severely limits the application effect of highway OD flow prediction in intelligent transportation systems, and there is an urgent need for a new method that can comprehensively solve the problems of high-dimensional sparsity, multi-scale periodicity, and long-term prediction error accumulation. SUMMARY

[0004] Therefore, the present application aims to provide a highway OD flow prediction method based on a multi-scale causal Transformer architecture, aiming to effectively solve the key technical problems of high-dimensional sparsity, multi-scale periodicity, long-term multi-step prediction error accumulation and insufficient adaptability of heterogeneous flow patterns in highway OD flow prediction.

[0005] The present application provides a highway OD flow prediction method based on a multi-scale causal Transformer architecture, comprising the following steps:

[0006] S1. Collecting highway OD flow through highway electronic toll gate;

[0007] S2. Preprocessing the highway OD flow data collected in step S1;

[0008] S3. Constructing an ODFormer model;

[0009] The ODFormer model includes a trend-seasonal decomposition module, a time series feature encoder and a causal dependence decoder;

[0010] Trend-seasonal decomposition module: used to decompose the input highway OD flow sequence into trend component and seasonal component to capture long-term trend and periodic time pattern;

[0011] Time series feature encoder: used to map the trend component and seasonal component to a high-dimensional feature space to provide high-quality input for the subsequent causal dependence encoder;

[0012] Causal dependence decoder: used to capture the temporal dependence relationship and spatial heterogeneity of highway OD flow, and generate stable multi-step prediction results;

[0013] S4. Inputting the preprocessed highway OD flow data in step S2 into the ODFormer model constructed in step S3, and finally outputting the prediction results of highway OD flow.

[0014] Further, the preprocessing of step S2 includes cleaning error records and filtering missing data.

[0015] Further, the trend-seasonal decomposition module in step S3 is designed as:

[0016] Taking the highway OD flow sequence as input, the trend component and seasonal component are generated by decomposition;

[0017] The calculation of the trend component uses a moving average filter based on kernel size k, and the formula is as follows:

[0018] T=AvgPool k (X)

[0019] where T denotes the trend component; AvgPool k denotes the average pooling operation with kernel size k, where the kernel size k can be adjusted according to the periodic characteristics of the data to adapt to traffic patterns of different time scales; X denotes the input sequence;

[0020] The seasonal component is obtained by subtracting the trend component from the input sequence, as follows

[0021] S = X - T

[0022] where S denotes the seasonal component.

[0023] Further, the design of the time series feature encoder in step S3 is as follows:

[0024] Taking the trend component and the seasonal component as inputs, the trend component and the seasonal component are projected to the model dimension D through a linear time embedding layer, as follows:

[0025] Z emb = Z · W emb + b emb

[0026] where Z emb denotes the embedding representation; Z denotes the input trend component T or the seasonal component S; W emb denotes the embedding weight matrix; b emb denotes the bias vector.

[0027] Sinusoidal positional encoding is introduced to preserve the time order of the sequence, as follows:

[0028]

[0029] where P denotes the position encoding matrix; t denotes the time step; i denotes the dimension index; D denotes the model dimension.

[0030] The encoding output is obtained by adding the embedding representation and the position encoding, as follows:

[0031] Z enc = Z emb + P

[0032] where Z enc denotes the encoding output.

[0033] Further, the design of the causal dependency decoder module in step S3 is as follows:

[0034] The causal dependency decoder module contains an N-layer parallel processing architecture and adopts a causal multi-head attention mechanism to ensure that the prediction only depends on the data of the current and historical time steps, as follows:

[0035]

[0036] where Attention denotes the attention output; Q denotes the query vector; K denotes the key vector; V denotes the value vector; M denotes the causal mask; d k denotes the attention dimension;

[0037] The multi-head attention mechanism integrates the output of the encoder, and the formula is as follows:

[0038] Multi-Head(Q e ,K e ,V e )=Concat(head1,...,head h )·W 0

[0039] where Multi-Head(Q e ,K e ,V e ) denotes the overall output of the multi-head attention mechanism, which is obtained by linear projection after splicing the outputs of multiple attention heads; Q e denotes the query matrix of the decoder; K e and V e denote the key matrix and value matrix of the encoder, respectively; head h denotes the output of the hth attention head; W 0 denotes the output projection matrix; h denotes the number of attention heads; h denotes the number of attention heads;

[0040] The calculation formula of each attention head is as follows:

[0041]

[0042] In the formula, head i denotes the output of the ith attention head; denote the projection matrices of the query, key, and value, respectively;

[0043] The feedforward network further processes the attention output, and the formula is as follows:

[0044] FFN(x)=ReLU(xW1+b1)W2+b2

[0045] In the formula, FFN(x) denotes the output of the feedforward network; x denotes the input feature; W1 and W2 denote the weight matrices; b1 and b2 denote the bias vectors.

[0046] Advantages:

[0047] The ODFormer model disclosed by the application fully excavates the spatio-temporal characteristics and multi-scale periodicity characteristics in the expressway OD flow data by integrating a trend-seasonal decomposition module, a time-series feature encoder, a causal dependence decoder and a fusion and output layer, significantly improves the prediction accuracy and robustness, and improves the prediction accuracy on the ODsummer and ODwinter data sets compared with the baseline model. The technical scheme effectively solves the problems of high-dimensional sparsity, long-term prediction error accumulation and insufficient adaptability of heterogeneous flow patterns, reveals the complex topological relationship and dynamic change between OD pairs through visual analysis, and provides accurate prediction support for real-time traffic management. The application not only optimizes expressway flow scheduling, congestion warning and resource allocation, but also provides an innovative technical path for the future development of intelligent transportation systems, and shows its wide practical value and far-reaching application prospect in the field of traffic management.

[0048] Other advantages, objects, and features of the application will be set forth in part in the following specification, and in part will become apparent to those skilled in the art from a consideration of the following specification, or can be learned from practice of the application. The objects and other advantages of the application can be realized and attained by the methods and instrumentalities particularly pointed out in the following specification. BRIEF DESCRIPTION OF DRAWINGS

[0049] Figure 1 The figure is an architecture diagram of the ODFormer model;

[0050] Figure 2 The figure is a research site and location distribution visualization;

[0051] Figure 3 The figure is an adjacency relationship visualization diagram;

[0052] Figure 4 The figure is a visualization curve of ablation experiment results;

[0053] Figure 5 The figure is the sensitivity experiment results of the model to different hyperparameters. DETAILED DESCRIPTION

[0054] In order to make the technical solutions, advantages and purposes of the application clearer, the technical solutions of the embodiments of the application will be described clearly and completely below in combination with the drawings of the embodiments of the application. Obviously, the described embodiments are part of the embodiments of the application, not all the embodiments. Based on the described embodiments of the application, all other embodiments obtained by those skilled in the art without creative labor are within the protection scope of the application.

[0055] The application provides an expressway OD flow prediction method based on a multi-scale causal Transformer architecture, including the following steps:

[0056] S1. Collecting highway OD flow through highway electronic toll gate;

[0057] S2. Preprocessing the highway OD flow data collected in step S1;

[0058] The preprocessing of step S2 includes cleaning error records and filtering missing data;

[0059] S3. Constructing an ODFormer model;

[0060] The architecture of ODFormer includes the following core components: a trend-season decomposition module, a time series feature encoder, and a causal dependency decoder, and the overall model architecture is as shown in Figure 1 .

[0061] Trend-season decomposition module

[0062] To solve the problem of high-dimensional sparsity, the present application proposes a trend-season decomposition module. Based on moving average and Fourier transform, this module decomposes the sparse OD flow sequence into long-term trend components and periodic components, extracting stable time series features from high-dimensional sparse OD matrices. Unlike traditional preprocessing decomposition methods, this module is embedded inside the model, dynamically adjusting the decomposition process, enhancing the adaptability to sparse data, and effectively alleviating the limitations of data sparsity on feature extraction. By decomposing the trend and seasonal components, ODFormer can mine effective patterns from limited non-zero data, significantly improving the prediction accuracy of low-flow periods, and ensuring stability and reliability in sparse scenarios.

[0063] The trend-season decomposition module is a preprocessing core component of the ODFormer model, which decomposes the input highway OD (Origin-Destination) flow sequence into trend components and seasonal components to capture long-term trends and periodic time patterns, effectively solving the problems of high-dimensional sparsity and multi-scale periodicity. This module receives an input sequence X∈R B×L×1 , where (B) represents the batch size, (L) represents the sequence length, and the decomposition generates a trend component T∈R B×L and a seasonal component S∈R B×L . Through decomposition, the model can extract reliable time series features from sparse OD matrices, providing high-quality input for subsequent encoding processes. The calculation of the trend component uses a moving average filter based on kernel size k, with the formula as follows:

[0064] T=AvgPool k (X)

[0065] In the formula, T represents the trend component; AvgPool kdenotes the average pooling operation with kernel size k, where the kernel size k can be adjusted according to the periodic characteristics of the data to adapt to traffic patterns of different time scales; X denotes the input sequence. This formula extracts long-term trend features by smoothing the input sequence, reflecting the overall trend of the highway traffic, such as overall traffic growth or seasonal adjustment

[0066] The seasonal component is obtained by subtracting the trend component from the input sequence, and the formula is as follows

[0067] S = X - T

[0068] In the formula, S denotes the seasonal component. This formula separates the short-term fluctuation features in the input sequence, capturing periodic changes such as daily peak or weekend patterns. The trend component reflects long-term traffic changes, such as annual traffic trends or seasonal adjustments, while the seasonal component captures short-term fluctuations, such as daily commuting peaks or holiday traffic patterns. The decomposition method has a computational complexity of (O(L)), which is efficient for high-dimensional sparse OD matrices, avoiding the computational overhead of complex frequency domain analysis or wavelet decomposition. The decomposition process does not require gradient calculation and supports parallelization, significantly improving the efficiency of real-time OD traffic prediction. By generating clear trend and seasonal components, this module provides reliable input for subsequent time series feature encoders, enhancing the model's ability to model multi-scale periodicity. At the same time, this module improves the model's robustness in complex traffic scenarios by adapting to diverse highway traffic patterns such as urban commuting and long-distance freight transport, providing efficient support for real-time scheduling, congestion warning, and other application scenarios.

[0069] Time series feature encoder

[0070] To solve the problem of insufficient multi-scale periodicity capture, the present invention proposes a time series feature encoder, which uses an improved Transformer encoder structure to generate high-dimensional periodicity feature representations combined with multi-scale sinusoidal embedding and position encoding. This encoder captures the dynamic temporal dependencies of OD traffic through a multi-head self-attention mechanism, optimizing the computational complexity in sparse data environments and enhancing the ability to model long-term dependencies. The multi-scale sinusoidal embedding technique can effectively capture periodic patterns at different time scales such as daily, weekly, and monthly, addressing the limitations of existing models that assume a single dominant period or ignore multi-scale characteristics. By preserving the sequence time order and modeling complex time series patterns, the time series feature encoder significantly improves the adaptability of ODFormer to dynamic changes in highway traffic.

[0071] The time feature encoder module is the core component of the ODFormer model, aiming to map the trend component and seasonal component into a high-dimensional feature space, providing high-quality input for the subsequent encoder, effectively solving the problem of insufficient multi-scale periodicity capture in highway OD (Origin-Destination) flow prediction. This module receives the trend component and seasonal component Z∈{T,S}, through a linear time embedding layer and a sinusoidal position encoding, generates a high-dimensional periodic feature representation, while preserving the time order of the sequence, enhancing the model's ability to model complex temporal patterns and dynamic dependencies, providing reliable feature support for highway flow prediction. For the input trend component or seasonal component (Z), the time feature encoder first projects it into the model dimension (D) through a linear time embedding layer, as follows:

[0072] Z emb =Z·W emb +b emb

[0073] In the formula, Z emb represents the embedding representation; Z represents the input trend component T or seasonal component S; W emb represents the embedding weight matrix; b emb represents the bias vector. This formula maps the input component to a high-dimensional feature space through linear transformation, providing rich feature expression for subsequent processing.

[0074] Sinusoidal position encoding is introduced to preserve the time order of the sequence, as follows:

[0075]

[0076] In the formula, P represents the position encoding matrix; t represents the time step; i represents the dimension index; D represents the model dimension. These formulas generate time step-dependent encoding vectors through sine and cosine functions, ensuring that the model can capture the time sequence information of the input sequence, adapting to the multi-scale periodicity features of highway OD flow, such as daily peak or weekly fluctuations.

[0077] The encoding output is obtained by adding the embedding representation and the position encoding, as follows:

[0078] Z enc =Z emb +P

[0079] In the formula, Z enc represents the encoding output. This formula integrates high-dimensional feature representation and time order information to generate input suitable for subsequent Transformer encoder.

[0080] The timing feature encoder effectively captures the dynamic time-dependent relationship and multi-scale periodicity characteristics of OD flow by combining linear embedding and sinusoidal position encoding, optimizing the computational complexity in the sparse data environment. Compared with traditional models, this module significantly improves the modeling ability of long-term dependence and complex periodic patterns by preserving the sequence time order and generating high-dimensional feature representations, enhancing the adaptability of the model in highway traffic scenarios such as urban commuting and long-distance freight transportation. The encoding process is computationally efficient and supports parallel processing, suitable for real-time OD flow prediction requirements, providing high-quality feature input for the subsequent causal dependence decoder to ensure the accuracy and stability of the prediction results.

[0081] Causal dependence decoder

[0082] To solve the problem of error accumulation in long-term multi-step prediction, the present application proposes a causal dependence decoder. The decoder adopts a multi-layer structure design, and through a causal multi-head attention mechanism, it only relies on data at the current and historical time steps for prediction, effectively avoiding error accumulation and ensuring the stability of long-term prediction. The time-aware feedforward network further integrates trend and seasonal features, enhancing the robustness of the model under dynamic traffic conditions such as sudden events or weather changes. Compared with traditional models, the causal dependence decoder significantly reduces the error accumulation problem in long-term multi-step prediction by limiting the dependence on future data, providing stable and reliable prediction results for real-time traffic management.

[0083] The causal dependence decoder module is the core component of the ODFormer model, aiming to process trend and seasonal feature inputs through a multi-layer structure design and a causal multi-head attention mechanism, and only rely on historical data for prediction, thereby effectively solving the problems of long-term multi-step prediction error accumulation and insufficient adaptability of heterogeneous flow patterns in highway OD (Origin-Destination) flow prediction. The module contains an (N) layer parallel processing architecture, which captures the temporal dependence relationship and spatial heterogeneity of OD flow through a causal multi-head attention mechanism and a time-aware feedforward network, generates stable multi-step prediction results, and is suitable for real-time scheduling, congestion warning, and other highway traffic management scenarios.

[0084] The causal dependence decoder adopts a causal multi-head attention mechanism to ensure that the prediction only relies on data at the current and historical time steps, as follows:

[0085]

[0086] where Attention represents the attention output; Q represents the query vector; K represents the key vector; V represents the value vector; M represents the causal mask; d kAttention dimension. The formula restricts the prediction of future data by causal mask, effectively avoids error accumulation in long-term multi-step prediction, and improves the robustness of the model under dynamic traffic conditions.

[0087] The multi-head attention mechanism integrates the output of the encoder, and the formula is as follows:

[0088] Multi-Head(Q e ,K e ,V e )=Concat(head1,...,head h )·W 0

[0089] Where Multi-Head(Q e ,K e ,V e ) represents the overall output of the multi-head attention mechanism, which is obtained by linear projection after splicing the outputs of multiple attention heads; Q e represents the query matrix of the decoder; K e and V e represent the key matrix and value matrix of the encoder, respectively; head h represents the output of the hth attention head; W 0 represents the output projection matrix; h represents the number of attention heads; h represents the number of attention heads.

[0090] The calculation formula of each attention head is as follows:

[0091]

[0092] In the formula, head i represents the output of the ith attention head; represent the projection matrices of queries, keys and values, respectively. The formula generates specific feature representations for each attention head through independent projection matrices, further improving the modeling accuracy of the model for complex OD traffic patterns.

[0093] The feedforward network further processes the attention output, and the formula is as follows:

[0094] FFN(x)=ReLU(xW1+b1)W2+b2

[0095] FFN(x) represents the output of the feedforward network; x represents the input features; W1, W2 represent the weight matrix; b1, b2 represent the bias vector. The formula integrates trend and seasonal features through a nonlinear activation function (ReLU) and multiple linear transformations, capturing the spatiotemporal characteristics of heterogeneous modes such as peak commuter traffic and holiday long-distance traffic. The causal dependency decoder processes the trend and seasonal branches through (N) layers in parallel, using causal multi-head attention mechanisms and time-aware feedforward networks to effectively model the topological relationships between OD pairs and the diversity of traffic patterns, adapting to complex traffic scenarios such as urban commuting and long-distance freight transport. Compared with traditional models, this module significantly reduces error accumulation in long-term prediction by limiting dependence on future data, while enhancing adaptability to heterogeneous traffic patterns through multi-head attention mechanisms.

[0096] S4. Input the expressway OD flow data preprocessed in step S2 into the ODFormer model constructed in step S3, and finally output the prediction result of the expressway OD flow.

[0097] Experimental verification

[0098] I. Experimental data and environment

[0099] The present application uses a real data set collected from the ETC (Electronic Toll Collection) gantry of Chongqing expressway to verify the performance of the ODFormer model in expressway OD flow prediction. The data set is divided into two subsets, covering different time periods: the summer data set (ODsummer) covers three months from June 1, 2024 to August 31, 2024, and the winter data set (ODwinter) covers three months from December 1, 2024 to February 28, 2025. The data set records the movement data of vehicles between ETC gantries in the suburban expressway area of Chongqing, including high-resolution OD flow records. To ensure data quality, the present application implements strict preprocessing steps, including cleaning error records and filtering missing data. From the original data, the present application selects the 24 gantries with the highest traffic, focusing on significant traffic patterns, to construct a representative OD flow data set.

[0100] The adopted dataset description is shown in Table 1. The time range of the ODsummer dataset is from June 1, 2024 00:00 to August 31, 2024 23:59, a total of 92 days, containing 2208 time steps (calculated in small time intervals). Its data structure is a three-dimensional tensor with dimensions of 24x24x2208, where the first two dimensions represent 24 pairs of starting and ending gantry, and the third dimension represents the time sequence, with a total of 1274112 elements (i.e. 24x24x2208). The time range of the ODwinter dataset is from December 1, 2024 00:00 to February 28, 2025 23:59, a total of 90 days, containing 2160 time steps, with dimensions of 24x24x2160, and a total of 1244160 elements (i.e. 24x24x2160). Both datasets use float32 data format and show high sparsity, with most OD pairs having zero or near-zero flow in most time periods, reflecting the typical sparse characteristics of highway OD data.

[0101] To further verify the high sparsity of highway OD data in the spatial dimension, the present application constructs a 24x24 adjacency matrix representing the direct highway connection relationship between gantries. Analysis shows that most elements in the adjacency matrix are close to blank, and only a few OD pairs show significant connection strength, indicating that the flow between most starting-end pairs is almost zero in most time periods. In addition, by drawing the average OD flow line chart of July 2024 and January 2025, it is observed that the overall level of OD flow is low, and the flow during off-peak hours is close to zero, showing a sparse and discontinuous time pattern. This sparsity in space and time dimensions further highlights the complexity of highway OD flow prediction, verifying the necessity of the ODFormer model in handling high-dimensional sparse data.

[0102] To support the training and evaluation of the model, the dataset is divided in the following proportions: 80% for training, 10% for validation, and 10% for testing. This division ensures the robustness of the model in evaluating diverse traffic scenarios. The experimental environment is based on a high-performance computing platform, combined with real-time data processing technology, which can efficiently process high-dimensional sparse OD matrices, supporting the training and real-time prediction needs of the ODFormer model.

[0103] Table 1 Highway OD flow dataset description

[0104]

[0105]

[0106] The experimental environment configuration used by the present application is shown in Table 2:

[0107] Table 2 Experimental environment configuration

[0108]

[0109] II. Experimental Setup

[0110] To verify the performance of the ODFormer model in freeway OD flow prediction, the present invention trains the ODFormer model and baseline models on the training dataset, determines the optimal hyperparameters using the validation set, and evaluates the performance on the test set. During training, the mean absolute error (MAE) on the validation set is calculated after each training cycle, and the model parameters are only updated when the validation error decreases, otherwise the training continues. To prevent overfitting and ensure efficient convergence, an early stopping mechanism is used, with a patience value of 5 cycles and a maximum training cycle of 100. The historical time step is set to P = 24, and the prediction time range is configured as horizon = 3, 6, 9, 12 to support multi-step prediction. All deep learning models are implemented, trained and tested in a GPU-based computing environment based on the PyTorch framework. The hyperparameters of the ODFormer model are optimized using a grid search strategy on the validation set, as shown in Table 3. To reduce the impact of randomness, each experiment is repeated three times, and the average performance is reported.

[0111] Table 3. Experimental parameter settings

[0112]

[0113]

[0114] Performance evaluation uses the following indicators: root mean square error (RMSE), mean absolute error (MAE), and mean absolute percentage error (MAPE).

[0115] III. Benchmark Models and Comparison Results

[0116] To evaluate the performance of the ODFormer model proposed in the present invention in freeway OD flow prediction, the present invention compares it with eight advanced baseline models widely used in traffic flow prediction and time series prediction tasks. These baseline models are selected for their relevance in modeling spatio-temporal traffic data and their excellent performance in intelligent transportation systems. The following is a description of each baseline model:

[0117] LSTM: A long short-term memory network (Long Short-Term Memory) that captures long-term temporal dependencies in traffic flow sequences through memory cells and gating mechanisms. This model uses forget gates, input gates and output gates to manage information flow, suitable for handling long-term trends in time series, but in high-dimensional sparse OD matrices, it is susceptible to gradient vanishing, resulting in limited long-term prediction performance.

[0118] BiLSTM: A bidirectional long short-term memory network that captures contextual time trends by processing traffic sequences in both forward and backward directions. Compared to unidirectional LSTM, BiLSTM can comprehensively consider the preceding and following information of the sequence, enhancing its ability to model short-term and medium-term trends. However, it is less efficient at processing sparse data and has higher computational complexity.

[0119] STGCN: A Spatio-Temporal Graph Convolutional Network that combines graph convolution and temporal convolution to model the spatial connectivity and temporal dynamics of traffic data. This model captures the spatial relationships between OD pairs through graph structures and utilizes temporal convolution to handle sequence dynamics. However, in high-dimensional sparse OD matrices, graph convolution has limited ability to model sparse connections.

[0120] ASTGCN is an attention-based spatio-temporal graph convolutional network that utilizes an attention mechanism to focus on key spatial and temporal traffic features. This model enhances the modeling of important OD pairs and time steps by dynamically assigning attention weights. However, in sparse data scenarios, the computational cost of the attention mechanism is significant, impacting real-time prediction efficiency.

[0121] Transformer: A Transformer model that uses a self-attention mechanism to capture global temporal dependencies in traffic sequences. This model processes sequence features in parallel through multi-head attention, making it suitable for long sequence prediction. However, it has weak processing capabilities for high-dimensional sparse OD matrices and has high computational complexity, making it difficult to effectively adapt to dynamic changes in highway traffic flow.

[0122] PDFormer is a Propagation Delay-aware Dynamic Long-range Transformer that models traffic propagation delays and long-term trends through a dynamic attention mechanism. While optimized for delay characteristics in traffic data, the effectiveness of dynamic attention is limited in sparse OD matrices, and it lacks adaptability to heterogeneous traffic patterns.

[0123] GPHT: A Generative Pretrained Hierarchical Transformer that models multi-scale traffic trends using hierarchical structures and pre-training techniques. This model enhances generalization through pre-training and is suitable for multi-scale periodic modeling, but its performance stability is slightly inferior in high-dimensional sparse data and long-term prediction, especially during low-traffic periods.

[0124] Robformer: A Robust Decomposition Transformer that achieves robust long-term prediction by decomposing time series into trend and seasonal components. This model performs well in handling periodic patterns, but its adaptability to high-dimensional sparsity and heterogeneous traffic patterns of highway OD data is limited, and its prediction accuracy decreases in complex scenarios.

[0125] The experimental results in Table 4 show that ODFormer significantly outperforms all baseline models on both ODsummer and ODwinter datasets. On the ODsummer dataset, ODFormer performs well in both short-term and long-term prediction tasks, especially in reducing MAPE. Although its MAE and MAPE are slightly lower than GPHT in the medium-term prediction task, ODFormer still achieves the lowest RMSE. On the ODwinter dataset, ODFormer exhibits leading performance in almost all prediction time ranges and evaluation metrics, with only a slightly higher MAE than PDFormer in the 12-hour prediction range. In terms of average performance, ODFormer achieves the best performance balance on both datasets, especially in reducing MAPE, demonstrating its high accuracy and robustness in real traffic prediction scenarios. Although baseline models such as Transformer, GPHT, and PDFormer perform competitively in certain indicators or prediction time ranges, ODFormer achieves more consistent and stable performance through overall optimization of multi-step prediction. The experimental results verify the effectiveness of ODFormer in modeling complex spatio-temporal relationships and long-term dependencies, providing a powerful solution for highway OD flow prediction tasks.

[0126] Table 4: Prediction results on two datasets

[0127]

[0128]

[0129] IV. Case Visualization Analysis

[0130] To further verify the performance and interpretability of the ODFormer model in highway OD flow prediction, the present invention conducts a visual analysis of data distribution and adjacency matrix based on the Chongqing highway electronic toll collection (ETC) gantry dataset. These analyses reveal the spatial sparsity and temporal dynamic characteristics of highway OD flow by studying the distribution map of data locations and the adjacency matrix visualization map plotted by stations, providing intuitive support for the advantages of the ODFormer model in handling high-dimensional sparsity and multi-scale periodicity issues. The following analyzes in detail from three aspects of data distribution, adjacency matrix characteristics, and model interpretability.

[0131] 1) Data distribution: The Chongqing highway dataset contains 24 high-flow transfer ETC gantries, covering the suburban highway area of Chongqing. Figure 2 The spatial positions of these gantries are shown, reflecting the topological structure of the highway network and the spatial distribution characteristics of traffic flow. Analysis shows that gantries are mainly distributed near urban commuter routes and major freight routes, with some gantries located in traffic hub areas, bearing high-density flow, while others are located in long-distance connecting sections, with low and sparse flow. This spatial distribution imbalance results in a large number of OD pairs in the OD matrix with zero or near-zero flow, presenting high-dimensional sparsity. The ODFormer model effectively extracts effective features from sparse data by separating long-term trends and short-term periodic fluctuations through the trend-seasonal decomposition module, solving the problem of insufficient prediction accuracy of traditional models at low-flow gantries. The distribution map further shows that the distance and connection between gantries have a significant impact on the flow pattern, for example, gantries near the city center exhibit a clear daily peak period, while suburban gantries exhibit weekly or seasonal fluctuations. The multi-scale sinusoidal embedding technology of ODFormer can capture these periodic patterns at different time scales, ensuring the adaptability of the model to complex spatial distribution.

[0132] 2) Adjacency matrix visualization analysis: 24x24 adjacency matrix visualization map constructed from 24 ETC gantry stations Figure 3The adjacency matrix visualization intuitively shows the direct highway connection relationship between gantries. The analysis shows that most elements in the adjacency matrix are close to blank, and only a few OD pairs show significant connection strength, reflecting the high sparsity of highway OD data in the spatial dimension. For example, OD pairs related to urban commuting (such as gantries connecting the main urban area and the suburbs) have higher flow strength, while long-distance OD pairs (such as gantries connecting suburban areas) have sparse and discontinuous flow. This sparsity makes it difficult for traditional models to capture effective spatial dependencies. The ODFormer model uses the causal multi-head attention mechanism in the causal dependency decoder to model the topological relationship between OD pairs, focusing on the dynamic characteristics of high-flow OD pairs while effectively extracting features from low-flow OD pairs. The adjacency matrix visualization also reveals the flow mutation characteristics of some OD pairs in certain time periods (such as holidays or peak hours), and the time-aware feedforward network of ODFormer can adapt to these heterogeneous flow patterns, enhancing the model's prediction stability in dynamic traffic scenarios.

[0133] 3) Model interpretability analysis: The ODFormer model provides strong interpretability through its modular design, combining data distribution plots and adjacency matrix visualizations to intuitively explain the generation process of model prediction results. First, the trend-seasonal decomposition module decomposes the OD flow sequence into trend and seasonal components, and the generated trend component reflects long-term flow changes (such as annual growth trends), while the seasonal component captures short-term fluctuations (such as daily peak or weekend patterns). By visualizing these components, we can clearly observe the flow pattern differences of different gantry pairs, such as the peak periodicity of urban commuting gantries and the low-frequency fluctuations of long-distance freight gantries. Second, the multi-scale sinusoidal embedding and position encoding of the time series feature encoder generate high-dimensional feature representations, which can be visualized through embedding vectors to analyze how the model captures multi-scale periodic features, such as the weight distribution of daily, weekly, and monthly periods. Finally, the causal multi-head attention mechanism of the causal dependency decoder reveals the model's attention to historical time steps and key OD pairs through attention weight visualization. For example, when predicting peak period flow, the model prioritizes urban commuting-related OD pairs, with more concentrated weight distribution, while in low-flow periods, the model can dynamically adjust attention to adapt to sparse data environments.

[0134] In summary, the data distribution map and the adjacency matrix visualization provide important support for the performance verification of the ODFormer model, revealing the high-dimensional sparsity and multi-scale periodicity of highway OD data. ODFormer effectively addresses the limitations of traditional models in sparse data and complex traffic scenarios through the collaborative design of trend-seasonal decomposition modules, temporal feature encoders, and causal dependency decoders. The explainability analysis further validates the model's superiority in capturing spatial topological relationships and temporal dynamic characteristics, providing reliable prediction support for real-time scheduling, congestion warning, and resource optimization in highway traffic management applications.

[0135] V. Ablation Experiment

[0136] To comprehensively evaluate the effectiveness of each key component of the ODFormer model, the present invention conducts an ablation experiment to systematically analyze the impact of trend-seasonal decomposition modules, temporal position encoding, causal multi-head attention mechanisms, and decoder layer numbers on highway OD flow prediction performance. The experiment compares the complete ODFormer model with variants that remove or adjust each component, verifying the role of each component in addressing high-dimensional sparsity, multi-scale periodicity, and long-term prediction error accumulation. The experimental results are presented through visualization curves, clearly showing the performance changes of each ablation variant under different prediction time ranges (3 hours, 6 hours, 12 hours). Root mean square error (RMSE), mean absolute error (MAE), and mean absolute percentage error (MAPE) are used as evaluation indicators. The specific content and significance of each ablation experiment are listed in detail below.

[0137] Visualization curves of experimental results Figure 4 The results show that the trend-seasonal decomposition module and the temporal position encoding are the core pillars of the ODFormer performance. Removing any module leads to a significant decrease in prediction accuracy, especially in long-term prediction scenarios. Reducing the number of decoder layers mainly affects long-term prediction, while the causal multi-head attention mechanism contributes relatively less but is still beneficial. ODFormer achieves optimal performance in RMSE, MAE, and MAPE indicators by integrating all components, verifying its effectiveness and robustness in modeling the complex spatiotemporal relationships and long-term dependencies of highway OD flow. This provides efficient prediction support for real-time traffic management.

[0138] VI. Results of Different Hyperparameter Settings for the Model

[0139] To evaluate the sensitivity of the ODFormer model to key hyperparameters, experiments were conducted to analyze the impact of model dimension (d_model), number of network layers (num_layers), feedforward network dimension (feedforward_dim), and convolution kernel size (kernel_size) on the performance of freeway OD (Origin-Destination) flow prediction. The experimental method is as follows: change the value of only one hyperparameter each time, keep other parameters fixed, repeat each setting 10 times, and show the performance changes of root mean square error (RMSE), mean absolute error (MAE), and mean absolute percentage error (MAPE) under different prediction time ranges (3, 6, 9, and 12 hours) through visual curves. The experimental results are shown in Figure 5 The results show that trend-season decomposition and time position encoding have the greatest impact on performance, and appropriate hyperparameter settings significantly improve model accuracy and stability.

[0140] It is declared that the above embodiments are only used to illustrate the technical solutions of the present application but not limit the present application. Although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or replaced equivalently without departing from the purpose and scope of the present application, and they should be covered in the scope of claims of the present application.

Claims

1. A method for predicting OD (Original Demand) traffic flow on highways based on a multi-scale causal Transformer architecture, characterized in that, Includes the following steps: S1. Collect highway OD (Original Discharge) flow through the highway electronic toll collection gantry; S2. Preprocess the highway OD traffic data collected in step S1; S3. Construct the ODFormer model; The ODFormer model includes a trend-seasonal decomposition module, a temporal feature encoder, and a causal dependency decoder. Trend-Season Decomposition Module: Used to decompose the input highway OD traffic sequence into trend and seasonal components to capture long-term trends and periodic time patterns. Temporal feature encoder: used to map trend and seasonal components to a high-dimensional feature space, providing high-quality input for subsequent causal dependency encoder; Causal dependency decoder: used to capture the temporal dependency and spatial heterogeneity of highway OD traffic and generate stable multi-step prediction results; S4. Input the highway OD flow data preprocessed in step S2 into the ODFormer model constructed in step S3, and finally output the prediction results of highway OD flow.

2. The highway OD traffic prediction method based on a multi-scale causal Transformer architecture according to claim 1, characterized in that: The preprocessing in step S2 includes cleaning up erroneous records and filtering out missing data.

3. The highway OD traffic prediction method based on a multi-scale causal Transformer architecture according to claim 2, characterized in that, The design of the trend-seasonal decomposition module in step S3 is as follows: Using the highway OD flow sequence as input, trend components and seasonal components are generated by decomposition. The trend component is calculated using a moving average filter based on kernel size k, as shown in the following formula: T=AvgPool k (X) In the formula, T represents the trend component; AvgPool k This represents an average pooling operation with a kernel size of k, where the kernel size k can be adjusted according to the periodicity of the data to adapt to different time scales of traffic patterns; X represents the input sequence. The seasonal component is obtained by subtracting the trend component from the input sequence, as shown in the following formula. S=XT In the formula, S represents the seasonal component.

4. The highway OD traffic prediction method based on a multi-scale causal Transformer architecture according to claim 3, characterized in that, The temporal feature encoder in step S3 is designed as follows: Using trend and seasonal components as inputs, a linear-time embedding layer projects the trend and seasonal components onto the model dimension D, as shown in the following formula: WITH emb =Z·W emb +b emb In the formula, Z emb Z represents the embedded representation; Z represents the trend component T or seasonal component S of the input; W emb b represents the embedding weight matrix; emb Represents the bias vector; A sinusoidal positional encoding is introduced to preserve the temporal order of the sequence, as shown in the following formula: In the formula, P represents the location encoding matrix; t represents the time step; i represents the dimension index; and D represents the model dimension. The encoded output is obtained by adding the embedding representation to the positional encoding, as shown in the following formula: WITH enc =Z emb +P In the formula, Z enc This indicates the encoded output.

5. The highway OD traffic prediction method based on a multi-scale causal Transformer architecture according to claim 4, characterized in that, The design of the causal dependency decoder module in step S3 is as follows: The causal dependency decoder module comprises an N-layer parallel processing architecture and employs a causal multi-head attention mechanism to ensure that the prediction depends only on the data from the current and historical time steps, as shown in the following formula: Where Attention represents the attention output; Q represents the query vector; K represents the key vector; V represents the value vector; M represents the causal mask; d k Indicates the attention dimension; The multi-head attention mechanism integrates the encoder's output, as shown in the following formula: Multi-Head(Q e ,K e ,V e )=Concat(head1,...,head h )·W 0 Among them, Multi-Head(Q e ,K e V e Q represents the overall output of the multi-head attention mechanism, which is obtained by concatenating the outputs of multiple attention heads and then linearly projecting them; e K represents the query matrix of the decoder. e and V e These represent the key matrix and value matrix of the encoder, respectively; head h W represents the output of the h-th attention head. 0 This represents the output projection matrix; h represents the number of attention heads; h represents the number of attention heads; The formula for calculating each attention point is as follows: In the formula, head i This represents the output of the i-th attention head; These represent the projection matrices of the query, key, and value, respectively. The feedforward network further processes the attention output, as shown in the following formula: FFN(x)=ReLU(xW1+b1)W2+b2 In the formula, FFN(x) represents the output of the feedforward network; x represents the input feature; W1 and W2 represent the weight matrices; and b1 and b2 represent the bias vectors.

Images