Traffic flow prediction method based on dynamic sparse causal attention network

Abstract

The invention provides a traffic flow prediction method based on a dynamic sparse causal attention network, and the method comprises the following steps: collecting road network traffic flow data, and carrying out the preprocessing and normalization operation of the data; constructing a dynamic graph, and generating a dynamic adjacency matrix by using a sliding window and Pearson correlation analysis; extracting spatial features and time features of the traffic flow through flow sub-sequence embedding and time sub-sequence embedding respectively; constructing a space-time prediction model based on a sparse causal attention mechanism, and fusing the dynamic graph and the causal attention mechanism; the spatial feature representation capability is enhanced by using Laplace position coding; and inputting the preprocessed data into the model for training, and predicting the future traffic flow by using the trained model. According to the method, a dynamic graph structure, a sparse causal attention mechanism and Laplacian position coding are introduced, embedding of a flow sub-sequence and a time sub-sequence is combined, an efficient space-time prediction model is constructed, and the requirement of an intelligent traffic system for high-precision prediction is met.

Description

Technical Field

[0001] The present invention relates to the field of intelligent transportation, and specifically to a traffic flow prediction method based on a dynamic sparse causal attention network. Background Art

[0002] With the continuous acceleration of the global urbanization process, the urban population has grown rapidly, and the number of motor vehicles in possession has shown an explosive growth trend. The problem of urban traffic congestion has become increasingly serious. Especially during the morning and evening rush hours and holidays, traffic congestion not only leads to low travel efficiency but also exacerbates air pollution and energy consumption, posing a huge challenge to the sustainable development of cities. As an effective means to solve this problem, the Intelligent Transportation System (ITS) provides important support for alleviating congestion and improving road use efficiency by real-time monitoring and optimizing traffic flow. As the core component of the intelligent transportation system, traffic flow prediction can provide accurate traffic information for travelers, support navigation systems to optimize route planning, and at the same time help traffic management departments dynamically adjust signal control strategies, reduce the waiting time at traffic lights, and achieve "green wave passing", thereby effectively alleviating traffic congestion, reducing accident risks, and improving the overall traffic efficiency. Therefore, developing efficient and accurate traffic flow prediction methods not only has important practical significance but also has a profound impact on promoting the construction of smart cities and realizing sustainable traffic development.

[0003] Traffic flow prediction is a highly challenging task, and its core difficulty lies in the complex spatio-temporal correlation of traffic flow data. Traffic flow data usually presents as graph data with a non-Euclidean structure, and its spatial dependence relationship shows dynamic changes due to factors such as road network topology, road grade, weather changes, emergencies (such as traffic accidents or construction), and holiday travel patterns. At the same time, in the time dimension, traffic flow data shows obvious periodic patterns (such as morning and evening rush hours on weekdays, travel patterns on weekends) and long-term trends (such as traffic flow fluctuations during holidays). In addition, traffic flow data is also interfered by external factors. For example, sudden weather changes may lead to a sharp drop in traffic flow, and emergencies may cause abnormal fluctuations in traffic flow on local sections, which further increases the complexity of prediction. Traditional traffic flow prediction methods, such as Autoregressive Integrated Moving Average (ARIMA) and Kalman filtering, are mainly based on linear assumptions and require data to be stationary. However, actual traffic flow data is usually non-linear and is interfered by various external factors non-linearly, which makes it difficult for traditional time series analysis methods to capture complex spatio-temporal patterns, and the prediction accuracy often cannot meet the actual needs.

[0004] With the development of data mining and machine learning technologies, machine learning-based methods have been introduced into the field of traffic flow prediction. For example, Vector Auto-Regression (VAR) and Support Vector Regression (SVR) can, through feature engineering and modeling, reveal the non-linear relationships in traffic data to a certain extent and predict traffic trends by continuously adjusting and optimizing model parameters. However, machine learning methods usually require professionals to conduct complex feature design and model tuning. The modeling process is rather cumbersome and the computational cost is relatively high. More importantly, most of the above methods only focus on the data changes in the time dimension and ignore the dependencies in the space dimension, such as the traffic flow transfer and mutual influence between different nodes in the road network, which limits their prediction performance in complex road network scenarios. In addition, when dealing with large-scale data sets, machine learning methods often face the problem of the curse of dimensionality, with insufficient generalization ability of the model and difficulty in adapting to dynamically changing traffic scenarios.

[0005] In recent years, the rapid development of deep learning technologies has provided new solutions for traffic flow prediction. Deep learning models can automatically extract complex features from data and have attracted wide attention in the field of traffic flow prediction. Methods based on Recurrent Neural Networks (RNNs), such as Long Short-Term Memory networks (LSTMs) and Gated Recurrent Units (GRUs), can capture certain non-linear time dependencies by modeling time series data and have achieved better results compared with traditional time series analysis and machine learning methods. To further learn the spatio-temporal correlations of traffic data simultaneously, researchers have proposed methods that fuse Convolutional Neural Networks (CNNs) and RNNs, using CNNs to extract spatial features and combining RNNs to model time dependencies. However, CNNs cannot effectively process road network data with non-Euclidean structures, resulting in the loss of spatial information. For this reason, Graph Convolutional Networks (GCNs) have been introduced into the field of traffic flow prediction, modeling spatial dependencies through graph structures and combining with Temporal Convolutional Networks (TCNs) to capture long-term time dependencies, significantly improving the prediction performance. In recent years, Transformer models based on self-attention mechanisms have also begun to be applied to traffic flow prediction. Their highly parallel self-attention mechanisms can adaptively capture long-term time dependencies and avoid the problem of gradient vanishing in RNN models. In addition, some studies have further enhanced the model's ability to capture dynamic spatial correlations by constructing adaptive adjacency matrices.

[0006] Although the above methods have made significant progress in traffic flow prediction, the existing technologies still face many challenges. First, the spatial correlation of traffic flow is dynamically changing, affected by factors such as road structure, weather changes, and emergencies. Traditional static adjacency matrices or simple adaptive adjacency matrices are difficult to accurately adapt to this dynamic nature, resulting in insufficient modeling ability of the model for spatial dependencies. Second, when dealing with large-scale road network data, existing models often ignore sparse modeling due to high computational complexity. For example, the associations between most nodes in the road network are weak, and forcibly modeling all node relationships will significantly increase the computational burden and reduce the model efficiency. Third, existing methods generally lack the modeling of causal relationships and are difficult to accurately capture the causal laws of traffic flow changes. For example, whether a sudden increase in the flow at a certain node is caused by congestion in the upstream section. The lack of such causal relationships limits the interpretability and prediction accuracy of the model. In addition, the insufficient representation ability of spatial features is also a key issue. Traditional methods usually rely on simple graph embeddings or position encodings and are difficult to fully express the spatial structure features of complex road networks, such as the spatial distribution and topological characteristics of nodes in the road network. Finally, for the prediction of long historical data, existing models still have limitations in capturing long-term time dependencies. For example, although TCN achieves a large receptive field through dilated convolutions, the stacked convolutional layers may lead to information loss and affect the model performance. Therefore, there is an urgent need for a traffic flow prediction method that can efficiently capture dynamic spatio-temporal correlations, take into account sparse and causal modeling, and improve the representation ability of spatial features to meet the requirements of intelligent transportation systems for high-precision predictions. Summary of the Invention

[0007] The present invention proposes a traffic flow prediction method based on a dynamic sparse causal attention network, aiming to solve the problems of insufficient dynamic spatio-temporal correlation capture, sparse modeling, causal modeling, and spatial feature representation ability in existing traffic flow prediction methods. By introducing a dynamic graph structure, a sparse causal attention mechanism, and Laplacian position encoding, combined with the embedding of traffic flow subsequences and time subsequences, an efficient spatio-temporal prediction model is constructed to improve the accuracy and robustness of traffic flow prediction and meet the requirements of intelligent transportation systems for high-precision predictions.

[0008] A traffic flow prediction method based on a dynamic sparse causal attention network includes the following steps: Step 1, collect road network traffic flow data and perform preprocessing and normalization operations on the data.

[0009] Step 2, construct a dynamic graph structure and generate a dynamic adjacency matrix using a sliding window and Pearson correlation analysis.

[0010] Step 3, extract the spatial features and time features of traffic flow through traffic flow subsequence embedding and time subsequence embedding respectively.

[0011] Step 4: Construct a spatio-temporal prediction model based on a sparse causal attention mechanism, integrating a dynamic graph and a causal attention mechanism.

[0012] Step 5: Enhance the spatial feature representation ability using Laplacian position encoding.

[0013] Step 6: Input the preprocessed data into the model for training, and use the trained model to predict future traffic flow.

[0014] Preferably, the process of data preprocessing in Step 1 is specifically as follows: The original data collected is , where represents the batch size, represents the time step, represents the number of nodes in the road network, represents the feature dimension, including traffic flow features and time features; perform a normalization operation on the data, calculate the mean and standard deviation of the data, and standardize the data to eliminate the influence of different feature dimensions. The formula is as follows: , where, is the mean of the data, is the standard deviation of the data, is a small constant to avoid division by zero errors.

[0015] Preferably, the construction of the dynamic graph in Step 2 is specifically as follows: Step 2.1, use a sliding window to extract the time window data from the normalized traffic flow data , is the window size, and perform a normalization operation on the window data: , where, and are the mean and standard deviation of the window data respectively.

[0016] Step 2.2, calculate the Pearson correlation matrix based on the labeled window data: , where, is 's transpose matrix.

[0017] Step 2.3, perform a normalization operation on the correlation matrix to generate a dynamic adjacency matrix. The formula is as follows: , and through the softmax operation, ensure that the sum of each row of the adjacency matrix is 1.

[0018] Preferably, the embedding of the traffic flow subsequence in Step 3 is specifically as follows: Perform a slicing operation on the normalized traffic flow data to generate traffic flow subsequences. The formula is as follows: , where, is the slicing length, is the step size, and the generated , where P is the number of subsequences.

[0019] The traffic subsequences are mapped to the embedding space through a linear transformation, and position encoding is added to preserve the position information of the time series. The formula is as follows: , where Linear represents the linear transformation layer, and PE represents the position encoding function, which is generated using a sine function to capture the relative position relationship in the sequence.

[0020] Preferably, the time subsequence embedding in step 3 is specifically: extracting historical time features and future time features from the preprocessed input data, which respectively include historical minute features , historical week features , future minute features and future week features ; the minute features and week features are respectively input into the corresponding embedding layers for embedding processing, input into the corresponding embedding layers for processing, and the two are concatenated to generate historical and future time embedding features: ; , where and are the embedding layers for minutes and weeks respectively.

[0021] Preferably, step 4 constructs a spatio-temporal prediction model based on a sparse causal attention mechanism, including the following operations: Step 4.1, based on the dynamic adjacency matrix generated in step 2, calculate the average degree of the adjacency matrix, and determine the number of nodes with sparse adjacency according to the maximum neighbor ratio , the formula is as follows: ; , where is the number of nodes, is the maximum neighbor ratio.

[0022] Step 4.2, extract the neighbor indices corresponding to the maximum values from the dynamic adjacency matrix, and construct a sparse matrix based on these indices; then based on the query matrix , sparse key matrix and sparse value matrix , calculate the sparse causal attention, the formula is as follows: , where is the dimension of the key, is the causal mask matrix, which is used to ensure that the attention mechanism follows the causality of the time series: .

[0023] Step 4.3, for self-attention calculation in the time dimension of the input features, the formula is as follows: , where , and are the query, key, and value matrices in the time dimension respectively.

[0024] Step 4.4, through a weighted fusion operation, fuse the sparse causal attention and temporal attention to generate comprehensive spatio-temporal features, and the formula is as follows: , where is the weight parameter, and are the outputs of the sparse causal attention and temporal attention respectively.

[0025] Preferably, the generation of the Laplacian positional encoding in step 5 is specifically: using the Laplacian matrix of the road network generate spatial positional encoding through linear transformation: .

[0026] Expand the spatial positional encoding to the same dimension as the temporal embedding feature and perform element-wise addition: .

[0027] Preferably, the model prediction in step 6 extracts high-dimensional spatio-temporal features through a spatio-temporal encoder, including the following operations: applying the sparse causal attention mechanism of step 4 to the input features and updating through a feed-forward neural network, and the formula is as follows: ; , where is the input spatio-temporal feature, represents the attention mechanism of step 4, represents the stochastic depth regularization, represents the RMS normalization, is the feed-forward neural network.

[0028] Map the extracted high-dimensional features to the prediction dimension through linear transformation to generate the normalized prediction result .

[0029] Preferably, in step 6, by stacking multiple spatio-temporal encoding blocks, the output of each layer is used as the input of the next layer, and the attention calculation and feed-forward network update operations are repeated until all layers are calculated.

[0030] Preferably, during the data preprocessing in step 1, save the mean and standard deviation calculated during the normalization process. In step 6, perform an inverse normalization operation on the normalized result, and the formula is as follows: .

[0031] Compared with the prior art, the beneficial effects achieved by the present invention are as follows: Through dynamic graph construction and sparse adjacency selection, the computational complexity is significantly reduced, enabling the model to efficiently process large-scale road network data; Through the sparse causal attention mechanism, the model's ability to model dynamic spatio-temporal dependencies and causal relationships is enhanced, improving the prediction accuracy.

[0032] In addition, the introduction of Laplacian positional encoding further enhances the model's ability to represent spatial features and better adapt to complex road network structures; At the same time, the combination of traffic flow subsequence and time subsequence embedding enables the model to capture both the periodic patterns and dynamic changes of traffic flow, making it applicable to diverse urban traffic scenarios, such as morning and evening rush hours, holiday travel, etc.

[0033] The present invention realizes efficient modeling and accurate prediction of traffic flow data through a dynamic sparse causal attention network, not only improving the prediction accuracy and robustness, but also reducing the computational cost. It has high practical value and promotion potential and is applicable to real-time traffic management, route planning, and congestion mitigation in intelligent transportation systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation to the present invention.

[0035] Figure 1 is a flowchart of the method of the present invention.

[0036] Figure 2 is a structural diagram of the DSCFormer model of the present invention.

[0037] Figure 3 is a structural diagram of the sparse causal attention module of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0038] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of them. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0039] To make the above objects, features, and effects of the present invention more obvious and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0040] Embodiment 1: A traffic flow prediction method based on a dynamic sparse causal attention network, as Figure 1As shown, it includes the following steps: Step 1, data collection and preprocessing, including the following operations: Step 1.1, collect road network traffic flow data, and the data is sourced from the urban traffic sensor network, including the traffic flow data and time feature data of road nodes. The collected original data is , where represents the batch size, represents the time step, represents the number of nodes in the road network, represents the feature dimension, usually including flow values, speed values, and time-related features.

[0041] Step 1.2, clean the collected original data to remove outliers and missing values. For missing values, use the linear interpolation method to fill them; for outliers, detect them through the threshold statistical method based on the mean and standard deviation, and replace them with the average value of adjacent time steps to ensure the integrity and reliability of the data.

[0042] Furthermore, perform normalization on the cleaned data. To calculate the mean and standard deviation of the data and standardize the data to eliminate the influence of different feature dimensions, the formula is as follows: , where is the mean of the data, is the standard deviation of the data, is a small constant to avoid division by zero error.

[0043] Step 1.3, save the mean and standard deviation calculated during the normalization process for inverse normalization operations in the subsequent prediction stage to ensure that the prediction results can be restored to the original data range.

[0044] Furthermore, perform a dimension check on the normalized data to ensure that the shape of the data meets the model input requirements. The specific method is to verify whether the dimension of the data is , and adjust the dimension of the data that does not meet the requirements.

[0045] Step 1.4, divide the normalized data into a training set and a test set to support the training and evaluation of the model. The specific method is to, in chronological order, use the first of the data as the training set and the last of the data as the test set to ensure the time continuity of the data.

[0046] Furthermore, perform batch division on the training set and the test set respectively to support the training method of mini-batch gradient descent. The specific method is to group the data according to the batch size to generate batch data ; extract time feature data , including minute features and week features, to support subsequent time series embedding operations.

[0047] Furthermore, normalize the time feature data to ensure that its numerical range is consistent with the traffic flow data. The specific method is to map the minute features to , and map the week features to , and perform standardization operations.

[0048] Step 1.5, store the preprocessed data and save it in tensor format to improve data loading efficiency and provide reliable input data for subsequent steps.

[0049] Step 2, dynamic graph construction, including the following operations: Step 2.1, extract time window data from the normalized traffic flow data to capture the dynamic spatio-temporal dependence relationships between nodes. The specific method is to use a sliding window to extract the data of the most recent time steps to generate window data .

[0050] Furthermore, centralize and standardize the adjusted window data to eliminate numerical biases. The specific method is to calculate the mean and standard deviation of the window data and standardize the data: , where and are the mean and standard deviation of the window data respectively.

[0051] Step 2.2, calculate the Pearson correlation matrix based on the centralized data to capture the dynamic correlations between nodes: , where is the time step, is 's transposed matrix.

[0052] Furthermore, numerically correct the correlation matrix by replacing the invalid values (NaN and infinity) in the matrix with 0 to ensure numerical stability: , through the above operations, the problem of numerical overflow in subsequent calculations is avoided.

[0053] Step 2.3, normalize the corrected correlation matrix to generate a dynamic adjacency matrix, the formula is as follows: , through the softmax operation, ensure that the sum of each row of the adjacency matrix is 1, which conforms to the characteristics of the probability distribution.

[0054] Furthermore, perform a dimension check on the generated dynamic adjacency matrix to ensure that its shape is , and verify whether its numerical range is within to support the calculation of subsequent attention mechanisms.

[0055] In addition, calculate the average degree of the dynamic adjacency matrix, that is, the sum of the elements of the matrix divided by the number of nodes, to support subsequent sparse adjacency selection: , through the above operations, this step completes the construction of the dynamic graph, generates an adjacency matrix that can adaptively capture the dynamic spatio-temporal dependence relationship between nodes, and provides important input data for the subsequent sparse causal attention mechanism.

[0056] Step 3, as Figure 2 shown, perform flow subsequence embedding and time subsequence embedding on the normalized traffic flow data to extract the spatial features and time dynamic features of the traffic flow, including the following operations: Step 3.1, through the slicing operation, divide the time series into multiple subsequences to capture local time patterns.

[0057] Step 3.2, apply the slicing operation to generate flow subsequences. The specific method is to use the Unfold operation to slice the time dimension. The formula is as follows: , where is the slice length, is the step size, and the generated .

[0058] Step 3.3, add positional encoding to retain the order information of the time series and enhance the model's perception ability of time dynamics. The formula for adding positional encoding is as follows: , where Linear represents the linear transformation layer, and PE represents the positional encoding function, which is generated using the sine function to capture the relative positional relationship in the sequence.

[0059] Step 3.4, extract time embedding features from the input data, including historical minute features and historical week features , future minute features and future week features , which are used to capture the periodic patterns of the traffic flow. Input the minute features and week features into the corresponding embedding layers for embedding processing respectively to generate the corresponding embedding vectors, and concatenate the two: , where and are the embedding layers for minutes and weeks respectively.

[0060] Furthermore, adjust the dimension of the time embedding features to match the dimension of the flow embedding features. The specific method is to adjust the dimension of to through linear transformation, where D is the embedding dimension, to ensure that the two features can be fused.

[0061] Step 3.5, fuse the traffic subsequence embedding and the time subsequence embedding to generate comprehensive spatio-temporal features. The specific method is to merge the two embedding features through an element-wise addition operation: .

[0062] Furthermore, normalize the fused features to stabilize subsequent calculations. The specific method is to apply RMS normalization, and the formula is as follows: , where is the reciprocal of the square root, is the variance, is a small constant, is a learnable weight.

[0063] Through the above operations, this step completes the embedding of the traffic subsequence and the time subsequence, generating comprehensive embedding features that can capture both spatial and temporal features, providing high-quality input data for the subsequent sparse causal attention mechanism.

[0064] Step 4, the sparse causal attention mechanism, as Figure 3 shown, includes the following operations: Step 4.1, based on the dynamic adjacency matrix generated in Step 2, perform sparse adjacency selection to reduce computational complexity and focus on important spatio-temporal dependencies. The specific method is to calculate the average degree of the adjacency matrix and determine the number of nodes for sparse adjacency. The formula is as follows: ; , where is the number of nodes, is the maximum neighbor ratio.

[0065] Step 4.2, use neighbor selection to generate sparse key and value matrices. The specific method is to extract the neighbor indices corresponding to the maximum values from the dynamic adjacency matrix and construct sparse matrices based on these indices to reduce redundant calculations.

[0066] Furthermore, calculate the sparse causal attention to capture dynamic spatio-temporal dependencies. The specific method is to calculate the attention weights based on the query matrix , the sparse key matrix and the sparse value matrix . The formula is as follows: , where is the dimension of the key, is the causal mask matrix, which is used to ensure that the attention mechanism follows the causality of the time series: .

[0067] Step 4.3, for performing self-attention calculation on the input features in the time dimension, the formula is as follows: .

[0068] Step 4.4, through a weighted fusion operation, fuse the sparse causal attention and time attention to generate comprehensive spatio-temporal features. The formula is as follows: , where is a learnable parameter, and are the outputs of the sparse causal attention and time attention respectively.

[0069] Furthermore, perform multi-head attention calculation on the fused features to enhance the expression ability of the model. The specific method is to divide the input features into multiple heads, calculate the attention weights respectively, and concatenate the results: ; , where is the number of attention heads, , , and are learnable projection matrices.

[0070] Step 4.5, perform normalization on the multi-head attention output to stabilize subsequent calculations. The specific method is to apply RMS normalization, and the formula is the same as the normalization method in Step 3 to ensure the numerical stability of the features.

[0071] Furthermore, adjust the dimension of the calculation result of the sparse causal attention mechanism to match the input requirements of the subsequent encoding block. The specific method is to adjust the dimension of the output features to to support multi-layer stacking.

[0072] Through the above operations, this step completes the construction of the sparse causal attention mechanism, generates features that can capture dynamic spatio-temporal dependencies, and provides important input data for subsequent model training.

[0073] Step 5, generate Laplacian position encoding, including the following operations: Step 5.1, use the Laplacian matrix of the road network ( represents the Laplacian feature dimension), generate spatial position encoding through linear transformation to enhance the model's representation ability of space. The formula is as follows: .

[0074] Furthermore, add the spatial position encoding to the time embedding features to enhance the model's perception ability of the road network spatial structure. The specific addition method is as follows: .

[0075] Through the element-wise addition operation, the model can fuse spatial topological information.

[0076] Step 5.2, normalize the added features to stabilize subsequent calculations. The specific method is to apply RMS normalization, and the formula is the same as the normalization method in Step 3.

[0077] Furthermore, save the generated in tensor format and check the consistency of the data dimensions to ensure it can be correctly input into subsequent modules.

[0078] This step completes the construction of Laplacian positional encoding, generates encoding features that can enhance the spatial feature representation ability, and provides important input data for subsequent model training and prediction.

[0079] Step 6, model training and prediction, includes the following operations: Step 6.1, input the embedding features generated in Step 5 into this model, and extract high-dimensional spatio-temporal features through a multi-layer spatio-temporal encoder to capture complex spatio-temporal patterns.

[0080] Step 6.2, in each spatio-temporal encoding block, first apply the sparse causal attention mechanism to update the feature representation. The specific method is to update the features based on the attention calculation in Step 4, and the formula is as follows: , where represents stochastic depth regularization, which is used to improve the generalization ability of the model.

[0081] Furthermore, use a feed-forward network (FFN) to further extract features and enhance the expressive ability of the model. The specific method is to update the features through two-layer linear transformation and activation function: , where , , are linear transformation matrices, is the activation function.

[0082] Step 6.3, update the feed-forward network, and the specific method is as follows: , where represents RMS normalization, which is used to stabilize the training process.

[0083] Furthermore, through the stacking of multi-layer spatio-temporal encoding blocks, gradually extract high-dimensional spatio-temporal features. The specific method is to take the output of each layer as the input of the next layer, and repeat the above attention calculation and feed-forward network update operations until all layers of calculations are completed.

[0084] Step 6.4, the extracted high-dimensional features Flatten and map it to the prediction dimension through a linear transformation to generate the normalized prediction result. Specifically as follows: ; , where is the reshaping operation, and the flattened feature dimension is the total number of all time and nodes, is the high-dimensional feature after flattening.

[0085] Furthermore, perform an inverse normalization operation on the normalized result to restore it to the original data range. The specific method is to use the mean and standard deviation saved in step 1 to inverse normalize the prediction result, and the formula is as follows: .

[0086] Furthermore, during the model training process, use the mini-batch gradient descent method for optimization, and update the model parameters through multiple rounds of iteration to improve the prediction accuracy. The specific method is to calculate the loss and update the weights based on the training set data.

[0087] Through the above operations, this step completes the training and prediction of the model, generates the prediction result of future traffic flow, and provides reliable technical support for the real-time application of the intelligent transportation system.

[0088] The following uses experimental examples to verify the effectiveness of the traffic flow prediction method DSCFormer based on the dynamic sparse causal attention network. This experiment aims to evaluate the performance of the method of the present invention in the traffic flow prediction task through a comparative experiment, and verify its advantages in capturing dynamic spatio-temporal dependence relationships and improving prediction accuracy.

[0089] Specifically, the experiment selects the PeMS08 dataset provided by the California Traffic Measurement System. The time range of the dataset is from July 1, 2016 to August 31, 2016, with a total of 60 days of traffic flow data. The data contains 170 detectors, and the traffic flow data is collected every 5 minutes, covering traffic patterns on weekdays and holidays.

[0090] Furthermore, preprocess the collected data to ensure data quality. The specific method is as follows: fill in the missing values by linear interpolation, detect the outliers based on the thresholds of the mean and standard deviation and replace them with the average value of the adjacent time steps, and extract the minute and week features to support time series embedding.

[0091] Furthermore, select 6 baseline methods for comparative experiments, namely: ARIMA, SVR, FC-LSTM, STGCN, GMAN, and ASTGNN. The following evaluation metrics are used in the experiment: mean absolute error (MAE), root mean square error (RMSE), and mean absolute percentage error (MAPE) to comprehensively evaluate the prediction performance of the model.

[0092] Specifically, the baseline methods are selected based on their extensive applications in the field of traffic flow prediction: ARIMA is a classical time series method; SVR uses support vector machines for regression; FC-LSTM combines long short-term memory networks; STGCN and GMAN are based on graph convolutional networks; ASTGNN combines attention mechanisms and graph neural networks.

[0093] The experimental results are shown in the following table. The comparison of the prediction performance of each method on the D4 dataset indicates that the DSCFormer of the present invention is superior to the baseline methods in all metrics.

[0094] Method MAE RMSE MAPE(%) ARIMA 25.73 45.30 19.88 SVR 24.99 36.05 17.39 FC-LSTM 24.71 34.66 18.50 STGCN 18.01 11.32 28.44 GMAN 14.66 22.89 9.47 ASTGNN 12.72 22.60 8.78 DSCFormer 12.07 20.33 8.58

[0095] The analysis of the experimental results shows that DSCFormer achieved performance of 12.07, 20.33, and 8.58% in MAE, RMSE, and MAPE respectively. Compared with the optimal baseline method ASTGNN, MAE decreased by 5.1%, RMSE decreased by 11.1%, and MAPE decreased by 2.2%. This indicates that the proposed method has significant advantages in capturing dynamic spatio-temporal dependencies and improving prediction accuracy.

[0096] In summary, the experimental results verify the superior performance of DSCFormer in traffic flow prediction tasks. Through dynamic graph construction, sparse causal attention mechanism, and Laplacian positional encoding, it can effectively capture complex spatio-temporal patterns and is applicable to real-time traffic management, route planning, and congestion mitigation in intelligent transportation systems.

[0097] Embodiment 2: The computer-readable storage medium of this embodiment stores a computer program, which when executed by a processor implements the steps in a traffic flow prediction method based on a dynamic sparse causal attention network in Embodiment 1.

[0098] The computer-readable storage medium of this embodiment can be an internal storage unit of the terminal, such as the hard disk or memory of the terminal; the computer-readable storage medium of this embodiment can also be an external storage device of the terminal, such as a plug-in hard disk, smart memory card, secure digital card, flash card, etc. equipped on the terminal; further, the computer-readable storage medium can also include both the internal storage unit and the external storage device of the terminal.

[0099] The computer-readable storage medium of this embodiment is used to store the computer program and other programs and data required by the terminal. The computer-readable storage medium can also be used to temporarily store the data that has been output or will be output.

[0100] Example 3: The computer device in this example includes a processor, a memory, and a computer program stored on the memory and executable on the processor. When the processor executes the program, it implements the steps in the traffic flow prediction method based on a dynamic sparse causal attention network in Example 1.

[0101] In this example, the processor can be a central processing unit, or it can be other general-purpose processors, digital signal processors, application-specific integrated circuits, field-programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc.; the memory can include read-only memory and random access memory, and provide instructions and data to the processor. A part of the memory can also include non-volatile random access memory. For example, the memory can also store information about the device type.

[0102] Those skilled in the art can clearly understand that each implementation can be realized by means of software plus a necessary general hardware platform, and of course, it can also be realized by hardware. Based on this understanding, the above technical solution, in essence, or the part that contributes to the prior art can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disc, etc., and includes several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0103] The above are only the preferred embodiments of the present invention and are not used to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, for those skilled in the art, they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A traffic flow prediction method based on a dynamic sparse causal attention network, characterized in that, It includes the following steps: Step 1: Collect road network traffic flow data and perform preprocessing and normalization operations on the data; Step 2: Construct a dynamic graph structure and generate a dynamic adjacency matrix using a sliding window and Pearson correlation analysis; Step 3, extract the spatial features and temporal features of traffic flow through flow subsequence embedding and time subsequence embedding respectively; Step 4, construct a spatio-temporal prediction model based on a sparse causal attention mechanism, integrating a dynamic graph and a causal attention mechanism, including the following operations: Step 4.1, based on the dynamic adjacency matrix generated in Step 2 , calculate the average degree of the adjacency matrix, and determine the number of nodes with sparse adjacency according to the maximum neighbor ratio , the formula is as follows: where is the number of nodes, is the maximum neighbor ratio; Step 4.2, extract from the dynamic adjacency matrix the neighbor indices corresponding to the maximum values, and construct a sparse matrix based on these indices; then based on the query matrix , the sparse key matrix and the sparse value matrix , calculate the sparse causal attention, and the formula is as follows: Among them, is the dimension of the key, is the causal mask matrix, which is used to ensure that the attention mechanism follows the causality of the time series: Step 4.3 is to perform self-attention calculation on the input features in the time dimension, and the formula is as follows: Among them, , and are the query, key, and value matrices in the time dimension respectively; Step 4.4 is to fuse the sparse causal attention and time attention through a weighted fusion operation to generate comprehensive spatio-temporal features, and the formula is as follows: Among them, is the weight parameter, and are the outputs of the sparse causal attention and time attention respectively; Step 5 is to enhance the spatial feature representation ability using Laplacian positional encoding; Step 6 is to input the preprocessed data into the model for training, and use the trained model to predict future traffic flows.

2. The traffic flow prediction method based on a dynamic sparse causal attention network according to claim 1, wherein, The process of Step 1 data preprocessing is specifically as follows: The original data collected is , where represents the batch size, represents the time step, represents the number of nodes in the road network, represents the feature dimension, including traffic features and time features; Perform normalization operations on the data. To calculate the mean and standard deviation of the data and perform standardization processing on the data to eliminate the influence of different feature dimensions, the formula is as follows: Among them, is the mean value of the data, is the standard deviation of the data, is a small constant to avoid division-by-zero errors.

3. A traffic flow prediction method based on a dynamic sparse causal attention network according to claim 2, characterized in that Step 2 for constructing the dynamic graph is specifically as follows: In Step 2.1, use a sliding window to extract the time window data from the normalized traffic flow data where the window size is , and perform standardization processing on the window data: Among them, and are the mean and standard deviation of the window data respectively; Step 2.2, calculate the Pearson correlation matrix based on the window data after annotation: Among them, is the transpose matrix of Step 2.3, normalize the correlation matrix to generate a dynamic adjacency matrix, with the formula as follows: Through the softmax operation, ensure that the sum of each row of the adjacency matrix is 1.

4. A traffic flow prediction method based on a dynamic sparse causal attention network according to claim 2, characterized in that The embedding of the traffic subsequence described in step 3 is specifically as follows: for the normalized traffic flow data perform a slicing operation to generate traffic subsequences, and the formula is as follows: Among them, is the shard length, is the step size, and the generated , where P is the number of subsequences; map the traffic subsequences to the embedding space through linear transformation, and add positional encoding to retain the positional information of the time series. The formula is as follows: Among them, Linear represents the linear transformation layer, and PE represents the positional encoding function, which is generated by a sine function to capture the relative positional relationship in the sequence.

5. A traffic flow prediction method based on a dynamic sparse causal attention network according to claim 2, characterized in that, The time subsequence embedding described in step 3 is specifically as follows: Extract historical time features and future time features from the preprocessed input data, which respectively include historical minute features , historical week features , future minute features and future week features ; respectively input the minute features and week features into the corresponding embedding layers for embedding processing, input them into the corresponding embedding layers for processing, and concatenate the two to generate historical and future time embedding features: Among them, and are the embedding layers for minutes and weeks respectively.

6. A traffic flow prediction method based on a dynamic sparse causal attention network according to claim 4, characterized in that Step 5 to generate Laplacian position encoding is specifically as follows: Use the Laplacian matrix of the road network Generate spatial position encoding through linear transformation: ; Extend the spatial position encoding to the same dimension as the temporal embedding feature and perform element-wise addition: .

7. A traffic flow prediction method based on a dynamic sparse causal attention network according to claim 1, characterized in that In Step 6, model prediction extracts high-dimensional spatio-temporal features through a spatio-temporal encoder, including the following operations: Apply the sparse causal attention mechanism in Step 4 to the input features and update through a feed-forward neural network. The formula is as follows: Among them, is the input spatio-temporal feature, represents the attention mechanism in step 4, represents stochastic depth regularization, represents RMS normalization, is a feed-forward neural network; Map the extracted high-dimensional features to the prediction dimension through linear transformation to generate the normalized prediction result .

8. A traffic flow prediction method based on a dynamic sparse causal attention network according to claim 7, characterized in that, In Step 6, through the stacking of multiple spatio-temporal encoding blocks, the output of each layer is used as the input of the next layer, and the attention calculation and feed-forward network update operations are repeated until all layer calculations are completed.

9. A traffic flow prediction method based on a dynamic sparse causal attention network according to any one of claims 2-8, characterized in that Step 1: Save the mean value calculated during the normalization process during data preprocessing and the standard deviation , Step 6 performs an inverse normalization operation on the normalized result, and the formula is as follows: 。

Images