A traffic flow prediction method based on a dynamic sparse causal attention network

Abstract

The present invention proposes a traffic flow prediction method based on a dynamic sparse causal attention network, comprising the following steps: collecting road network traffic flow data, and performing preprocessing and normalization operations on the data; constructing a dynamic graph, and generating a dynamic adjacency matrix by using a sliding window and Pearson correlation analysis; respectively extracting the spatial features and temporal features of the traffic flow through flow subsequence embedding and time subsequence embedding; constructing a spatio-temporal prediction model based on a sparse causal attention mechanism, and integrating the dynamic graph and the causal attention mechanism; enhancing the spatial feature representation ability by using Laplacian positional encoding; inputting the preprocessed data into the model for training, and using the trained model to predict the future traffic flow. By introducing a dynamic graph structure, a sparse causal attention mechanism and Laplacian positional encoding, and combining the embedding of flow subsequences and time subsequences, the present invention constructs an efficient spatio-temporal prediction model to meet the requirements of the intelligent transportation system for high-precision prediction.

Description

Technical Field

[0001] The present invention relates to the field of intelligent transportation, and specifically to a traffic flow prediction method based on a dynamic sparse causal attention network. Background Art

[0002] With the continuous acceleration of the global urbanization process, the urban population has grown rapidly, and the number of motor vehicles has shown an explosive growth trend. The problem of urban traffic congestion has become increasingly serious. Especially during the morning and evening rush hours and holidays, traffic congestion not only leads to low travel efficiency but also exacerbates air pollution and energy consumption, posing a huge challenge to the sustainable development of cities. As an effective means to solve this problem, the intelligent transportation system (ITS) provides important support for alleviating congestion and improving road use efficiency by real-time monitoring and optimizing traffic flow. As the core component of the intelligent transportation system, traffic flow prediction can provide accurate traffic information for travelers, support navigation systems to optimize route planning, and at the same time help traffic management departments dynamically adjust signal control strategies, reduce the waiting time at traffic lights, and achieve "green wave passing", thereby effectively alleviating traffic congestion, reducing accident risks, and improving overall traffic efficiency. Therefore, developing efficient and accurate traffic flow prediction methods not only has important practical significance but also has a profound impact on promoting the construction of smart cities and achieving sustainable development of transportation.

[0003] Traffic flow prediction is a highly challenging task, and its core difficulty lies in the complex spatio-temporal correlation of traffic flow data. Traffic flow data usually presents as graph data with a non-Euclidean structure, and its spatial dependence relationship shows dynamic changes due to factors such as the road network topology, road grade, weather changes, emergencies (such as traffic accidents or construction), and holiday travel patterns. At the same time, in the time dimension, traffic flow data shows obvious periodic patterns (such as the morning and evening rush hours on weekdays, travel patterns on weekends) and long-term trends (such as traffic flow fluctuations during holidays). In addition, traffic flow data is also disturbed by external factors. For example, sudden weather changes may lead to a sharp reduction in traffic flow, and emergencies may cause abnormal fluctuations in traffic flow on local sections, which further increases the complexity of prediction. Traditional traffic flow prediction methods, such as autoregressive integrated moving average (ARIMA) and Kalman filtering, are mainly based on linear assumptions and require data to be stationary. However, actual traffic flow data is usually non-linear and is subject to non-linear interference from various external factors, which makes it difficult for traditional time series analysis methods to capture complex spatio-temporal patterns, and the prediction accuracy often fails to meet actual needs.

[0004] With the development of data mining and machine learning technologies, machine learning-based methods have been introduced into the field of traffic flow prediction. For example, Vector Auto-Regression (VAR) and Support Vector Regression (SVR) can, through feature engineering and modeling, reveal the non-linear relationships in traffic data to a certain extent, and predict traffic trends by continuously adjusting and optimizing model parameters. However, machine learning methods usually require professionals to conduct complex feature design and model tuning. The modeling process is rather cumbersome and the computational cost is relatively high. More importantly, most of the above methods only focus on the data changes in the time dimension, ignoring the dependencies in the spatial dimension, such as the traffic flow transfer and mutual influence between different nodes in the road network, which limits their prediction performance in complex road network scenarios. In addition, when dealing with large-scale datasets, machine learning methods often face the problem of the curse of dimensionality, with insufficient generalization ability of the model and difficulty in adapting to dynamically changing traffic scenarios.

[0005] In recent years, the rapid development of deep learning technologies has provided new solutions for traffic flow prediction. Deep learning models can automatically extract complex features from data, which has attracted wide attention in the field of traffic flow prediction. Methods based on Recurrent Neural Networks (RNNs), such as Long Short-Term Memory networks (LSTMs) and Gated Recurrent Units (GRUs), can capture certain non-linear time dependencies by modeling time series data, achieving better results compared to traditional time series analysis and machine learning methods. To further learn the spatio-temporal correlations of traffic data simultaneously, researchers have proposed methods that fuse Convolutional Neural Networks (CNNs) and RNNs, using CNNs to extract spatial features and combining RNNs to model time dependencies. However, CNNs cannot effectively process road network data with non-Euclidean structures, resulting in the loss of spatial information. For this reason, Graph Convolutional Networks (GCNs) have been introduced into the field of traffic flow prediction. By modeling spatial dependencies through graph structures and combining with Temporal Convolutional Networks (TCNs) to capture long-term time dependencies, the prediction performance has been significantly improved. In recent years, Transformer models based on self-attention mechanisms have also begun to be applied to traffic flow prediction. Their highly parallel self-attention mechanisms can adaptively capture long-term time dependencies, avoiding the problem of gradient vanishing in RNN models. In addition, some studies have further enhanced the model's ability to capture dynamic spatial correlations by constructing adaptive adjacency matrices.

[0006] Although the above methods have made remarkable progress in traffic flow prediction, the existing technologies still face many challenges. First, the spatial correlation of traffic flow is dynamically changing, affected by factors such as road structure, weather changes, and emergencies. Traditional static adjacency matrices or simple adaptive adjacency matrices are difficult to accurately adapt to this dynamicity, resulting in insufficient modeling ability of the model for spatial dependence. Second, when dealing with large-scale road network data, existing models often ignore sparse modeling due to high computational complexity. For example, the correlation between most nodes in the road network is weak, and forcibly modeling all node relationships will significantly increase the computational burden and reduce the model efficiency. Third, existing methods generally lack modeling of causal relationships and are difficult to accurately capture the causal laws of traffic flow changes. For example, whether a sudden increase in the flow of a certain node is caused by congestion on the upstream section. The lack of such causal relationships limits the interpretability and prediction accuracy of the model. In addition, the insufficient representation ability of spatial features is also a key issue. Traditional methods usually rely on simple graph embeddings or position encodings and are difficult to fully express the spatial structure features of complex road networks, such as the spatial distribution and topological characteristics of nodes in the road network. Finally, for the prediction of long historical data, existing models still have limitations in capturing long-term time dependencies. For example, although TCN achieves a large receptive field through dilated convolutions, the stacked convolutional layers may cause information loss and affect the model performance. Therefore, there is an urgent need for a traffic flow prediction method that can efficiently capture dynamic spatio-temporal correlations, take into account sparse and causal modeling, and improve the representation ability of spatial features to meet the requirements of intelligent transportation systems for high-precision predictions. Summary of the Invention

[0007] The present invention proposes a traffic flow prediction method based on a dynamic sparse causal attention network, aiming to solve the problems of insufficient capture of dynamic spatio-temporal correlations, sparse modeling, causal modeling, and representation ability of spatial features in existing traffic flow prediction methods. By introducing a dynamic graph structure, a sparse causal attention mechanism, and Laplacian position encoding, combined with the embedding of traffic flow subsequences and time subsequences, an efficient spatio-temporal prediction model is constructed to improve the accuracy and robustness of traffic flow prediction and meet the requirements of intelligent transportation systems for high-precision predictions.

[0008] A traffic flow prediction method based on a dynamic sparse causal attention network includes the following steps: Step 1, collect road network traffic flow data and perform preprocessing and normalization operations on the data.

[0009] Step 2, construct a dynamic graph structure and generate a dynamic adjacency matrix using a sliding window and Pearson correlation analysis.

[0010] Step 3, extract the spatial features and time features of traffic flow through traffic flow subsequence embedding and time subsequence embedding respectively.

[0011] Step 4: Construct a spatio-temporal prediction model based on a sparse causal attention mechanism, integrating a dynamic graph and a causal attention mechanism.

[0012] Step 5: Use Laplacian positional encoding to enhance the spatial feature representation ability.

[0013] Step 6: Input the preprocessed data into the model for training, and use the trained model to predict future traffic flows.

[0014] Preferably, the process of data preprocessing in Step 1 is specifically as follows: The original data collected is , where represents the batch size, represents the time step, represents the number of nodes in the road network, represents the feature dimension, including traffic features and time features; perform a normalization operation on the data, calculate the mean and standard deviation of the data, and perform standardization processing on the data to eliminate the influence of different feature dimensions. The formula is as follows: , where, is the mean of the data, is the standard deviation of the data, is a small constant to avoid division by zero errors.

[0015] Preferably, the construction of the dynamic graph in Step 2 is specifically as follows: Step 2.1, use a sliding window to extract the time window data from the normalized traffic flow data , is the window size, and perform standardization processing on the window data: , where, and are the mean and standard deviation of the window data respectively.

[0016] Step 2.2, calculate the Pearson correlation matrix based on the labeled window data: , where, is 's transposed matrix.

[0017] Step 2.3, perform normalization processing on the correlation matrix to generate a dynamic adjacency matrix. The formula is as follows: , through the softmax operation, ensure that the sum of each row of the adjacency matrix is 1.

[0018] Preferably, the embedding of the traffic subsequence in Step 3 is specifically as follows: Perform a slicing operation on the normalized traffic flow data to generate traffic subsequences. The formula is as follows: , where, is the slicing length, is the step size, and the generated , where P is the number of subsequences.

[0019] The traffic subsequences are mapped to the embedding space through a linear transformation, and positional encoding is added to preserve the positional information of the time series. The formula is as follows: , where Linear represents the linear transformation layer, and PE represents the positional encoding function, which is generated using a sine function to capture the relative positional relationships in the sequence.

[0020] Preferably, the time subsequence embedding in step 3 is specifically: extracting historical time features and future time features from the preprocessed input data, which respectively include historical minute features , historical week features , future minute features and future week features ; respectively inputting the minute features and week features into the corresponding embedding layers for embedding processing, inputting them into the corresponding embedding layers for processing, and concatenating the two to generate historical and future time embedding features: ; , where and are the embedding layers for minutes and weeks respectively.

[0021] Preferably, step 4 constructs a spatio-temporal prediction model based on a sparse causal attention mechanism, including the following operations: Step 4.1, based on the dynamic adjacency matrix generated in step 2, calculating the average degree of the adjacency matrix, and determining the number of nodes with sparse adjacency according to the maximum neighbor ratio , the formula is as follows: ; , where is the number of nodes, is the maximum neighbor ratio.

[0022] Step 4.2, extracting the neighbor indices corresponding to the maximum values from the dynamic adjacency matrix, and constructing a sparse matrix based on these indices; then based on the query matrix , sparse key matrix and sparse value matrix , calculating the sparse causal attention, the formula is as follows: , where is the dimension of the key, is the causal mask matrix, which is used to ensure that the attention mechanism follows the causality of the time series: .

[0023] Step 4.3, for performing self-attention calculation on the input features in the time dimension, the formula is as follows: , where , and are the query, key, and value matrices in the time dimension, respectively.

[0024] Step 4.4: Through weighted fusion operation, fuse the sparse causal attention and time attention to generate comprehensive spatio-temporal features. The formula is as follows: , where is the weight parameter, and are the outputs of the sparse causal attention and time attention, respectively.

[0025] Preferably, the generation of Laplacian position encoding in step 5 is specifically as follows: Use the Laplacian matrix of the road network to generate spatial position encoding through linear transformation: .

[0026] Expand the spatial position encoding to the same dimension as the time embedding feature and perform element-wise addition: .

[0027] Preferably, the model prediction in step 6 extracts high-dimensional spatio-temporal features through a spatio-temporal encoder, including the following operations: Apply the sparse causal attention mechanism of step 4 to the input features and update through a feed-forward neural network. The formula is as follows: ; , where is the input spatio-temporal feature, represents the attention mechanism of step 4, represents stochastic depth regularization, represents RMS normalization, is the feed-forward neural network.

[0028] Map the extracted high-dimensional features to the prediction dimension through linear transformation to generate the normalized prediction result .

[0029] Preferably, in step 6, through the stacking of multiple spatio-temporal encoding blocks, the output of each layer is used as the input of the next layer, and the attention calculation and feed-forward network update operations are repeated until all layers are calculated.

[0030] Preferably, during the data preprocessing in step 1, save the mean and standard deviation calculated during the normalization process. In step 6, perform denormalization on the normalized result. The formula is as follows: .

[0031] Compared with the prior art, the beneficial effects achieved by the present invention are as follows: By constructing dynamic graphs and selecting sparse adjacencies, the computational complexity is significantly reduced, enabling the model to efficiently process large-scale road network data; Through the sparse causal attention mechanism, the model's ability to model dynamic spatio-temporal dependencies and causal relationships is enhanced, improving the prediction accuracy.

[0032] In addition, the introduction of Laplacian positional encoding further enhances the model's ability to represent spatial features and better adapt to complex road network structures; At the same time, the combination of traffic flow subsequence and time subsequence embeddings enables the model to capture both the periodic patterns and dynamic changes of traffic flow, making it applicable to diverse urban traffic scenarios, such as morning and evening rush hours, holiday travel, etc.

[0033] Through the dynamic sparse causal attention network, the present invention realizes the efficient modeling and accurate prediction of traffic flow data, not only improving the prediction accuracy and robustness, but also reducing the computational cost. It has high practical value and promotion potential, and is applicable to real-time traffic management, route planning, and congestion mitigation in intelligent transportation systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The accompanying drawings are used to provide a further understanding of the present invention, and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention, but do not constitute a limitation to the present invention.

[0035] Figure 1 is a flowchart of the method of the present invention.

[0036] Figure 2 is a structural diagram of the DSCFormer model of the present invention.

[0037] Figure 3 is a structural diagram of the sparse causal attention module of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0038] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0039] To make the above objects, features, and effects of the present invention more obvious and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0040] Embodiment 1: A traffic flow prediction method based on a dynamic sparse causal attention network, as Figure 1As shown, it includes the following steps: Step 1, data collection and preprocessing, including the following operations: Step 1.1, collect road network traffic flow data, the data is sourced from the urban traffic sensor network, including traffic flow data and time feature data of road nodes. The collected original data is , where represents the batch size, represents the time step, represents the number of nodes in the road network, represents the feature dimension, usually including traffic flow values, speed values, and time-related features.

[0041] Step 1.2, clean the collected original data to remove outliers and missing values. For missing values, use the linear interpolation method to fill them; for outliers, detect them through the threshold statistical method based on the mean and standard deviation, and replace them with the average value of adjacent time steps to ensure the integrity and reliability of the data.

[0042] Furthermore, perform normalization on the cleaned data. To calculate the mean and standard deviation of the data and standardize the data to eliminate the influence of different feature dimensions, the formula is as follows: , where is the mean of the data, is the standard deviation of the data, is a small constant to avoid division by zero error.

[0043] Step 1.3, save the mean and the standard deviation calculated during the normalization process for inverse normalization operations in the subsequent prediction stage to ensure that the prediction results can be restored to the original data range.

[0044] Furthermore, perform dimension checking on the normalized data to ensure that the shape of the data meets the model input requirements. The specific method is to verify whether the dimension of the data is , and adjust the dimensions of the data that do not meet the requirements.

[0045] Step 1.4, divide the normalized data into a training set and a test set to support model training and evaluation. The specific method is to, in chronological order, use the first of the data as the training set and the last of the data as the test set to ensure the time continuity of the data.

[0046] Furthermore, perform batch partitioning on the training set and the test set respectively to support the training method of mini-batch gradient descent. The specific method is to group the data according to the batch size to generate batch data ; extract time feature data , including minute features and week features, to support subsequent time series embedding operations.

[0047] Furthermore, normalize the time feature data to ensure that its numerical range is consistent with the traffic flow data. The specific method is to map the minute features to , map the week features to , and perform standardization operations.

[0048] Step 1.5, store the preprocessed data and save it in tensor format to improve data loading efficiency and provide reliable input data for subsequent steps.

[0049] Step 2, dynamic graph construction, including the following operations: Step 2.1, extract time window data from the normalized traffic flow data to capture the dynamic spatio-temporal dependence relationships between nodes. The specific method is to use a sliding window to extract the data of the most recent time steps to generate window data .

[0050] Furthermore, centralize and standardize the adjusted window data to eliminate numerical biases. The specific method is to calculate the mean and standard deviation of the window data and standardize the data: , where and are the mean and standard deviation of the window data respectively.

[0051] Step 2.2, calculate the Pearson correlation matrix based on the centralized data to capture the dynamic correlations between nodes: , where is the time step, is 's transposed matrix.

[0052] Furthermore, numerically correct the correlation matrix by replacing invalid values (NaN and infinity) in the matrix with 0 to ensure numerical stability: , through the above operations, the problem of numerical overflow in subsequent calculations is avoided.

[0053] Step 2.3, normalize the corrected correlation matrix to generate a dynamic adjacency matrix, with the formula as follows: , through the softmax operation, ensure that the sum of each row of the adjacency matrix is 1, which conforms to the characteristics of a probability distribution.

[0054] Furthermore, check the dimensions of the generated dynamic adjacency matrix to ensure that its shape is , and verify whether its numerical range is within to support the calculation of subsequent attention mechanisms.

[0055] In addition, calculate the average degree of the dynamic adjacency matrix, that is, the sum of the elements of the matrix divided by the number of nodes, to support subsequent sparse adjacency selection: , through the above operations, this step completes the construction of the dynamic graph, generates an adjacency matrix that can adaptively capture the dynamic spatio-temporal dependence relationship between nodes, and provides important input data for the subsequent sparse causal attention mechanism.

[0056] Step 3, as Figure 2 shown, perform flow subsequence embedding and time subsequence embedding on the normalized traffic flow data to extract the spatial features and time dynamic features of the traffic flow, including the following operations: Step 3.1, through the slicing operation, divide the time series into multiple subsequences to capture local time patterns.

[0057] Step 3.2, apply the slicing operation to generate flow subsequences. The specific method is to use the Unfold operation to slice the time dimension. The formula is as follows: , where is the slicing length, is the step size, and the generated .

[0058] Step 3.3, add positional encoding to retain the order information of the time series and enhance the model's perception ability of time dynamics. The formula for adding positional encoding is as follows: , where Linear represents the linear transformation layer, and PE represents the positional encoding function, which is generated using the sine function to capture the relative position relationship in the sequence.

[0059] Step 3.4, extract the time embedding features from the input data, including the historical minute feature and the historical week feature , the future minute feature and the future week feature , which are used to capture the periodic patterns of the traffic flow. Input the minute feature and the week feature into the corresponding embedding layers for embedding processing respectively, generate the corresponding embedding vectors, and concatenate the two: , where and are the embedding layers for minutes and weeks respectively.

[0060] Furthermore, adjust the dimension of the time embedding features to match the dimension of the flow embedding features. The specific method is to adjust the dimension of to through linear transformation, where D is the embedding dimension, ensuring that the two features can be fused.

[0061] Step 3.5, fuse the traffic subsequence embedding and the time subsequence embedding to generate comprehensive spatio-temporal features. The specific method is to merge the two embedding features through an element-wise addition operation: .

[0062] Furthermore, normalize the fused features to stabilize subsequent calculations. The specific method is to apply RMS normalization, and the formula is as follows: , where is the reciprocal of the square root, is the variance, is a small constant, is a learnable weight.

[0063] Through the above operations, this step completes the embedding of the traffic subsequence and the time subsequence, generating comprehensive embedding features that can capture both spatial and temporal features, providing high-quality input data for the subsequent sparse causal attention mechanism.

[0064] Step 4, the sparse causal attention mechanism, as Figure 3 shown, includes the following operations: Step 4.1, based on the dynamic adjacency matrix generated in Step 2, perform sparse adjacency selection to reduce computational complexity and focus on important spatio-temporal dependencies. The specific method is to calculate the average degree of the adjacency matrix and determine the number of nodes for sparse adjacency. The formula is as follows: ; , where is the number of nodes, is the maximum neighbor ratio.

[0065] Step 4.2, use neighbor selection to generate sparse key and value matrices. The specific method is to extract the neighbor indices corresponding to the maximum values from the dynamic adjacency matrix and construct sparse matrices based on these indices to reduce redundant calculations.

[0066] Furthermore, calculate the sparse causal attention to capture dynamic spatio-temporal dependencies. The specific method is to calculate the attention weights based on the query matrix , the sparse key matrix and the sparse value matrix . The formula is as follows: , where is the dimension of the key, is the causal mask matrix, which is used to ensure that the attention mechanism follows the causality of the time series: .

[0067] Step 4.3, for performing self-attention calculation on the input features in the time dimension, the formula is as follows: .

[0068] Step 4.4, through a weighted fusion operation, fuse the sparse causal attention and the time attention to generate comprehensive spatio-temporal features, the formula is as follows: , where is a learnable parameter, and are the outputs of the sparse causal attention and the time attention respectively.

[0069] Furthermore, perform multi-head attention calculation on the fused features to enhance the expression ability of the model. The specific method is to divide the input features into multiple heads, calculate the attention weights respectively, and splice the results: ; , where is the number of attention heads, , , and are learnable projection matrices.

[0070] Step 4.5, perform normalization on the multi-head attention output to stabilize subsequent calculations. The specific method is to apply RMS normalization, and the formula is the same as the normalization method in Step 3 to ensure the numerical stability of the features.

[0071] Furthermore, adjust the dimension of the calculation result of the sparse causal attention mechanism to match the input requirements of the subsequent encoding block. The specific method is to adjust the dimension of the output features to to support multi-layer stacking.

[0072] Through the above operations, this step completes the construction of the sparse causal attention mechanism, generates features that can capture dynamic spatio-temporal dependencies, and provides important input data for subsequent model training.

[0073] Step 5, generate Laplacian position encoding, including the following operations: Step 5.1, use the Laplacian matrix of the road network ( represents the Laplacian feature dimension), generate spatial position encoding through linear transformation to enhance the model's representation ability of space. The formula is as follows: .

[0074] Furthermore, add the spatial position encoding to the time embedding feature to enhance the model's perception ability of the road network spatial structure. The specific addition method is as follows: .

[0075] Through element-wise addition operations, the model can fuse spatial topological information.

[0076] Step 5.2, normalize the added features to stabilize subsequent calculations. The specific method is to apply RMS normalization, and the formula is the same as the normalization method in Step 3.

[0077] Furthermore, the generated is saved in tensor format, and the consistency of the data dimensions is checked to ensure it can be correctly input into subsequent modules.

[0078] This step completes the construction of Laplacian positional encoding, generates encoding features that can enhance the spatial feature representation ability, and provides important input data for subsequent model training and prediction.

[0079] Step 6, model training and prediction, includes the following operations: Step 6.1, input the embedding features generated in Step 5 into this model, and extract high-dimensional spatio-temporal features through a multi-layer spatio-temporal encoder to capture complex spatio-temporal patterns.

[0080] Step 6.2, in each spatio-temporal encoding block, first apply the sparse causal attention mechanism to update the feature representation. The specific method is to update the features based on the attention calculation in Step 4, and the formula is as follows: , where represents stochastic depth regularization, which is used to improve the generalization ability of the model.

[0081] Furthermore, use a feed-forward network (FFN) to further extract features and enhance the expressive ability of the model. The specific method is to update the features through two-layer linear transformation and activation function: , where , , are linear transformation matrices, is the activation function.

[0082] Step 6.3, update the feed-forward network, and the specific method is as follows: , where represents RMS normalization, which is used to stabilize the training process.

[0083] Furthermore, by stacking multi-layer spatio-temporal encoding blocks, high-dimensional spatio-temporal features are gradually extracted. The specific method is to use the output of each layer as the input of the next layer, and repeat the above attention calculation and feed-forward network update operations until all layers are calculated.

[0084] Step 6.4, the extracted high-dimensional features Flattened and mapped to the prediction dimension through a linear transformation to generate the normalized prediction result. The specific process is as follows: ; , where is a reshaping operation, and the flattened feature dimension is the total number of all time steps and nodes, is the high-dimensional feature after flattening.

[0085] Furthermore, an inverse normalization operation is performed on the normalized result to restore it to the original data range. The specific method is to use the mean and standard deviation saved in step 1 to inverse-normalize the prediction result. The formula is as follows: .

[0086] Furthermore, during the model training process, the mini-batch gradient descent method is used for optimization, and the model parameters are updated through multiple rounds of iteration to improve the prediction accuracy. The specific method is to calculate the loss and update the weights based on the training set data.

[0087] Through the above operations, this step completes the training and prediction of the model, generates the prediction result of future traffic flow, and provides reliable technical support for the real-time application of the intelligent transportation system.

[0088] The following uses experimental examples to verify the effectiveness of the traffic flow prediction method DSCFormer based on the dynamic sparse causal attention network. This experiment aims to evaluate the performance of the method of the present invention in the traffic flow prediction task through comparative experiments, and verify its advantages in capturing dynamic spatio-temporal dependencies and improving prediction accuracy.

[0089] Specifically, the PeMS08 dataset provided by the California Traffic Measurement System is selected for the experiment. The time range of the dataset is from July 1, 2016 to August 31, 2016, with a total of 60 days of traffic flow data. The data contains 170 detectors, and the traffic flow data is collected every 5 minutes, covering traffic patterns on weekdays and holidays.

[0090] Furthermore, the collected data is preprocessed to ensure data quality. The specific method is as follows: linearly interpolate to fill in missing values, detect outliers based on the thresholds of the mean and standard deviation and replace them with the average of adjacent time steps, and extract minute and week features to support time series embedding.

[0091] Furthermore, 6 baseline methods are selected for comparative experiments, namely: ARIMA, SVR, FC-LSTM, STGCN, GMAN, and ASTGNN. The following evaluation metrics are used in the experiment: Mean Absolute Error (MAE), Root Mean Square Error (RMSE), and Mean Absolute Percentage Error (MAPE) to comprehensively evaluate the prediction performance of the model.

[0092] Specifically, the choice of the baseline method is based on its wide application in the field of traffic flow prediction: ARIMA is a classic time series method; SVR uses support vector machines for regression; FC-LSTM combines long short-term memory networks; STGCN and GMAN are based on graph convolutional networks; ASTGNN combines attention mechanisms and graph neural networks.

[0093] The experimental results are shown in the following table. The comparison of the prediction performance of each method on the D4 dataset shows that the DSCFormer of the present invention is superior to the baseline method in all metrics.

[0094] Method MAE RMSE MAPE(%) ARIMA 25.73 45.30 19.88 SVR 24.99 36.05 17.39 FC-LSTM 24.71 34.66 18.50 STGCN 18.01 11.32 28.44 GMAN 14.66 22.89 9.47 ASTGNN 12.72 22.60 8.78 DSCFormer 12.07 20.33 8.58

[0095] Analysis of the experimental results shows that DSCFormer achieved performance of 12.07, 20.33, and 8.58% in MAE, RMSE, and MAPE, respectively. Compared with the optimal baseline method ASTGNN, MAE decreased by 5.1%, RMSE decreased by 11.1%, and MAPE decreased by 2.2%, indicating that the proposed method has significant advantages in capturing dynamic spatio-temporal dependencies and improving prediction accuracy.

[0096] In summary, the experimental results verify the superior performance of DSCFormer in traffic flow prediction tasks. Through dynamic graph construction, sparse causal attention mechanism, and Laplacian positional encoding, it can effectively capture complex spatio-temporal patterns and is suitable for real-time traffic management, route planning, and congestion mitigation in intelligent transportation systems.

[0097] Example 2: The computer-readable storage medium of this example stores a computer program, which when executed by a processor implements the steps in a traffic flow prediction method based on a dynamic sparse causal attention network in Example 1.

[0098] The computer-readable storage medium of this example can be an internal storage unit of the terminal, such as the hard disk or memory of the terminal; the computer-readable storage medium of this example can also be an external storage device of the terminal, such as a plug-in hard disk, smart memory card, secure digital card, flash card, etc. equipped on the terminal; further, the computer-readable storage medium can also include both the internal storage unit and the external storage device of the terminal.

[0099] The computer-readable storage medium of this example is used to store the computer program and other programs and data required by the terminal. The computer-readable storage medium can also be used to temporarily store the data that has been output or will be output.

[0100] Example 3: The computer device in this example includes a processor, a memory, and a computer program stored on the memory and executable on the processor. When the processor executes the program, it implements the steps in the traffic flow prediction method based on a dynamic sparse causal attention network in Example 1.

[0101] In this example, the processor can be a central processing unit, or it can be other general-purpose processors, digital signal processors, application-specific integrated circuits, off-the-shelf programmable gate arrays, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor, or the processor can also be any conventional processor, etc.; the memory can include a read-only memory and a random access memory, and provide instructions and data to the processor. A part of the memory can also include a non-volatile random access memory. For example, the memory can also store information about the device type.

[0102] Those skilled in the art can clearly understand that each implementation can be achieved by means of software plus a necessary general hardware platform, and of course, it can also be achieved by hardware. Based on such an understanding, the above technical solution, in essence, or the part that contributes to the prior art can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each example or some parts of the examples.

[0103] The above are only the preferred embodiments of the present invention and are not used to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, for those skilled in the art, they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A traffic flow prediction method based on dynamic sparse causal attention network, characterized by: The following steps are involved: Step 1: Collect road network traffic flow data and perform preprocessing and normalization operations on the data; Step 2: Construct a dynamic graph structure and generate a dynamic adjacency matrix using sliding windows and Pearson correlation analysis; Step 3: Extract the spatial and temporal features of traffic flow through flow subsequence embedding and time subsequence embedding respectively; Step 4: Construct a spatiotemporal prediction model based on sparse causal attention mechanism, integrating dynamic graph and causal attention mechanism, including the following operations: Step 4.1: Based on the dynamic adjacency matrix generated in step 2 , calculate the average degree of the adjacency matrix, and determine the number of nodes with sparse adjacency based on the maximum neighbor ratio , the formula is as follows: in, is the number of nodes, is the maximum neighbor ratio; Step 4.2, extract from the dynamic adjacency matrix The neighbor indices corresponding to the maximum values are calculated, and a sparse matrix is constructed based on these indices; then, based on the query matrix , sparse key matrix and sparse value matrices , calculate sparse causal attention, the formula is as follows: in, is the dimension of the key, is the causal mask matrix, which is used to ensure that the attention mechanism follows the causality of the time series: Step 4.3 is to calculate the self-attention of the input features in the time dimension. The formula is as follows: in, 、 and are query, key, and value matrices of the time dimension respectively; in step 4.4, sparse causal attention and temporal attention are fused through weighted fusion operation to generate comprehensive spatiotemporal features, as shown in the following formula: in, is the weight parameter, and are the outputs of sparse causal attention and temporal attention respectively; Step 5, use Laplacian position encoding to enhance the spatial feature representation capability; Step 6, input the preprocessed data into the model for training, and use the trained model to predict future traffic flow.

2. The traffic flow prediction method based on dynamic sparse causal attention network according to claim 1 is characterized in that: The data preprocessing process in step 1 is as follows: the collected raw data is ,in Indicates the batch size, represents the time step, represents the number of nodes in the road network, Represents feature dimensions, including traffic features and time features; Normalize the data to calculate the mean and standard deviation of the data and standardize the data to eliminate the influence of different feature dimensions. The formula is as follows: in, is the mean of the data, is the standard deviation of the data, is a small constant to avoid division by zero errors.

3. The traffic flow prediction method based on dynamic sparse causal attention network according to claim 2 is characterized in that: Step 2 dynamic graph construction is as follows: Step 2.1, use the sliding window to extract the normalized traffic flow data Time window data extracted from , For the window size, normalize the window data: in, and are the mean and standard deviation of the window data respectively; Step 2.2, calculate the Pearson correlation matrix based on the labeled window data: in, for The transposed matrix of Step 2.3: Normalize the correlation matrix to generate a dynamic adjacency matrix. The formula is as follows: The softmax operation ensures that the sum of each row of the adjacency matrix is 1.

4. The traffic flow prediction method based on dynamic sparse causal attention network according to claim 2 is characterized in that: The flow subsequence embedding in step 3 is as follows: Perform sharding operations to generate traffic subsequences. The formula is as follows: in, is the fragment length, is the step size, generated , where P is the number of subsequences; the traffic subsequences are mapped to the embedding space through linear transformation, and position encoding is added to retain the position information of the time series. The formula is as follows: Among them, Linear represents the linear transformation layer, PE represents the position encoding function, which is generated by the sine function to capture the relative position relationship in the sequence.

5. The traffic flow prediction method based on dynamic sparse causal attention network according to claim 2 is characterized in that: The time subsequence embedding described in step 3 is specifically as follows: extracting historical time features and future time features from the preprocessed input data, including historical minute features , Historical Week Characteristics 、Future minute features and future week characteristics ; Input the minute feature and week feature into the corresponding embedding layer for embedding processing respectively, and concatenate the two to generate historical and future time embedding features: in, and Embedding layers for minutes and days, respectively.

6. The traffic flow prediction method based on dynamic sparse causal attention network according to claim 4 is characterized in that: Step 5 generates the Laplacian position code by using the Laplacian matrix of the road network. Generate spatial position encoding through linear transformation: ; Expand the spatial position encoding to the same dimension as the temporal embedding features and perform element-wise addition: .

7. The traffic flow prediction method based on dynamic sparse causal attention network according to claim 1 is characterized in that: Step 6: Model prediction is to extract high-dimensional spatiotemporal features through the spatiotemporal encoder. The following operations are included: Apply the sparse causal attention mechanism of step 4 to the input features and update them through a feedforward neural network. The formula is as follows: in, is the input spatiotemporal feature, represents the attention mechanism of step 4, represents stochastic depth regularization, represents RMS normalization, It is a feedforward neural network; Map the extracted high-dimensional features to the prediction dimension through linear transformation to generate normalized prediction results .

8. The traffic flow prediction method based on dynamic sparse causal attention network according to claim 7 is characterized in that: Step 6 stacks multiple layers of spatiotemporal coding blocks, uses the output of each layer as the input of the next layer, and repeats the attention calculation and feedforward network update operations until the calculation of all layers is completed.

9. A traffic flow prediction method based on a dynamic sparse causal attention network according to any one of claims 2 to 8, characterized in that: Step 1: Save the mean calculated during the normalization process during data preprocessing and standard deviation , step 6 performs denormalization on the normalized result, the formula is as follows: 。

Images