A traffic situation prediction method based on cross-attention neural network

Abstract

The present application relates to a traffic situation prediction method based on a cross-attention neural network. The method introduces a space-time cross-attention STCANN module, combines traffic flow and speed space-time blocks and cross-attention mechanisms to capture the space-time relationship and characteristics in traffic flow data. The STCANN module is composed of stacked space-time cross-attention modules and classification prediction layers, wherein each stacked space-time cross-attention module is composed of a space cross-attention block and a time cross-attention block, which can be used to extract space-time characteristics in the road network in detail, and deeper learning is achieved through stacked modules. The classification prediction layer is composed of two convolutional layers, which combine the space-time characteristics of the last cross-attention module to realize multi-step prediction, and the final output is the classified road service level. The method exhibits uniqueness, innovation and superiority in the traffic situation prediction task.

Description

Technical Field

[0001] This invention relates to the fields of deep learning, neural networks, and traffic situation prediction, and specifically to a traffic situation prediction method that integrates a deep learning model. Background Technology

[0002] With the deepening of urbanization, the number of urban motor vehicles and the total traffic volume are increasing year by year. Correspondingly, the carrying capacity of urban roads is relatively weak, and infrastructure development is lagging behind, further increasing road pressure and travel pressure on people. Therefore, accurate traffic condition assessment is crucial for urban planning and traffic management. Traditional traffic condition assessment methods face certain limitations in dealing with increasingly complex traffic scenarios, thus requiring more efficient and flexible methods to improve assessment accuracy.

[0003] In recent years, with the development of deep learning technology and graph neural networks, traffic situation prediction methods incorporating deep learning models have gradually attracted researchers' attention. Deep learning models can automatically learn complex spatial and temporal dependencies through multi-layered neural network structures, better capturing the influence of various features (such as speed and flow) in traffic networks. Graph neural networks have also been shown to be more suitable for modeling the underlying graph structure of traffic data, improving the accuracy and reliability of predictions. The graph neural network model ultimately uses the level of service (LSG) to output the prediction results, compensating for the lack of research on LSG in recent years. LSG mainly reflects the service quality of roads by comprehensively considering the ratio of road speed and flow to basic capacity. In China, LSG is generally divided into six levels (Level 1 to Level 6, with increasing congestion in traffic flow states). This classification is used to describe the traffic load on highways, qualitatively describing the stages of traffic flow change from free flow and steady flow to saturated flow and forced flow.

[0004] The proposed traffic situation prediction method integrating deep learning models aims to leverage the powerful modeling capabilities of deep learning to address the shortcomings of traditional methods in traffic situation prediction. By fusing multiple deep learning models, their respective advantages can be utilized to improve the accuracy and stability of predictions. This approach has broad application prospects in urban traffic planning, road management, and user travel, providing a new approach to better understand and predict changes in traffic situations.

[0005] Existing technologies, such as patent application CN1117116048A ("Knowledge-Driven Traffic Prediction Method Based on Knowledge Representation Model and Graph Neural Network") and patent application 115830865A ("Traffic Flow Prediction Method and Device Based on Adaptive Hypergraph Convolutional Neural Network"), both discuss the use of artificial intelligence technology to predict traffic conditions. However, the main shortcomings of these technologies are:

[0006] First, both are models based on graph neural networks. A single graph neural network has limitations in handling long-term predictions, especially dynamic spatial dependencies. This invention integrates an attention mechanism with a graph neural network. The addition of the attention mechanism allows the model to dynamically focus on the importance of different parts of the input sequence. This improvement enables the neural network to better handle long sequence data and enhances the model's ability to represent complex data.

[0007] Secondly, both of the above focus on solving the prediction problem of a single traffic feature, but a single traffic feature cannot fully reflect the actual road conditions. This invention utilizes the Level of Service (LOS) as a comprehensive evaluation index of traffic flow and speed, which has advantages in assessing the operational status of the road traffic system and its ability to meet user travel needs. Summary of the Invention

[0008] This invention is a traffic situation prediction method that integrates deep learning models, and is mainly applied in the field of traffic situation prediction.

[0009] Traditional machine learning-based methods for predicting traffic conditions have limitations in terms of accuracy and handling complex road sections. To overcome these limitations, recent research trends have focused on integrating deep learning models and graph neural network methods.

[0010] Based on this, the present invention proposes a new method:

[0011] First, to overcome the limitations of traditional traffic situation prediction which only considers a single traffic feature and ignores the impact of correlations between features on the prediction results, thus failing to fully reflect the actual traffic situation, this prediction model takes two important traffic situation features, "speed" and "flow," as input, and outputs a classified road service level, which can more comprehensively reflect the actual traffic situation.

[0012] Secondly, the prediction model integrates local geographic neighborhood and global semantic neighborhood information into a self-attention mechanism, which can simultaneously capture short-term and long-term spatial correlations in traffic data. Combined with graph neural networks, this improves the model's prediction performance.

[0013] Finally, to better handle the nonlinear relationship between traffic flow and speed, this invention employs a cross-attention mechanism in both temporal and spatial attention blocks, improving the accuracy of feature extraction. It fully utilizes a combination of multi-layered neural network structures and graph-structured data to better capture changes in traffic conditions.

[0014] This method has broad application prospects in urban traffic planning, road management, and user travel, providing a new approach to better understand and predict changes in traffic conditions.

[0015] This invention innovates and improves upon traditional methods by incorporating deep learning models for traffic prediction. By introducing a cross-attention mechanism and traffic road service levels, it enhances the expressive power of features and predictive performance. This method can better capture the impact of complex road networks, traffic accidents, and other factors on traffic conditions, extract important feature details, and better model changes in traffic conditions. This method has broad application prospects in urban traffic planning, road management, and user travel, significantly promoting research and application in related fields.

[0016] This invention relates to a method for predicting traffic road service levels by integrating deep learning models. This method incorporates a module of the Spatial-Temporal Cross-Attention Neural Network (STCANN), combining flow-speed spatio-temporal blocks and cross-attention mechanisms to capture the spatio-temporal relationships and features in traffic flow data.

[0017] The STCANN module consists of stacked spatiotemporal cross-attention modules and classification prediction layers, where:

[0018] 1. Each stacked spatiotemporal cross-attention module consists of a spatial cross-attention block and a temporal cross-attention block, which can be used to extract spatiotemporal features in the road network in detail, and achieve deeper learning through stacked modules.

[0019] 2. The classification prediction layer consists of two convolutional layers, which combine the spatiotemporal features of the last cross-attention module to achieve multi-step prediction, and the final output is the classified road service level.

[0020] This method demonstrates uniqueness, innovation, and superiority in traffic situation prediction tasks.

[0021] In practical applications, 60% of the data is used for training, 20% for validation and saving the best model, and finally, 20% for testing and evaluation. This phased prediction process effectively improves prediction accuracy, verifies the model's generalization ability, and supports research and applications in the field of traffic situation prediction.

[0022] The method of this invention divides the entire dataset into three parts: training, validation, and testing. This data segmentation is intended to effectively evaluate the performance and generalization ability of deep neural network models.

[0023] During the training phase:

[0024] First, select a subset of the feature dataset, whose features are likely to be most relevant to the problem or task.

[0025] Then, these feature datasets are used to train a deep neural network model. The goal of training is to enable the model to learn the features and patterns in the dataset by adjusting the model's weights and parameters, thereby making accurate predictions on unseen data.

[0026] After each round of training, the trained deep neural network model is validated using a validation set. The validation set is a subset of data extracted from the entire dataset; it is not used in the model training process but is used to evaluate the model's performance and tune hyperparameters. Validation on the validation set allows for timely detection of issues such as overfitting or underfitting.

[0027] During the validation process, the parameters of the network model that performs best are recorded and saved. The purpose of this is to find the model that performs optimally on the validation set for use in subsequent testing phases. By saving the parameters of the best model, we can ensure that the model achieves the best predictive performance when applied to new data.

[0028] Once the model has been trained and validated, the trained network model can be used to perform detection on the remaining dataset. These datasets are usually referred to as test sets.

[0029] By evaluating the model's performance on the test set, we can determine the model's ability to generalize to unknown data.

[0030] The results of the test set are an important metric for model performance, helping to understand how well the model predicts in the real world. Attached Figure Description

[0031] Figure 1 This is a schematic diagram of the cross-attention neural network model of this method;

[0032] Figure 2 This is a schematic diagram of the structure of a spatial double-intersecting attention block;

[0033] Figure 3 This is a schematic diagram illustrating the crossover process between flow rate and velocity.

[0034] Figure 4 This is a schematic diagram of the temporal dual-cross attention module;

[0035] Figure 5 This is a structural diagram of the classification module;

[0036] Figure 6 This is a flowchart of the traffic situation prediction method provided in an embodiment of the present invention. Detailed Implementation

[0037] To better understand the technical solution of the present invention, the embodiments of the present invention will be described in detail below, and will be illustrated with reference to the accompanying drawings. It should be understood that all other embodiments that can be obtained by those skilled in the art without inventive effort are within the protection scope of the present invention.

[0038] This invention discloses a traffic situation prediction method based on a cross-attention neural network, comprising the following steps: first, constructing a prediction model; then, using collected traffic state data as input to the prediction model, and the output of the prediction model being a classified road service level.

[0039] Traffic flow data refers to speed and flow in traffic conditions; the prediction model is based on a cross-attention neural network model.

[0040] Figures 1 to 5 This is a flowchart illustrating the overall algorithm framework of the present invention. The specific steps of this prediction method are as follows:

[0041] Step 1. Based on relevant data on spatiotemporal prediction and traffic situation in deep learning, further clarify the data format and problem definition for traffic situation prediction.

[0042] First, the traffic network is defined as an undirected graph G = (Ψ, E, A), where Ψ is the set of detector nodes, |Ψ| = N is the number of sensor nodes, E is the edge set reflecting the physical connectivity between sensors, and A is the adjacency matrix. Assume that each detector in the traffic network G detects f traffic data points at the same sampling frequency, and each node generates a feature vector of length f in each time slice. Formally, using... This represents the feature of node i in dimension c∈(flow, velocity) at time t. Represent the feature of node i in all dimensions at time t. It represents the feature set of all nodes in all dimensions at time t.

[0043] Next, our goal is to, given spatial road network data G, analyze historical observations X... flow =(X flow,1 ,X flow,2 ,…,X flow,τ ) T ∈R N×f× / au and X speed =(X speed,1 ,X speed,2 ,…,Xspeed,τ ) T ∈R N×f× / au Where τ represents the total past time, our goal is to predict the road service level at future moments.

[0044] X flow X represents flow rate. speed T represents velocity; τ represents the total past time; N represents the number of detector nodes; T represents the total time. P Indicates the prediction of the future T P Road service levels at all times.

[0045] Step 2. Convert the observed traffic state X obtained in Step 1 to the current time. flow =(X flow,1 ,X flow,2 ,…,X flow,τ ) T ∈R N×f×M and X speed =(X speed,1 ,X speed,2 ,...,X speed,τ ) T ∈R N×f×M The input is in the spatiotemporal cross-attention module of the stack;

[0046] Stacked spatiotemporal cross-attention modules, such as Figure 1 As shown, it is composed of n spatio-temporal dual cross attention modules stacked together, where each spatio-temporal cross attention module is composed of a temporal cross attention block and a spatial cross attention block.

[0047] (The superscript S indicates a spatial attention block, the subscript flow indicates flow rate, the subscript i indicates the i-th spatiotemporal cross-attention module, and the superscript d indicates the spatial attention block.) G (This represents the dimensional feature, with the superscript N indicating the number of nodes and the superscript M indicating the time length.) (The subscript "speed" indicates velocity.) First, the data is input into the spatial crossattention block. The spatial multi-dimensional crossattention block, combined with the graph adjacency matrix A, extracts spatial features. and The formula is shown below:

[0048]

[0049]

[0050] This section details the spatial cross attention block, such as... Figure 2 As shown, the spatial double-cross attention block contains the following five parts:

[0051] Module 1: Position Embedding Layer

[0052] In the spatial multi-feature cross-attention module, it is necessary to inject the "positional" information of feature vectors of different dimensions into the input sequence. Specifically, the spatiotemporal embedding matrices are represented by a dictionary D. S (flow,speed)∈R N×N and D T (flow,speed)∈R M×M (S represents spatial embedding, T represents temporal embedding, N represents the number of nodes, and M represents the time length) We initialize the dictionary using a graph neighborhood matrix and a one-hot time encoding, respectively. The graph neighborhood matrix contains important connectivity and distance information for spatial dependency modeling, while the one-hot time encoding effectively injects time step information into each node. The dictionary is then updated during training. Subsequently, D... S (flow, speed) matrix and D T The (flow, speed) matrix is ​​extended along the spatiotemporal dimensions to obtain D. S (flow,speed)∈R M×N×N ,D T (flow,speed)∈R M×N×M The final encoded features are obtained. and Where F t It is a 1×1 convolutional layer.

[0053] Module 2: Fixed Graph Convolutional Layers

[0054] This algorithm learns node features by aggregating adjacency information based on learned weights and a predefined graph, thus effectively learning structure-aware node features. It uses graph convolution based on Chebyshev polynomial approximation to capture the fixed spatial dependencies of prior road topology. In this invention, A represents the adjacency matrix, X represents the input node features from the sensor's real-time traffic conditions, and T... k It is a k-th order Chebyshev polynomial, D denotes the degree matrix, and L = I n -D -1 / 2 AD -1 / 2 Represents the normalized Laplace matrix. (λ max(where L is the largest eigenvalue) is the scaled Laplacian matrix of the Chebyshev polynomial. Finally, the graph is convolved with a k-order Chebyshev polynomial to obtain the structure-aware node features.

[0055]

[0056] ( It is the i-th channel of the node characteristics, θ ij,k (Represents learnable parameters)

[0057] Module 3: Spatial Cross-Attention Layer

[0058] In transportation systems, flow rate and speed often exhibit complex relationships. To capture the hidden dependencies of flow rate and speed changes in time and space, this module employs a cross-attention mechanism to compute and learn the interdependencies of dimensional features in multiple latent high-dimensional subspaces, such as... Figure 3 As shown.

[0059] In this layer, different dimensions of the embedded features are used at each time step. and The data is projected into a high-dimensional latent subspace via a feedforward neural network. Each node is trained in three latent subspaces, including the query subspace. and Key space and and value subspace and The representations are spatial dimensional features, where f represents flow rate, s represents velocity, and N represents the number of nodes; their calculation formulas are as follows:

[0060]

[0061]

[0062]

[0063] ( and and and

[0064] (These represent the weight matrices respectively)

[0065] To capture the dependencies between different features in high-dimensional space, we perform cross-computation on the high-dimensional subspaces of different features. The specific process of cross-computation between flow rate and velocity is as follows:

[0066] Process 1: Calculate the flow characteristic X through normalization operations. flow and velocity characteristics X speedDependence on dynamic space, using and express.

[0067]

[0068]

[0069] Process 2: Deconstruct the velocity key subspace Query subspace after feature mapping to high-dimensional subspace For dynamic spatial dependency computation, and considering the speed-to-traffic dependency, we project speed onto the query subspace of the higher-dimensional subspace. Key subspace of traffic Perform dynamic cross-calculation to form... and

[0070]

[0071]

[0072] Process 3: Subspace of flow values velocity value subspace V s S The dynamic matrix representing the nodes respectively The dynamic dependency matrix representing the interaction between flow rate and velocity. Perform calculations to generate flow and speed characteristics for the new node.

[0073]

[0074]

[0075]

[0076]

[0077] Module 4: Feedforward Neural Networks

[0078] To further improve the predictive power of the learned node features, a shared feedforward neural network with non-linear activation is applied to different feature nodes to explore the interactions between feature channels and update the node features as follows:

[0079]

[0080]

[0081] ( This is the weight matrix. (for residual connections)

[0082] The obtained data, along with the original data, is fed into a multilayer feedforward neural network with a non-linear activation function to further improve the predictive ability of the nodes.

[0083]

[0084]

[0085] ( The weight matrix, (for residual connections)

[0086] Module 5: Gating Mechanism

[0087] To integrate the spatial relationships between the modules mentioned above, we added a gating mechanism to the model. First, we learn the gate g:

[0088] g = sigmoid(f S (Φ S )+f G (X G )+b)

[0089] (f s and f G (This is a fully connected layer, where b is the bias term)

[0090] Ultimately, the output of the space transformer is

[0091] Y S =gU S +(1-g)X G

[0092] Step 3. The product generated in Step 2 and and Combined, to generate input For temporal cross attention blocks, the superscript T indicates that it is a spatial attention block, the subscript flow indicates flow rate, the subscript speed indicates velocity, and the subscript i indicates the i-th temporal cross attention block.

[0093]

[0094]

[0095] The internal structure of the temporal dual-cross attention module is the same as that of the spatial dual-cross attention module, such as... Figure 4 As shown.

[0096] The input to this module is and Among them G t It is a 1×1 convolutional layer.

[0097] In this layer, the three potential subspaces trained by each node include the query subspace. and Key space and and value subspace

[0098] and The time dimension represents the feature, where f represents flow rate, s represents velocity, and M represents time duration; their calculation formulas are as follows:

[0099]

[0100]

[0101]

[0102] ( and and and (These represent the weight matrices respectively)

[0103] The specific process of the interaction between flow rate and speed is as follows: Figure 3 As shown:

[0104] Process 1: Calculate the flow characteristic X through normalization operations. flow and velocity characteristics X speed Dependence on time, using and express.

[0105]

[0106]

[0107] Process 2: Deconstruct the velocity key subspace Query subspace after feature mapping to high-dimensional subspace For dynamic spatial dependency computation, and considering the speed-to-traffic dependency, we project speed onto the query subspace of the higher-dimensional subspace. Key subspace of traffic Perform dynamic cross-calculation to form... and

[0108]

[0109]

[0110] Process 3: Subspace of flow values velocity value subspace V s τ The dynamic matrix representing the nodes respectively The dynamic dependency matrix representing the interaction between flow rate and velocity. Perform calculations to generate flow and speed characteristics for the new node.

[0111]

[0112]

[0113]

[0114]

[0115] Step 4. Input the i-th output after processing by the spatiotemporal cross-attention module into the (i+1)-th module, and so on, to form a stacked spatiotemporal cross-attention module.

[0116]

[0117]

[0118] Step 5. The final classification module integrates various features, such as... Figure 5 As shown in the figure, the final classification result is obtained through iterative learning. The outputs of the cross-attention module are stacked and fused within the classification module, resulting in an increase in length and forming a new tensor D. LOS Subsequently, tensor D LOS Dimensionality reduction is performed using two convolutional layers, and the number of categories is expanded to 6, resulting in a transformed tensor. Finally, by applying the known label tensor input to the cross-entropy layer... Through continuous training and learning, the final prediction results can be obtained.

[0119] The formulas for the two processes are shown below.

[0120]

[0121]

[0122] Then, the prediction model is trained using the mean absolute loss method. The prediction model is represented as follows:

[0123] L=∥YY gt ∥1

[0124] Step 6. Define the loss function: Based on the task type and the characteristics of the model output, adopt an appropriate loss function, such as Mean Absolute Error (MAE) or Mean Absolute Percentage Error (MAPE).

[0125] Step 7. Select an optimization algorithm: Choose an appropriate optimization algorithm based on the prediction model type and dataset size, such as stochastic gradient descent (SGD), Adam, RMSProp, etc. Adjust the initial values ​​of the learning rate and other hyperparameters to improve the model's convergence and performance.

[0126] Step 8. Train the model: Input the data into the model and calculate the model's output through forward propagation. Calculate the loss value based on the loss function and the model output. Update the model's parameters using the selected optimization algorithm. Iterate the above steps repeatedly until the predetermined stopping condition is met (e.g., reaching the maximum number of iterations or the loss function converging).

[0127] Step 9. Model Evaluation: Use a validation set or cross-validation to evaluate the model's performance. Select an appropriate evaluation metric based on the task type, such as accuracy or mean squared error. Adjust model hyperparameters, loss functions, or optimization algorithms based on the evaluation results to further improve performance.

[0128] Step 10. Model Testing and Deployment: Perform final performance testing on the trained and evaluated model using the test set. Deploy the model to a production environment, such as a local environment, server, or cloud platform, depending on actual needs. When deploying, consider factors such as model performance, reliability, and security. Dataset Partitioning: Select approximately 60% of the dataset for training the prediction model; then, after each training iteration, select approximately 20% of the dataset to validate the prediction model and save the best-performing model; finally, use the remaining approximately 20% of the dataset to test the trained prediction model and evaluate its detection performance.

[0129] In step 2, the inputs are two important features for assessing traffic conditions: speed and flow. Compared to previous single traffic prediction models, this model captures changes in traffic conditions more comprehensively, thus better predicting future road service levels.

[0130] Steps 3 and 4 introduce a cross-attention mechanism to handle the complex nonlinear relationship between two characteristic speeds and flow rates that reflect traffic conditions.

[0131] Specifically, in traffic situation prediction tasks, speed and flow are two very important traffic features. Internally, we use a cross-attention mechanism, which not only captures the spatiotemporal correlation between these two factors but also effectively analyzes and handles the nonlinear correlation between speed and flow, enabling more accurate predictions of future traffic situation changes.

[0132] In summary, by using a cross-attention mechanism in the model, we can better analyze and process two important features in traffic situation assessment: speed and flow, thereby more accurately predicting the level of service. Compared with general attention mechanisms, the cross-attention mechanism can better handle the correlation between different dimensions, helping the model to better integrate information and thus improve prediction accuracy.

[0133] Steps 3 and 4 respectively incorporate cross-attention mechanisms into the spatial and temporal cross-attention blocks to enhance the fusion of two key features, speed and flow, in traffic situation prediction.

[0134] In step 5, the output data processed by the multi-layer spatiotemporal cross-attention mechanism is input into the classification prediction layer for prediction. The classification prediction layer passes through two convolutional layers to finally achieve the classification prediction of the road service level.

[0135] The entire module constructs a complete spatiotemporal data modeling network. This connectivity structure enables the model to capture the spatiotemporal relationships and features of traffic data during the training phase and to make accurate predictions on new data during the testing phase. By selecting appropriate loss functions and optimization algorithms, the model can be effectively trained and optimized, further improving its performance and generalization ability.

[0136] In summary, the STCANN module possesses uniqueness, innovation, and superiority in traffic situation prediction. Through the interconnection of cross-functional spatiotemporal attention modules and deep learning networks, this module is able to comprehensively model the spatiotemporal relationships and characteristics of traffic data. It brings significant improvements to traffic situation prediction tasks, enhancing prediction accuracy.

[0137] Specifically, in this embodiment...

[0138] Figure 6 This is a flowchart of the traffic situation prediction method provided in this embodiment. The implementation of the present invention includes the following steps:

[0139] Step 101. To better conduct the experiment, the dataset comes from the Caltrans Performance Measurement System (PeMS), which contains over 44,681 detectors covering highways in major metropolitan areas of California, USA. Traffic data (including flow, speed, and vehicle distance traveled) is provided every 5 minutes and aggregated into a single data time slice. It contains traffic information for 12 time slices per hour and 288 time slices per day. Each detector corresponds to one node. The datasets used in this paper are PeMS04 and PeMS08, which contain three traffic features: flow, speed, and service level.

[0140] Step 102. Based on the requirements of steps 1 to 6 above, this example uses the Python programming language to build the STCANN model. The PyTorch framework was used in the development of the STCANN model.

[0141] Step 103. Select a loss function (such as Mean Absolute Error (MAE), Mean Absolute Percentage Error (MAPE), or Root Mean Square Error (RMSE)) to evaluate the degree of deviation between the predicted values ​​and observed values ​​of the prediction model, thereby measuring the accuracy of the current model. All loss functions in this invention are as follows:

[0142]

[0143]

[0144]

[0145] ( Y represents the predicted value. i (Representative observation value)

[0146] Step 104. Select an optimization function (such as stochastic gradient descent, momentum optimization, or adaptive learning rate optimization algorithm) to help the prediction model converge to the optimal or near-optimal solution and handle complex problems, so as to quickly and effectively search the parameter space and find a better solution.

[0147] Step 105. By performing multiple forward and backward propagation iterations on the training data, the prediction model can automatically adjust its parameters based on the patterns and regularities in the training data. This iterative process improves the model's ability to generalize to new data, enabling it to make accurate predictions on the input data.

[0148] Step 106. After completing each training epoch on all training data in Step 105, the trained model is validated using the validation set to assess its prediction accuracy. If the model exhibits excellent performance, proceed directly to Step 109 for evaluation. However, if the model's performance is poor, model improvement needs to be considered. In this case, proceed directly to Step 108 for model improvement. Conversely, if no further improvement is needed, proceed to Step 107.

[0149] Step 107. To find the optimal combination of parameters to improve model performance, common hyperparameters include learning rate, batch size, number of iterations, regularization parameter, and the number of layers and nodes in the neural network. Appropriate hyperparameter selection can accelerate model convergence, improve generalization ability, and prevent overfitting. After step 107, continue with step 105.

[0150] Step 108. Improve the performance of the deep learning model by adjusting the network structure, parameters, activation functions, regularization techniques, data augmentation, and optimization algorithms. During this iteration, different methods and techniques are tried, and the effects are measured using evaluation metrics. Proceed to Step 102 to rebuild the model, changing the structure, adjusting parameters, selecting appropriate activation functions, applying regularization, data augmentation, and optimization algorithms to improve generalization ability and robustness, thereby enhancing practical application performance.

[0151] Step 109. In addition to considering model accuracy, it is also necessary to evaluate the model's time and space complexity. Time complexity focuses on the computational resources required for model inference and training, including inference speed and training time. Space complexity focuses on the space occupied by the model in memory or storage, including model size and memory usage. Considering both time and space complexity helps evaluate the model's efficiency and scalability, allowing for the selection of a model suitable for specific application scenarios.

[0152] The embodiments described above are merely preferred embodiments of the present invention and do not limit the present invention in any way. Other variations and modifications may exist without departing from the technical solutions described in the claims. These variations and modifications still fall within the scope of the present invention.

Claims

1. A traffic situation prediction method based on a cross-attention neural network, comprising the following steps: First, a prediction model is constructed; then, the collected traffic state data is used as the input to the prediction model, and the output of the prediction model is the classified road service level. Its characteristics include traffic flow data consisting of speed and flow in the traffic situation; the prediction model is based on a cross-attention neural network model. The steps of the prediction method include: Step 1. Define the data format and problem definition for traffic situation prediction. First, the transportation network is defined as an undirected graph. ,in For the set of detector nodes, | |=N is the number of detector nodes, Let A be the edge set representing the physical connectivity between the detectors, and let A be the adjacency matrix. Establish a transportation network Each detector in the system detects f traffic data points at the same sampling frequency, and each node generates a feature vector of length f in each time slice. In terms of form, using Let ∈R represent the feature of node i at time t in dimension c∈(flow, velocity). ∈ Represent the feature of node i in all dimensions at time t. Represents the feature set of all nodes in all dimensions at time t; Next, given a spatial road network data G, from historical observations... and To predict road service levels at future moments X flow X represents flow rate. speed T represents velocity; τ represents the total past time; N represents the number of detector nodes; T represents the total time. P Indicates the prediction of the future T P Road service level at all times; Take the current time X obtained in step 1 flow and X speed The input is in the spatiotemporal cross-attention module of the stack; Stacked spatiotemporal cross-attention modules are composed of multiple stacked spatiotemporal cross-attention modules; Step 2. A spatiotemporal cross-attention module includes a spatial cross-attention block and a temporal cross-attention block; for the i-th spatiotemporal cross-attention module: First, the flow space input With velocity space input The inputs are respectively fed into the spatial cross-attention block; the superscript S indicates a spatial attention block, the subscript flow indicates flow rate, the subscript speed indicates velocity, the subscript i indicates the i-th spatiotemporal cross-attention block, and the superscript d... G The superscript N represents the number of nodes, and the superscript M represents the time length. Spatial cross-attention blocks, combined with adjacency matrix A, extract spatial features of both from the input. and As shown in the following formula: ， ； Step 3. Take the product generated in Step 2. , Separately and , Combined, flow-time features are generated for the input of the temporal cross-attention block. and flow time characteristics The superscript T indicates a spatial attention block, the subscript flow indicates flow rate, the subscript speed indicates velocity, and the subscript i indicates the i-th spatiotemporal cross-attention module; As shown in the following formula: ； Step 4. Input the i-th output after processing by the i-th spatiotemporal cross-attention module into the (i+1)-th spatiotemporal cross-attention module, as shown in the following formula: ， Similarly, the spatiotemporal features of flow and velocity are obtained through the processing of stacked spatiotemporal cross-attention modules. and The superscript ST indicates a spatiotemporal cross-attention module, the subscript flow indicates flow rate, and the subscript speed indicates speed. Step 5. Use the classification module to... and After being integrated, the final classification result is obtained through iterative learning; In step 2, the structure of the spatial cross-attention block includes the following five parts: Module 1: Spatial embedding layer In the spatial cross-attention block, the "positional" information of feature vectors of different dimensions is injected into the input sequence. Specifically: First, the spatiotemporal embedding matrices are respectively represented by dictionaries. and The text indicates that the words are initialized using graph adjacency matrix and one-hot time coding, respectively; the superscript S indicates spatial embedding, the superscript T indicates temporal embedding, the superscript N indicates the number of nodes, and the superscript M indicates the time length. Then, update the dictionary during training; Then, and Extending along the space-time dimension, we obtain , ; Ultimately, the encoded features are and in It is a 1 × 1 convolutional layer; Module 2: Fixed Graph Convolution Layer Based on the learned weights and a predefined graph, node features are learned by aggregating adjacency information, thereby effectively learning structure-aware node features, specifically: Graph convolution based on Chebyshev polynomial approximation is used to capture fixed spatial dependencies in the prior road topology; A represents the adjacency matrix, and X represents the input node features of the detector's real-time traffic conditions. It is a k-th order Chebyshev polynomial, and D denotes the degree matrix. Represents the normalized Laplace matrix. It is a scaled Laplace matrix of the Chebyshev polynomial. The largest eigenvalue of L is used; by approximating the graph convolution with a k-th order Chebyshev polynomial, the structure-aware node features are obtained: ， in, It is the i-th channel of the node characteristics. Represents learnable parameters; Module 3: Spatial Cross-Attention Layer A cross-attention mechanism is employed to compute and learn the interdependencies of dimensional features in multiple latent high-dimensional subspaces; in this layer, different dimensions are embedded in features at each time step. and It is projected into a high-dimensional latent subspace through a feedforward neural network; The three potential subspaces trained at each node include the query subspace. and key space and and value subspace and ; The representation represents the spatial dimensionality feature, where the subscript f represents flow rate, the subscript s represents velocity, and the superscript N represents the number of nodes; their calculation formulas are as follows: ， in, and , and , and These represent the weight matrices respectively; The specific process of cross-calculation between flow rate and velocity in high-dimensional subspaces with different characteristics is as follows: Process 1: Calculate the flow characteristics through normalization operations. and velocity characteristics Dependence on dynamic space, respectively using and express, ， Process 2: Deconstruct the velocity key subspace Query subspace after feature mapping to high-dimensional subspace Perform dynamic spatial dependency computation; for the speed-to-flow dependency, project speed onto the query subspace of a higher-dimensional subspace. Key subspace of traffic Perform dynamic cross-calculation to form... and : ， ； Process 3: Subspace of flow values Velocity value subspace The dynamic matrix representing the nodes respectively , The dynamic dependency matrix representing the interaction between flow rate and velocity. , Perform calculations to generate traffic and speed characteristics for the new node: ； Module 4: Feedforward Network A feedforward neural network with a non-linear activation function is applied to different feature nodes, and the node features are updated as follows: ， in, , This is the weight matrix. For residual connections; The obtained data, along with the original data, is fed into a feedforward neural network with a non-linear activation function to further improve the predictive ability of the nodes. ， in, The weight matrix, For residual connections; Module 5: Gating Mechanism A gating mechanism is used to integrate the spatial relationships between modules; First, learn gate g: ， in, and For a fully connected layer, b is the bias term; Ultimately, the output of Spatial Transformer Networks (STN) is... ； In the time-crossing attention module, the inputs are respectively and ;in It is a 1 × 1 convolutional layer, in which each node is trained on three latent subspaces, each including the query subspace. and key space and and value subspace and , This represents the time dimension feature, where the subscript f represents flow rate, the subscript s represents velocity, and the superscript M represents time duration; their calculation formulas are as follows: ， in, and , and , and These represent the weight matrices respectively; In step 3, the steps involving the intersection of flow rate and speed include: Process 1: Calculate the flow characteristics through normalization operations. and velocity characteristics Dependence on time, respectively using and express: ； Process 2: Deconstruct the velocity key subspace Query subspace after feature mapping to high-dimensional subspace Perform dynamic spatial dependency calculations; simultaneously, for the speed-to-traffic dependency, project speed onto the query subspace of the high-dimensional subspace. Key subspace of traffic Perform dynamic cross-calculation to form... and : ； Process 3: Subspace of flow values velocity value subspace The dynamic matrix representing the nodes respectively , The dynamic dependency matrix representing the interaction between flow rate and velocity. , Perform calculations to generate flow and speed characteristics for the new node. ； In step 5 First, the outputs of the stacked spatiotemporal cross-attention modules are stacked and fused within the stacked layers of the classification module, increasing their length to form a new tensor. , represented as ; Subsequently, tensor Dimensionality reduction and category expansion are performed using multiple convolutional layers to obtain a transformed tensor. , represented as ; Finally, by applying the known label tensor input to the Crossentropy Layer... Continuous training and learning are performed to obtain the final prediction result; the prediction model is trained using the mean absolute loss function, expressed as follows: .

2. The traffic situation prediction method based on cross-attention neural network according to claim 1 is characterized in that: firstly, about 60% of the dataset is selected for training the prediction model; then, after each training iteration, about 20% of the dataset is selected to validate the prediction model and the best-performing prediction model is saved; finally, the remaining about 20% of the dataset is used to test the trained prediction model and evaluate the detection performance of the prediction model.

3. The traffic situation prediction method based on cross-attention neural network according to claim 1, characterized in that: The training of the prediction model is as follows: first, the data is input into the prediction model, and the output of the model is calculated through forward propagation; then, the loss value is calculated based on the loss function and the output of the prediction model; finally, the parameters of the prediction model are updated using an optimization algorithm. Repeat the above steps until the predetermined stopping condition is met, i.e., the maximum number of iterations is reached or the loss function converges.

4. The traffic situation prediction method based on cross-attention neural network according to claim 1, characterized in that: The mean absolute loss function is the mean absolute error (MAE) or the mean absolute percentage error (MAPE).

5. The traffic situation prediction method based on cross-attention neural network according to claim 1, characterized in that: The optimization algorithm for the prediction model is the stochastic gradient descent (SGD) algorithm, the Adam algorithm, or the RMSProp algorithm.

6. The traffic situation prediction method based on cross-attention neural network according to claim 1, characterized in that: The methods for evaluating prediction models are as follows: use validation sets or cross-validation to evaluate the model's performance; select evaluation metrics based on the task type; and adjust the hyperparameters, loss function, or optimization algorithm of the prediction model based on the evaluation results.

7. The traffic situation prediction method based on cross-attention neural network according to claim 1, characterized in that: The testing and deployment methods for the prediction model are as follows: use a test set to perform a final performance test on the trained and evaluated prediction model; deploy the prediction model to the production environment.

Images