A traffic flow prediction method based on a Transformer adaptive adversarial graph neural network

Abstract

The application discloses a traffic flow prediction method based on a Transformer adaptive adversarial graph neural network, and comprises the following steps: performing embedding operation on original data by using a degree embedding algorithm and a distance embedding algorithm to obtain traffic flow data; inputting the traffic flow data into a TDN module of a generator to perform attention mechanism operation to obtain aggregated time information Y; taking Y as input of an SDGCN graph neural network to obtain aggregated space information Z, and then inputting Z into an MLP with two hidden layers to obtain prediction data; connecting the prediction data with the traffic flow data X, denoted as X r ; inputting X p and the real data X p into a discriminator together; calculating loss functions L r of the discriminator and the generator; calculating L d and L g again; calculating a loss function L of the whole model; and finally updating parameters of the model through stochastic gradient descent. The application improves the accuracy of traffic flow prediction and the global consistency of prediction results, and has certain advantages in convergence speed.

Description

Technical Field

[0001] This invention relates to the field of deep learning, and more particularly to a traffic flow prediction method based on a Transformer adaptive adversarial graph neural network. Background Technology

[0002] With societal development and the continuous expansion of cities, traffic congestion is increasing, impacting people's daily travel. Intelligent Transportation Systems (ITS) can alleviate traffic congestion, facilitate travel, and improve traffic safety. Traffic flow prediction, a crucial component of ITS, can accurately predict real-time traffic flow at urban road junctions, thereby assisting vehicles in making traffic decisions and identifying traffic problems on road segments, ultimately reducing road congestion.

[0003] Most existing traffic flow prediction methods are based on deep learning, but they still have some problems. First, they do not explicitly model the temporal patterns of traffic flow data, resulting in a lack of ability to capture spatiotemporal correlations. Second, they do not model key nodes in the road network, which are often important factors affecting traffic congestion.

[0004] Furthermore, road networks are constantly changing, and capturing the spatial dynamics of traffic flow remains a challenge. Finally, it is difficult to guarantee that predicted data will have the same distribution as historical data. All of these issues limit the accuracy of traffic flow prediction methods. Summary of the Invention

[0005] The technical problem to be solved by this invention is to provide a traffic flow prediction method based on Transformer adaptive adversarial graph neural network to address the shortcomings of the existing technology, and to improve the accuracy of traffic flow prediction by improving the traditional structure for traffic flow data.

[0006] The technical solution adopted by this invention to solve its technical problem is:

[0007] This invention provides a traffic flow prediction method based on Transformer adaptive adversarial graph neural network, the method comprising the following steps:

[0008] Step 1: Obtain raw data from the road network, and perform degree embedding and distance embedding operations on the raw data to obtain preprocessed traffic flow data;

[0009] Step 2: Construct a generator by inputting traffic flow data into the generator's TDN module to perform attention mechanism operations to obtain aggregated time information Y;

[0010] Step 3: Construct the SDGCN graph neural network. Use Y as the input to the SDGCN graph neural network to obtain the aggregated spatial information Z. Then, input Z into an MLP with two hidden layers to obtain the prediction data.

[0011] Step 4: Obtain real data X r Build a discriminator to analyze the predicted data. The operation of joining with traffic flow data X is denoted as X. p and X p And real data X r They are input together into the discriminator;

[0012] Step 5: Calculate the loss function L for the discriminator and generator. d and L g Then, the loss function L of the entire model is calculated, and finally the model parameters are updated by stochastic gradient descent to obtain a trained Transformer-based adaptive adversarial graph neural network model; the trained model is used to predict traffic flow.

[0013] Furthermore, the specific implementation of step 1 of the present invention includes the following sub-steps:

[0014] Step 1.1: For each node v in the road network i The degree deg(v) of each node is calculated based on the adjacency matrix A of the road network. i ), based on the degree of the node deg(v i Generate a learnable parameter Compare it with the original data x i Add them together to achieve Degree Embedding, i.e., degree embedding algorithm;

[0015] Step 1.2: Calculate node v i and v j The distance is denoted as f(v) i ,v j According to f(v) i ,v j Generate learnable parameters In calculating v i and v j Attention score S ij At that time, It is added as an offset to the calculation of the attention score, i.e. This enables distance embedding, i.e., the distance embedding algorithm.

[0016] Furthermore, the specific implementation process of step 2 of the present invention includes:

[0017] Traffic flow data X is input into two different one-dimensional convolutional modules, and the results are processed using ReLU and sigmoid functions. The results are then concatenated and used as the input I for the multi-head self-attention mechanism. The formula for this operation is as follows:

[0018]

[0019] in, Represented as a concatenation operation, ReLU and σ are activation functions, and Θ1 and Θ2 are two different one-dimensional convolution operations. Therefore, the attention mechanism operation can be described as follows:

[0020] Y = Attention(W) Q I,W K I,W V I)

[0021] Where Attention() represents the attention mechanism operation, W Q W K W V Let Q, K, and V represent the learnable weight matrices, respectively.

[0022] Furthermore, step 3 of the present invention is specifically implemented by including the following sub-steps:

[0023] Step 3.1: Construct a mask matrix C by multiplying the output Y of the TDN by its transpose, thereby representing the dynamics of the traffic road network. The specific operation is as follows:

[0024]

[0025] Where softmax() represents the normalization function, d model Where N is the vector dimension of the model, and N is the number of nodes in the road network;

[0026] Step 3.2: Incorporate the obtained mask matrix C into the graph convolution operation, along with the adjacency matrix with self-connections. The formula for performing the Hadamard product is as follows:

[0027]

[0028] Where σ is the activation function, This represents the inverse of the degree matrix with self-connections. This represents an adjacency matrix with self-connections, where ⊙ is the Hadamard product, and W and b are learnable parameters.

[0029] Step 3.3: Repeat steps 2 and 3 for the set number of layers l, and pass the final SDGCN output Z through an MLP with two hidden layers to obtain the prediction result. Represented as:

[0030]

[0031] Where W1, W2, b1, and b2 are learnable parameters.

[0032] Furthermore, the specific implementation process of step 4 of the present invention is as follows:

[0033] Predicted data Perform a concatenation operation with traffic flow data X, and record the result as X. p and X p And real data X r They are input together into the discriminator, that is:

[0034] X r =X t-T+1:t ∥X t+1:t+H

[0035] Where T represents the time step of traffic flow data, H represents the time step of prediction, and ∥ represents the stitching operation.

[0036] Furthermore, step 5 of the present invention includes the following sub-steps:

[0037] Step 5.1: Based on the output D(X) of the discriminator p ) and D(X r Calculate the loss function L of the discriminator. d The loss function L of the generator g The formula is expressed as:

[0038]

[0039] in, and X represents r and X p The expected value of the distribution;

[0040] Step 5.2: Combine L1 loss and generator loss L g The sums are the global loss L, L = L1 + L g L1 is represented as:

[0041]

[0042] Step 5.3: Update the network parameters θ of the generator and discriminator using stochastic gradient descent. g and θd The specific operation is as follows:

[0043]

[0044] Where, η g and η d These represent the learning rates of the generator and the discriminator, respectively.

[0045] The training epochs are set, meaning that when the training reaches the specified number of epochs, the model parameters are stopped from being updated. In addition, an early stopping epoch is set, meaning that if the model's performance has not improved after the early stopping epoch, training is stopped.

[0046] This invention provides a traffic flow prediction system based on a Transformer adaptive adversarial graph neural network, comprising:

[0047] Memory, used to store executable computer programs;

[0048] When a processor executes an executable computer program stored in memory, it implements the traffic flow prediction method based on the Transformer adaptive adversarial graph neural network described above.

[0049] The present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described traffic flow prediction method based on a Transformer adaptive adversarial graph neural network.

[0050] The beneficial effects of this invention are:

[0051] The purpose of this invention is to design an adversarial neural network based on the Transformer architecture and improve the traditional structure for traffic flow data, thereby improving the accuracy of traffic flow prediction.

[0052] (1) This invention designs Degree Embedding and Distance Embedding, and clearly proposes to model key nodes in the road network, thereby improving the graph representation capability of the model.

[0053] (2) This invention combines attention mechanism and one-dimensional convolutional neural network to design a time-dependent network (TDN) that captures the temporal patterns of traffic flow.

[0054] (3) Based on graph convolutional neural networks, this invention designs a mask matrix to capture the spatial dynamics of traffic flow data.

[0055] (4) The present invention designs an adversarial training method, which improves the prediction accuracy of the model and the global consistency of the prediction data. Attached Figure Description

[0056] The present invention will be further described below with reference to the accompanying drawings and embodiments. In the accompanying drawings:

[0057] Figure 1 This is a model structure diagram of an embodiment of the present invention;

[0058] Figure 2 This is a comparison chart of experimental results from an embodiment of the present invention. Detailed Implementation

[0059] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0060] like Figure 1 As shown, this invention provides a traffic flow prediction method based on a Transformer adaptive adversarial graph neural network. The method generates prediction data using a generator, and then optimizes the generator using a discriminator to improve prediction accuracy. The method is characterized by the following steps:

[0061] Step 1: Obtain raw data through the road network, and perform Degree Embedding and Distance Embedding operations on the raw data to obtain traffic flow data;

[0062] The specific implementation includes the following sub-steps:

[0063] Step 1.1: For each node v i The degree deg(v) of each node is calculated based on the adjacency matrix A of the road network. i ), based on the degree of the node deg(v i Generate a learnable parameter Compare it with the original data x i Add them together to achieve Degree Embedding;

[0064] Step 1.2: Calculate node v i and v j The distance is denoted as f(v) i ,v j According to f(v) i ,v j Generate learnable parameters In calculating v i and v j Attention score S ij At that time, It is added as an offset to the calculation of the attention score, i.e. This is how Distance Embedding is achieved.

[0065] Step 2: Input traffic flow data into the TDN module of the generator and perform an attention mechanism operation to obtain aggregated time information Y; the specific implementation process is as follows:

[0066] Traffic flow data X is input into different one-dimensional convolutional modules, and the results are processed using ReLU and Sigmoid functions. The results are then concatenated and used as the input I for the multi-head self-attention mechanism. This operation can be expressed by the following formula:

[0067]

[0068] in, Represented as a connection operation, ReLU and σ are activation functions, and Θ1 and Θ2 are two different one-dimensional convolution operations. Therefore, the attention mechanism operation can be described as follows:

[0069] Y = Attention(W) Q I,W K I,W V I)

[0070] Where Attention() represents the attention mechanism operation, W Q W K W V Let Q, K, and V represent the learnable weight matrices, respectively.

[0071] Step 3: Y is used as the input to the designed SDGCN graph neural network to obtain the aggregated spatial information Z. Then, Z is input into an MLP with two hidden layers to obtain the prediction data.

[0072] The specific implementation includes the following sub-steps:

[0073] Step 3.1: Construct a mask matrix C by multiplying the output Y of the TDN by its transpose, thereby representing the dynamics of the traffic road network. The specific operation can be expressed as follows:

[0074]

[0075] Where softmax() represents the normalization function, d model Where N is the vector dimension of the model, and N is the number of nodes in the road network;

[0076] Step 3.2: Incorporate the obtained mask matrix C into the graph convolution operation, along with the adjacency matrix with self-connections. The Hadamard product operation can be expressed by the following formula:

[0077]

[0078] Where σ is the activation function, This represents the inverse of the degree matrix with self-connections. This represents an adjacency matrix with self-connections, where ⊙ is the Hadamard product, and W and b are learnable parameters.

[0079] Step 3.3: Repeat steps 2 and 3 as set by this invention for the number of layers l times, and pass the final SDGCN output Z through an MLP with two hidden layers to obtain the prediction result. It can be represented as:

[0080]

[0081] Where W1, W2, b1, and b2 are learnable parameters.

[0082] Step 4: Prepare the predicted data Perform a concatenation operation with historical data X, and record the result as X. p and X p And real data X r They are input together into the discriminator, that is:

[0083] X r =X t-T+1:t ∥X t+1:t+H

[0084] Where T represents the time step of historical data, H represents the time step of prediction, and ∥ represents the splicing operation.

[0085] Step 5: Calculate the loss function L for the discriminator and generator. d and L g Then, the loss function L of the entire model is calculated, and finally the model parameters are updated using stochastic gradient descent.

[0086] The specific implementation process includes the following sub-steps:

[0087] Step 5.1: Based on the output D(X) of the discriminator p ) and D(X r Calculate the loss function L of the discriminator. d The loss function L of the generator g It can be expressed by the following formula:

[0088]

[0089] in, and X represents r and X p The expected value of the distribution.

[0090] Step 5.2: Combine L1 loss and generator loss L g The sums are the global loss L, L = L1 + L g L1 can be represented as:

[0091]

[0092] Step 5.3: Update the network parameters θ of the generator and discriminator using stochastic gradient descent. g and θ d The specific steps are as follows:

[0093]

[0094] Where, η g and η d These represent the learning rates of the generator and discriminator, respectively.

[0095] The training epochs are set, and in this embodiment, the epochs are set to 100, meaning that when the training epochs reach 100, the model parameters are stopped from being updated. In addition, an early stopping epoch is also set, and in this embodiment, the early stopping epoch is set to 15, meaning that if the model's performance has not improved after 15 training epochs, training is stopped.

[0096] This invention provides an adversarial neural network based on the Transformer architecture. The model structure is redesigned to address the characteristics of traffic flow data, and adversarial training is introduced, resulting in better predictive performance than traditional models. Four modules are designed on top of the original model: degree embedding, distance embedding, time-dependent network, and spatial dynamic graph convolution network. Based on this design, the accuracy of model predictions and the global consistency of predicted data are improved, and the model convergence speed has a certain advantage over traditional models.

[0097] Experiments were conducted on four publicly available traffic flow datasets: PEMS03, PEMS04, PEMS07, and PEMS08. Mean Absolute Error (MAE), Root Mean Squared Error (RMSE), and Mean Absolute Percentage Error (MAPE) were used as evaluation metrics. Traffic flow data from the past hour was used to predict traffic flow data for the next hour, and the aforementioned evaluation metrics were calculated. The experimental results are shown in the attached figure. Figure 2 As shown in the figure. The results indicate that the proposed model outperforms the traditional model in traffic flow prediction.

[0098] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0099] It should be understood that those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims.

Claims

1. A traffic flow prediction method based on Transformer adaptive adversarial graph neural network, characterized in that, The method includes the following steps: Step 1: Obtain raw data from the road network, and perform degree embedding and distance embedding operations on the raw data to obtain preprocessed traffic flow data; Step 2: Construct a generator. Input traffic flow data into the generator's TDN module and perform attention mechanism operations to obtain aggregated time information. ; Step 3: Construct the SDGCN graph neural network, and... Aggregated spatial information is obtained by inputting it into the SDGCN graph neural network. Then Inputting data into an MLP with two hidden layers yields the predicted data. ; Step 4: Obtain real data Build a discriminator to analyze the predicted data. Traffic flow data The splicing operation is denoted as and will and real data They are input together into the discriminator; Step 5: Calculate the loss functions for the discriminator and generator. and Then calculate the loss function of the entire model. Finally, the model parameters are updated using stochastic gradient descent to obtain a trained Transformer-based adaptive adversarial graph neural network model; the trained model is then used to predict traffic flow.

2. The traffic flow prediction method based on Transformer adaptive adversarial graph neural network according to claim 1, characterized in that, Step 1 includes the following sub-steps: Step 1.1: For each node in the road network According to the adjacency matrix of the road network Calculate the degree of each node. Based on the degree of the node Generate a learnable parameter Compare it with the original data Add them together to achieve Degree Embedding, i.e., degree embedding algorithm; Step 1.2: Calculate the nodes and The distance is denoted as ,according to Generate learnable parameters In calculation and Attention score At that time, It is added as an offset to the calculation of the attention score, i.e. This is how Distance Embedding, or distance embedding algorithm, is implemented. This represents the vector dimension of the model.

3. The traffic flow prediction method based on Transformer adaptive adversarial graph neural network according to claim 2, characterized in that, The specific implementation process of step 2 includes: Traffic flow data Input two different one-dimensional convolutional modules respectively, and then use the results to... and The function processes the data and concatenates the results as input to the multi-head self-attention mechanism. The formula for this operation is expressed as: in, This represents a splicing operation. and For activation function, and For two distinct one-dimensional convolution operations, the attention mechanism operation can be described as follows: in, This indicates the operation of the attention mechanism. , , They represent The corresponding learnable weight matrix.

4. The traffic flow prediction method based on Transformer adaptive adversarial graph neural network according to claim 3, characterized in that, Step 3 includes the following sub-steps: Step 3.1: Aggregation time information obtained from TDN Construct a mask matrix by multiplying it by its own transpose. This represents the dynamic nature of the transportation network, specifically as follows: in, Represented as a normalization function, R represents the vector dimension of the model, N is the number of nodes in the road network, and R represents a real matrix. Step 3.2: Calculate the obtained mask matrix Integrating into graph convolution operations, and adjacency matrices with self-connections. The formula for performing the Hadamard product is as follows: in, For activation function, This represents the inverse of the degree matrix with self-connections. This represents an adjacency matrix with self-connections. For Hadama accumulation, and These are learnable parameters; Step 3.3: Repeat the number of layers set in Steps 2 and 3. Next, the final aggregated spatial information of SDGCN will be obtained. The prediction results are obtained through an MLP with two hidden layers. , is represented as: in, , , These are learnable parameters.

5. The traffic flow prediction method based on Transformer adaptive adversarial graph neural network according to claim 4, characterized in that, The specific implementation process of step 4 is as follows: Predicted data Traffic flow data Perform the splicing operation and record the result as... and will and real data They are input together into the discriminator, that is: ， in, This indicates the time step of traffic flow data. Indicates the time step of the prediction. This indicates a splicing operation.

6. The traffic flow prediction method based on Transformer adaptive adversarial graph neural network according to claim 5, characterized in that, Step 5 includes the following sub-steps: Step 5.1: Based on the output of the discriminator and Calculate the loss function of the discriminator Loss function of generator The formula is expressed as: in, and express and The expected value of the distribution; Step 5.2: [The sentence is incomplete and requires more context to be translated accurately.] Loss and generator loss The sum gives the global loss. , ,in Represented as: ； Step 5.3: Update the network parameters of the generator and discriminator using stochastic gradient descent. and The specific operation is as follows: in, and These represent the learning rates of the generator and the discriminator, respectively. The training epochs are set, meaning that when the training reaches the specified number of epochs, the model parameters are stopped from being updated. In addition, early stopping epochs are set, meaning that if the model's performance has not improved after 15 early stopping epochs, training is stopped.

7. A traffic flow prediction system based on Transformer adaptive adversarial graph neural network, characterized in that, include: Memory, used to store executable computer programs; A processor, when executing an executable computer program stored in memory, implements the traffic flow prediction method based on a Transformer adaptive adversarial graph neural network as described in any one of claims 1 to 6.

8. A computer-readable storage medium, characterized in that, The device contains a computer program that, when executed by a processor, implements the traffic flow prediction method based on the Transformer adaptive adversarial graph neural network as described in any one of claims 1 to 6.

Images