Transform and graph neural network-based traffic flow prediction method

Abstract

The invention relates to the technical field of intelligent traffic, in particular to a traffic flow prediction method based on Transform and a graph neural network, and the method comprises the steps: obtaining flow time sequence data from different traffic sensor nodes; extracting a time sequence characteristic of each node by using a Transform encoder, and generating an embedded vector of each traffic node; based on the embedded vector between the nodes, calculating the similarity between the nodes by using cosine similarity, and constructing a dynamic graph structure to reflect the spatial relationship between the nodes; neighbor information of the nodes is aggregated by using a graph attention mechanism in the graph neural network, and learning of spatial features is enhanced; according to the method, the time sequence features and the space features are fused, the future traffic flow is predicted through the full connection layer, the complex space-time relation in the traffic data can be effectively captured, and the method adapts to the change of the real road structure through the dynamic graph structure, so that the accuracy and robustness of traffic flow prediction are improved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent transportation technology, and in particular to a traffic flow prediction method based on Transformer and graph neural network. Background Art

[0002] In today's society, the rapid advancement of urbanization has made urban traffic flow increasingly complex and dynamically changing. The accurate management and prediction of traffic flow has become a key core issue in the construction and development of intelligent transportation systems. Efficient and accurate traffic flow prediction can not only provide a scientific basis for the intelligent control of traffic signals and achieve the rational optimization of traffic resources, but also significantly alleviate traffic congestion, greatly improve the actual use efficiency of roads, and promote the orderly operation of urban traffic.

[0003] In recent years, deep learning technology has been increasingly used in the field of traffic flow prediction, and models that combine spatiotemporal data have attracted much attention because they are expected to capture the complex and changing trends of traffic flow. Among many technologies, graph neural networks, with their unique advantages, can effectively mine the inherent spatial dependencies of traffic networks and extract valuable information from the topological structure of traffic networks; while sequence models such as Transformer focus on capturing the evolution of traffic flow over time and perform well in processing time series data.

[0004] However, in-depth research has found that most existing traffic flow prediction methods have an obvious flaw, that is, they often process spatial and temporal features separately, and fail to fully consider the close internal connection between the two. This separate processing method makes the model unable to cope with real-time traffic changes and emergencies, and it is difficult to respond accurately and promptly.

[0005] Further analysis of existing technologies shows that they have significant limitations in integrating spatiotemporal features and responding to real-time dynamic traffic changes, which directly leads to the difficulty of prediction accuracy in meeting actual needs. Specifically, when dealing with the complex spatial dependencies and time series evolution of traffic networks, the current model cannot achieve efficient coordinated processing of the two, and it is difficult to dynamically adjust feature weights according to different spatiotemporal conditions, thus failing to accurately reflect the real changing patterns of traffic flow in different scenarios.

[0006] This limitation is even more prominent in actual application scenarios. Since the existing models lack sufficient adaptability to the dynamic changes of traffic flow, when encountering emergencies (such as traffic accidents, bad weather, etc.) or common dynamic change scenarios such as road congestion, they cannot flexibly adapt to the changes in topological structures and adjust the prediction strategy, resulting in a large deviation between the prediction results and the actual traffic conditions.

[0007] In summary, there is an urgent need to develop an innovative traffic flow prediction method that can organically integrate temporal and spatial characteristics and has the ability to dynamically adjust the graph structure to effectively cope with the complex changes in traffic flow, thereby providing strong technical support for solving many difficult problems in traffic management. This has become a key task and necessary means for research and practice in the current transportation field. Summary of the invention

[0008] The purpose of the present invention is to provide a traffic flow prediction method based on Transformer and graph neural network, which can effectively capture the complex spatiotemporal relationships in traffic data and adapt to changes in the actual road structure through a dynamic graph structure, thereby improving the accuracy and robustness of traffic flow prediction to solve the problems raised in the above background technology.

[0009] In order to solve the above technical problems, the present invention provides the following technical solutions: A traffic flow prediction method based on Transformer and graph neural network, the method comprising: S100, obtaining time series data from different traffic sensor devices in the city and an initial road topology map, and normalizing the input of the model by preprocessing the data and using a maximum-minimum normalization method; The S200 and Transformer encoder modules capture long-term dependencies through the self-attention mechanism and output the embedding vector of each node for the fusion calculation of the temporal features of the downstream modules and the dynamic update of the graph structure; S300, based on the embedding vectors between the nodes, using cosine similarity to calculate the similarity between the nodes, and determining whether to update the dynamic graph according to the degree of change of the topological graph; S400, use the graph attention mechanism in the graph neural network to aggregate the neighbor information of the node and strengthen the learning of spatial features; S500, integrates the node-embedded temporal features and spatial features through the graph neural network, and performs traffic prediction through the fully connected layer.

[0010] Preferably, S100 includes: S101, deploying sensor clusters at traffic nodes to collect traffic flow data, including speed, flow, time occupancy, etc. The sensor clusters collect and upload traffic flow data to the data center at preset time intervals; S102, pre-processing the collected traffic flow data, including removing abnormal data and performing standard processing to ensure the integrity and consistency of the data; S103. Based on the physical connection relationship between traffic nodes (sensors) and roads, an initial traffic network topology structure of traffic nodes and edges is established. The topology structure uses traffic nodes (sensors) as graph vertices and roads as graph edges to reflect the flow path of traffic flow between different nodes.

[0011] Preferably, S200 includes: S201, the shape of the preprocessed multivariate time series dataset is ,in is the number of features, Represents the length of the time series; S202, define the size of the embedding time window as W, divide the entire data set X into multiple such time slices, and store the data in each slice Enter Transformer Encoder in turn; S203, Transformer Encoder slices each time Mapped to an embedding vector , where H is the dimension of the embedding vector; take the average of T / W embedding vectors to get the embedding vector of each node .

[0012] Preferably, S300 includes: S301. During neural network training, after each batch is embedded in a node, the cosine similarity between two node vectors needs to be recalculated. The formula is: ; S302, each node selects a maximum of K nodes with a cosine similarity greater than 0 as its neighbor node j, and then connects i and j to obtain the adjacency matrix A' of the graph; S303, subtract A' from the graph adjacency matrix that the model is using, and use the function to obtain the number q of non-zero elements in the subtraction matrix, where q / 2 is the number of edges that have changed; When q / 2>Q, it is set to be updated to a new graph adjacency matrix A; wherein Q can be flexibly set according to the number of nodes in the distributed network.

[0013] Preferably, S400 includes: constructing a graph neural network based on the adjacency matrix A, and using a graph attention-based feature extractor to fuse the information of the node with its neighbors: Aggregate features for node i The calculation method is: ; in, is the output feature of the graph attention layer, which is consistent with the input have the same shape, is the input time window of node i, are the neighbor nodes of node i obtained from the graph adjacency matrix A, is a trainable weight matrix, Indicates that the number of points marked as 1 in the adjacency matrix is ​​the number of all neighbor nodes of node i; The attention coefficient The calculation method is: ; ; Where ⊕ represents a link operation, is the coefficient vector learned by the attention mechanism, LeakyReLU is used as the nonlinear activation to calculate the attention coefficient, and the softmax function is used to normalize the attention coefficient.

[0014] Preferably, S500 includes: S501, after the neighborhood features are extracted by the graph neural network, we get The new time series representation of nodes ; In order to integrate the rich feature information in time and space, we embed the node vector obtained by previous training Through a linear layerLinear, and Perform residual connection to obtain , the formula is as follows: ; Where W and b are the weight matrix and bias vector in the Linear layer respectively; S502, through a fully connected layer, the traffic flow time window The next time point is predicted, and the mean absolute error MAE and the root mean square error RMSE are used as the loss function of the present invention.

[0015] A system using a traffic flow prediction method based on Transformer and graph neural network, the system comprising: a traffic flow collection module and a central server; The central server includes a node embedding module, a graph construction module and a traffic prediction module; The traffic flow collection module is placed on each device that needs to be detected, and is used to collect traffic in a distributed manner; The node embedding module is used to preprocess the traffic time series of each node and form respective node embedding vectors through Transformer Encoder; The graph construction module is used to implement similarity calculation of node embedding vectors and update the topological graph structure in real time; The traffic prediction module is used to fuse node embedding vector features and spatial features passed through the graph neural network, and perform prediction through a fully connected layer.

[0016] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps in the above-mentioned traffic flow prediction method based on Transformer and graph neural network.

[0017] A computer device includes a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the program, the steps in the above-mentioned traffic flow prediction method based on Transformer and graph neural network are implemented.

[0018] Compared with the prior art, the beneficial effects achieved by the present invention are: By combining Transformer and graph neural network, the spatiotemporal relationship in traffic flow can be effectively captured, and the accuracy and robustness of prediction can be improved. The Transformer module uses the self-attention mechanism to extract long-term time dependencies, while the graph neural network strengthens spatial feature learning through the graph attention mechanism, ensuring a deep understanding of the complex spatial dependencies between nodes in the traffic network. Combined with the fusion strategy of spatiotemporal features, the model can adaptively respond to dynamic changes in traffic flow, especially in emergencies or complex scenarios, and can flexibly adjust the prediction strategy. In addition, the design of dynamically updating the graph structure effectively responds to changes in road topology, further improving the adaptability and accuracy of the model in practical applications. On the whole, the present invention provides a more accurate and adaptive prediction tool for urban intelligent traffic management. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings: Figure 1 is a schematic diagram of a technical process of traffic flow prediction in an embodiment of the present invention; Figure 2 It is a schematic diagram of the architecture of a Transformer and graph neural network model for traffic flow prediction in an embodiment of the present invention; Figure 3 Schematic diagram of a node embedding module in an embodiment of the present invention. DETAILED DESCRIPTION

[0020] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] See also Figure 1-Figure 3 , the present invention provides a technical solution: Example 1: Combination Figure 1 and Figure 2 , the present invention provides a technical solution: a traffic flow prediction method based on Transformer and graph neural network, firstly, the time series data of each sensor node is input into TransformerEncoder for node embedding, the time series characteristics of traffic flow are obtained, and the node embedding is used as the basis for updating the dynamic road topology map; then the graph attention network constructed by the road topology map is used to obtain the neighborhood spatial characteristics of each node, and the corresponding node embedding is linearly weighted fused to strengthen the time series and spatial characteristics; finally, the traffic flow prediction is made through the fully connected layer. The specific steps are as follows: Step 1: Obtain time series data from different traffic sensor devices in the city and the initial road topology map, and normalize the model input by preprocessing the data and using the maximum-minimum normalization method; Deploy sensor clusters at traffic nodes to collect traffic flow data, including speed, flow, time occupancy, etc. The sensor clusters collect and upload data to the data center at preset time intervals; Pre-process the collected traffic flow data, including removing abnormal data and performing standard processing to ensure the integrity and consistency of the data; Based on the physical connection relationship between traffic nodes (sensors) and roads, an initial traffic network topology structure of traffic nodes and edges is established. The topology structure uses traffic nodes (sensors) as graph vertices and roads as graph edges to reflect the flow path of traffic flow between different nodes.

[0022] Step 2: The Transformer encoder module captures long-term dependencies through the self-attention mechanism and outputs the embedding vector of each node for the fusion calculation of the temporal features of the downstream modules and the dynamic update of the graph structure; Graph representation learning is a specific application of representation learning on graphs. It aims to map each node representation into a low-dimensional and dense vector space by utilizing the connection relationship between nodes in the graph while effectively maintaining the graph structure. This study aims to provide a method that can dynamically construct the graph structure without providing the initial graph structure information. This requires generating a vector representation that can accurately describe the characteristics of each node involved in the graph construction.

[0023] In the traffic system, it is common for sensor devices at different intersections to have similar fluctuations within a time window due to the connectivity of roads. The purpose of node embedding is to convert the original high-dimensional time series node features into low-dimensional vectors. This has two purposes: one is to facilitate the calculation of the relationship between nodes, thereby building a graph structure; the other is to continuously learn during the training process and perform feature fusion on behalf of time series features and neighborhood features after the graph neural network layer, thereby making better use of feature information.

[0024] The shape of the preprocessed multivariate time series dataset is ,in is the number of features, Represents the length of the time series.

[0025] This paper defines a time window size of W, which is the size of the entire data set. Divide into multiple such time slices, and divide the data in each slice Enter Transformer Encoder one by one.

[0026] Transformer Encoder slices each time Mapped to an embedding vector ,in is the dimension of the embedding vector. The embedding vectors are averaged to get the embedding vector of each node .

[0027] The following is combined with Figure 3 A detailed explanation of the node embedding module is given.

[0028] (1) Transformer Encorder layer For the Transformer encoder, it will input time series data Convert to embedding vector . Each layer of the Transformer encoder can be expressed as: ; in, is the index of the layer, , and each Transformer encoder layer contains a self-attention mechanism and a feed-forward neural network.

[0029] (2) Positional encoding Position encoding PE is a time series data A matrix of the same shape is used to provide the position information of the time step. The position encoding can be expressed as: , ; in is the position of the time step, i is the index of the dimension, and d is the dimension of the feature vector. Here the sine and cosine functions are used to generate different frequencies in each dimension. We will Added to the positional encoding and input into the feedforward network.

[0030] (3) Transformer self-attention The purpose of using Transformer Encoder is to extract long-term features of each sensor node feature, which relies on the self-attention mechanism first proposed in 2017.

[0031] The computational steps of each Transformer encoder layer can be expressed as follows: ; in, , ; Where Q, K, and V are query, key, and value respectively. , , , is the weight matrix.

[0032] (4) Transformer Encoder Output Batch normalization normalizes the data for each feature dimension i, which can be expressed as: ; in, and are the mean and variance of feature i in the batch of data, and is a learnable parameter is a small constant to avoid division by zero.

[0033] The input is then transformed nonlinearly through the ReLU (Rectified Linear Unit) activation function, which can be expressed as ReLU(x)=max(0,x), and applied to the output of batch normalization.

[0034] To sum up, the output of Transformer encoder is The complete formula after batch normalization and ReLU activation function is expressed as: ; here It is the final output after batch normalization and ReLU activation function. It contains the characteristics and time-dependent information of time series data. For each node feature, there will be T / W embedding vectors, and the average of them is used to get the embedding vector of each node. : ; These embedding vectors will initialize the graph structure and will be fused with spatial features in the downstream attention module to jointly influence the prediction value.

[0035] Step 3: Based on the embedding vectors between nodes, the cosine similarity is used to calculate the similarity between nodes, and the dynamic graph is updated based on the degree of change in the topology graph.

[0036] When training a neural network, after each batch is embedded in the node, the cosine similarity between the node vectors needs to be recalculated. The formula is: ; Each node selects up to K nodes with cosine similarity greater than 0 as its neighbor node j, and then connects i and j to obtain the adjacency matrix A' of the graph.

[0037] Subtract A' from the graph adjacency matrix that the model is using, and use the function to get the number of non-zero elements q in the subtracted matrix, then q / 2 is the number of edges that have changed. Then set it to the new graph adjacency matrix A when q / 2>Q (Q can be flexibly set according to the number of nodes in the distributed network).

[0038] Step 4: Use the graph attention mechanism in the graph neural network to aggregate the node’s neighbor information and strengthen the learning of spatial features.

[0039] A graph neural network is constructed based on the adjacency matrix A, and a feature extractor based on graph attention is used to fuse the information of a node with its neighbors. For node i, the aggregated features The calculation method is as follows: ; in is the output feature of the graph attention layer, which is consistent with the input Have the same shape. is the input time window of node i, are the neighbor nodes of node i obtained from the graph adjacency matrix A. is a trainable weight matrix. Represents the points marked as 1 in the adjacency matrix, and the number is the number of all neighbor nodes of node i.

[0040] The attention coefficient is calculated as follows: ; ; Where ⊕ represents a link operation, is the coefficient vector learned by the attention mechanism. We use LeakyReLU as the nonlinear activation to calculate the attention coefficient and normalize it using the softmax function.

[0041] Step 5: Fusion node embedded temporal features and spatial features through graph neural network, and perform traffic prediction through the fully connected layer.

[0042] After neighborhood feature extraction through graph neural network, we can get The new time series representation of nodes In order to integrate the rich feature information in time and space, we embed the node vector obtained by previous training Through a linear layerLinear, and Perform residual connection to obtain , the formula is as follows: ; Where W and b are the weight matrix and bias vector in the Linear layer respectively.

[0043] Through a fully connected layer, the traffic flow time window The next time point is predicted, and the mean absolute error MAE and the root mean square error RMSE are used as the loss function of the present invention.

[0044] Embodiment 2: The computer-readable storage medium of this embodiment stores a computer program, which, when executed by a processor, implements the steps in a traffic flow prediction method based on Transformer and graph neural network in Embodiment 1.

[0045] The computer-readable storage medium of this embodiment may be an internal storage unit of the terminal, such as a hard disk or memory of the terminal; the computer-readable storage medium of this embodiment may also be an external storage device of the terminal, such as a plug-in hard disk, a smart memory card, a secure digital card, a flash memory card, etc. equipped on the terminal; further, the computer-readable storage medium may also include both an internal storage unit of the terminal and an external storage device.

[0046] The computer-readable storage medium of this embodiment is used to store computer programs and other programs and data required by the terminal. The computer-readable storage medium can also be used to temporarily store data that has been output or is to be output.

[0047] Embodiment 3: The computer device of this embodiment includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps in the traffic flow prediction method based on Transformer and graph neural network in Embodiment 1 are implemented.

[0048] In this embodiment, the processor may be a central processing unit, or other general-purpose processors, digital signal processors, application-specific integrated circuits, readily available programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The memory may include a read-only memory and a random access memory, and provide instructions and data to the processor. A portion of the memory may also include a non-volatile random access memory. For example, the memory may also store information about the device type.

[0049] Those skilled in the art will appreciate that the contents disclosed in the embodiments may be provided as methods, systems, or computer program products. Therefore, the present solution may take the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware. Moreover, the present solution may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) containing computer-usable program codes.

[0050] The present solution is described with reference to the method according to the embodiment of the present solution and the flowchart and / or block diagram of the computer program product. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions; these computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 one or more processes and / or methods Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0051] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 one or more processes and / or methods Figure 1 A function specified in one or more boxes.

[0052] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 one or more processes and / or methods Figure 1 The steps for the functions specified in one or more boxes.

[0053] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0054] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A traffic flow prediction method based on Transformer and graph neural network, characterized by: The method comprises: S100, obtaining time series data from different traffic sensor devices in the city and an initial road topology map, and normalizing the input of the model by preprocessing the data and using a maximum-minimum normalization method; The S200 and Transformer encoder modules capture long-term dependencies through the self-attention mechanism and output the embedding vector of each node for the fusion calculation of the temporal features of the downstream modules and the dynamic update of the graph structure; S300, based on the embedding vectors between the nodes, using cosine similarity to calculate the similarity between the nodes, and determining whether to update the dynamic graph according to the degree of change of the topological graph; S400, use the graph attention mechanism in the graph neural network to aggregate the neighbor information of the node and strengthen the learning of spatial features; S500, integrates the node-embedded temporal features and spatial features through the graph neural network, and performs traffic prediction through the fully connected layer.

2. A traffic flow prediction method based on Transformer and graph neural network as claimed in claim 1, characterized in that: The S100 includes: S101, deploying a sensor cluster at a traffic node to collect traffic flow data, including speed, flow and time occupancy, and the sensor cluster collects and uploads the data to a data center at a preset time interval; S102, pre-processing the collected traffic flow data, including removing abnormal data and performing standard processing to ensure the integrity and consistency of the data; S103. Based on the physical connection relationship between the traffic sensor nodes and the roads, an initial traffic network topology structure of traffic nodes and edges is established. The topology structure uses traffic nodes as graph vertices and roads as graph edges to reflect the flow path of traffic flow between different nodes.

3. The traffic flow prediction method based on Transformer and graph neural network as claimed in claim 1, characterized in that: The S200 includes: S201, the shape of the preprocessed multivariate time series dataset is ,in is the number of features, Represents the length of the time series; S202, define the size of the embedding time window as W, divide the entire data set X into multiple such time slices, and store the data in each slice Enter Transformer Encoder in turn; S203, Transformer Encoder slices each time Mapped to an embedding vector , where H is the dimension of the embedding vector; take the average of T / W embedding vectors to get the embedding vector of each node .

4. The traffic flow prediction method based on Transformer and graph neural network as claimed in claim 1, characterized in that: The S300 includes: S301. During neural network training, after each batch is embedded in a node, the cosine similarity between two node vectors needs to be recalculated. The formula is: ; S302, each node selects a maximum of K nodes with a cosine similarity greater than 0 as its neighbor node j, and then connects i and j to obtain the adjacency matrix A' of the graph; S303, subtract A' from the graph adjacency matrix that the model is using, and use the function to obtain the number q of non-zero elements in the subtraction matrix, where q / 2 is the number of edges that have changed; When q / 2>Q, it is set to be updated to a new graph adjacency matrix A; wherein Q can be flexibly set according to the number of nodes in the distributed network.

5. The traffic flow prediction method based on Transformer and graph neural network as claimed in claim 1, characterized in that: The S400 includes: constructing a graph neural network based on the adjacency matrix A, and using a graph attention-based feature extractor to fuse the information of the node with its neighbors: Aggregate features for node i The calculation method is: ; in, is the output feature of the graph attention layer, which is consistent with the input have the same shape, is the input time window of node i, are the neighbor nodes of node i obtained from the graph adjacency matrix A, is a trainable weight matrix, Indicates that the number of points marked as 1 in the adjacency matrix is ​​the number of all neighbor nodes of node i; The attention coefficient The calculation method is: ； ； Where ⊕ represents a link operation, is the coefficient vector learned by the attention mechanism, LeakyReLU is used as the nonlinear activation to calculate the attention coefficient, and the softmax function is used to normalize the attention coefficient.

6. The traffic flow prediction method based on Transformer and graph neural network as claimed in claim 1, characterized in that: The S500 comprises: After the graph neural network performs neighborhood feature extraction, we get The new time series representation of nodes ; In order to integrate the rich feature information in time and space, we embed the node vector obtained by previous training Through a linear layerLinear, and Perform residual connection to obtain , the formula is as follows: ； Where W and b are the weight matrix and bias vector in the Linear layer respectively.

7. A system using the traffic flow prediction method based on Transformer and graph neural network as described in any one of claims 1 to 6, characterized in that: The system includes: a traffic flow collection module and a central server; The central server includes a node embedding module, a graph construction module and a traffic prediction module; The traffic flow collection module is placed on each device that needs to be detected, and is used to collect traffic in a distributed manner; The node embedding module is used to preprocess the traffic time series of each node and form respective node embedding vectors through TransformerEncoder; The graph construction module is used to implement similarity calculation of node embedding vectors and update the topological graph structure in real time; The traffic prediction module is used to fuse node embedding vector features and spatial features passed through the graph neural network, and perform prediction through a fully connected layer.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps in a traffic flow prediction method based on Transformer and graph neural network as described in any one of claims 1-6 are implemented.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the program, the steps in the traffic flow prediction method based on Transformer and graph neural network as described in any one of claims 1-6 are implemented.

Images