A traffic prediction method based on deep learning

Abstract

The present invention provides a traffic prediction method based on deep learning, comprising: obtaining a historical representation representing traffic state information and spatiotemporal information for a first number of historical time steps, and a future representation representing spatiotemporal information for a second number of future time steps; processing the historical representation using a first BERT model to obtain a first state code; adding the first state code to the future representation to obtain a predicted representation; and processing the predicted representation using a second BERT model to obtain a predicted traffic state. Through the above approach, the present invention can effectively capture the spatiotemporal dependencies hidden in traffic data and improve the accuracy of long-term predictions.

Description

Technical Field

[0001] The present invention belongs to the field of intelligent transportation technology, and specifically relates to a traffic prediction method based on deep learning. Background Art

[0002] With the acceleration of urbanization and rapid economic development, urban populations and the number of motor vehicles are constantly growing. To maximize urban operational efficiency, cities are developing intelligent transportation systems, in which traffic forecasting plays a key role. Accurate forecasts can effectively alleviate urban traffic congestion and provide a more meaningful basis for traffic management decisions. Traffic forecasting faces two major challenges: temporal dependence and spatial dependence. Temporal dependence refers to the influence of current traffic conditions on previous traffic conditions. Temporal dependence exhibits characteristics such as proximity, periodicity, and trend. Spatial dependence refers to the influence of the surrounding environment on traffic conditions in a region. Different adjacent regions have different impacts. Generally speaking, the closer the distance, the greater the impact. Temporal and spatial dependencies are often intertwined, resulting in more complex relationships.

[0003] With the tremendous achievements of deep learning methods in fields such as computer vision and natural language processing, many researchers have attempted to introduce deep learning methods into traffic forecasting. Convolutional Neural Networks (CNNs) and Graph Neural Networks (GNNs) have been used to learn the spatial correlations hidden in grid-structured and graph-structured traffic data, respectively. Recursive Neural Networks (RNNs) have been instructive for modeling temporal correlations. RNN variants such as the long short-term memory model and the gated recurrent unit can be applied to predict short-term traffic flow because they address the gradient explosion and gradient vanishing problems of traditional RNN models.

[0004] Traditional RNN models still have shortcomings in capturing temporal dependencies. In traffic forecasting, traffic conditions in the current time period may be affected by traffic conditions long before. However, traditional RNN models have difficulty remembering traffic conditions from long before, which means they suffer from long-term dependencies. Furthermore, existing machine learning methods can only model temporal dependencies and cannot capture spatial dependencies. Summary of the Invention

[0005] The present invention provides a traffic prediction method based on deep learning to solve the problem of low accuracy in existing long-term traffic state prediction.

[0006] To solve the above technical problems, the present invention proposes a traffic prediction method based on deep learning, including: obtaining a historical representation representing traffic state information and spatiotemporal information of a first number of historical time steps, and a future representation representing spatiotemporal information of a second number of future time steps; applying a first BERT model to process the historical representation to obtain a first state code; adding the first state code to the future representation to obtain a predicted representation; and applying a second BERT model to process the predicted representation to obtain a predicted traffic state.

[0007] Optionally, applying the first BERT model to process the historical representation to obtain a first state code includes: performing temporal attention calculation on the historical representation to obtain a first temporal attention code; performing spatial attention calculation on the first temporal attention code to obtain a first spatial attention code; and performing layer normalization on the first spatial attention code to obtain a first state code.

[0008] Optionally, the temporal attention calculation is performed on the historical representation to obtain a first temporal attention code, including: decomposing the historical representation into time steps and node granularity, calculating the time input vector of the temporal attention of the current layer at any time step of any node in the first BERT model according to the historical representation, the time input vector including a time query vector, a time key vector and a time value vector; applying an activation function to calculate the first temporal attention weight of the current layer according to the time query vector and the time key vector; performing a weighted summation of the first temporal attention weight of the current layer and the time value vector, and performing a residual connection with the first temporal attention code of the previous layer to obtain the first temporal attention code of the current layer.

[0009] Optionally, the spatial attention calculation is performed on the first temporal attention code to obtain the first spatial attention code, including: calculating the spatial input vector of the spatial attention of the current layer at any time step of any node in the first BERT model based on the first spatial attention code, the spatial input vector including a spatial query vector, a spatial key vector and a spatial value vector; calculating the first spatial attention weight of the current layer by applying an activation function according to the spatial query vector and the spatial key vector; performing a weighted summation of the first spatial attention weight of the current layer and the spatial value vector, and performing a residual connection with the first temporal attention code of the current layer to obtain the first spatial attention code of the current layer.

[0010] Optionally, the layer normalization processing is performed on the first spatial attention code to obtain a first state code, including: applying a feedforward network to process the first spatial attention code; and superimposing the first spatial attention code with the output of the feedforward network to obtain a first state code.

[0011] Optionally, adding the first state code to the future representation to obtain a predicted representation includes: if the first number is less than the second number, padding the first state code of the first number of time steps with random numbers, extending it to the second number of time steps; if the first number is greater than the second number, padding the future representation of the second number of time steps with zeros, extending it to the first number of time steps.

[0012] Optionally, the application of the second BERT model to process the predicted representation to obtain the predicted traffic status includes: performing temporal attention calculation on the predicted representation to obtain a second temporal attention code; performing spatial attention calculation on the second temporal attention code to obtain a second spatial attention code, and performing layer normalization on the second spatial attention code to obtain a second state code, where the second state code is the predicted traffic status.

[0013] Optionally, the temporal attention calculation of the predicted representation to obtain the second temporal attention code includes: decomposing the predicted representation into time steps and node granularity, calculating the time input vector of the temporal attention of the current layer at any time step of any node in the second BERT model according to the predicted representation, the time input vector including a time query vector, a time key vector and a time value vector; applying an activation function to calculate the second temporal attention weight of the current layer according to the time query vector and the time key vector; performing weighted summation of the second temporal attention weight of the current layer and the time value vector, and performing a residual connection with the second temporal attention code of the previous layer to obtain the second temporal attention code of the current layer.

[0014] Based on the same inventive concept, an embodiment of the present invention also proposes an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method described in any one of the above is implemented.

[0015] Based on the same inventive concept, an embodiment of the present invention further proposes a computer storage medium, wherein the storage medium stores at least one executable instruction, and the executable instruction enables a processor to execute any one of the aforementioned methods.

[0016] From the above, it can be seen that the present invention proposes a traffic prediction method based on deep learning, including: obtaining a historical representation of traffic state information and spatiotemporal information representing a first number of historical time steps and a future representation of spatiotemporal information representing a second number of future time steps; applying a first BERT model to process the historical representation to obtain a first state code; adding the first state code to the future representation to obtain a predicted representation; applying a second BERT model to process the predicted representation to obtain a predicted traffic state, which can effectively capture the hidden spatiotemporal dependencies in traffic data and improve the accuracy of long-term predictions. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0018] Figure 1 Schematic diagram of a flow chart of a traffic prediction method based on deep learning in an embodiment of the present invention;

[0019] Figure 2 Schematic diagram of input representation in a traffic prediction method based on deep learning in an embodiment of the present invention;

[0020] Figure 3 Schematic diagram of separating spatiotemporal attention in an embodiment of the present invention;

[0021] Figure 4 This is a schematic diagram of a process for obtaining a first state code in an embodiment of the present invention;

[0022] Figure 5 Schematic diagram of traffic prediction based on deep learning in an embodiment of the present invention;

[0023] Figure 6 The figure is a schematic structural diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0024] In order to make the objectives, technical solutions and advantages of the present disclosure more clearly understood, the present disclosure is further described in detail below in conjunction with specific embodiments and with reference to the accompanying drawings.

[0025] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present invention should have the usual meanings understood by people with ordinary skills in the field to which the present disclosure belongs. The "first", "second" and similar words used in the embodiments of the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0026] The embodiment of the present invention proposes a traffic prediction method based on deep learning, such as Figure 1 As shown, a traffic prediction method based on deep learning includes:

[0027] Step S1: Acquire a historical representation of traffic state information and spatiotemporal information representing a first number P of historical time steps and a future representation of spatiotemporal information representing a second number Q of future time steps.

[0028] Bidirectional Encoder Representation from Transformer (BERT) is a milestone model in natural language processing. The attention mechanism it contains reduces the distance between any time steps in the time window to 1, thereby effectively solving the problem of long-term dependencies. In addition, BERT is a pre-trained model that can be equipped with different lightweight outputs according to different tasks, without the need to design a custom model for a specific task. Therefore, after modification, BERT is expected to become a general model for traffic status, which can be used for multiple downstream tasks such as traffic status classification and traffic status clustering. In view of the shortcomings of traditional methods in capturing spatial dependencies and temporal dependencies and the advantages of the BERT model, the embodiment of the present invention modifies the BERT model to make it a model suitable for traffic prediction scenarios, referred to as TPBERT.

[0029] Before step S1, the road network is represented as a directed graph G = (V, E, A), where V is the set of nodes, E is the set of edges between nodes, and A is the adjacency matrix. In particular, N = |V| is the number of nodes, ij ∈A represents v i and v jThe physical distance between them. The flow state of all vertices on the directed graph G at time step t is represented by the vector Denote, where C is the number of traffic state observations. Based on the directed graph G and the traffic state data observed in P historical time steps, the traffic prediction task can be formulated as learning a function f to predict the traffic state in the next Q time steps: f(G, X = Y;

[0030] in,

[0031] Because traffic prediction is affected by temporal and spatial dependencies, encoding temporal and spatial information and incorporating them into the model is crucial. Furthermore, considering the positional relationship between historical and future time steps, it is necessary to encode traffic data and its temporal, spatial, and location information to obtain traffic state embedding information, temporal embedding information, spatial embedding information, and location embedding information. The size of all embeddings is set to D. Each type of information is described below:

[0032] Traffic state embedding information: The original traffic state observation at time step t is represented as To keep the same size as other embeddings, t Through a fully connected network, the final representation is obtained

[0033] Time embedding information: Periodicity is an important feature of time dependency in traffic prediction. Time embedding mainly includes daily periodicity and weekly periodicity. Daily periodicity means that traffic conditions are more similar at the same time of the day, and weekly periodicity means that traffic conditions on the same day of the week have the same pattern. For example, there are seven days in a week, and the weekly periodicity requires 7 different embedding vectors to represent it. The representation of daily periodicity is related to the time interval of data collection. Assuming the time interval is 5 minutes, there are 24×60÷5=288 time steps in a day. Therefore, 288 different embedding vectors will be used to represent daily periodicity. The daily periodicity and weekly periodicity embeddings of the present invention are randomly initialized. Time embedding information It is obtained by adding the daily and weekly periodic embeddings, which can be continuously updated during training.

[0034] Spatial embedding information: BERT can encode the elements and relationships between elements in a sequence, but it cannot model spatial dependencies. To solve this problem, spatial embedding information is proposed based on graph embedding. It retains the key information of the node in a vector and uses the node embedding algorithm to learn node representations. This is a biased random walk, where the hyperparameters p and q control the walking strategy. All node representation vectors are pre-trained to facilitate spatial embedding, represented as

[0035] Position embedding information: There are two options for position encoding, absolute position encoding and relative position encoding. In the embodiment of the present invention, relative position encoding is selected because absolute position encoding requires a known position in the entire time series, while relative position encoding does not. For P consecutive historical time steps and Q future time steps, their relative positions can be encoded by different embeddings of P+Q. Like time embedding information, position embedding information It is also randomly initialized and can be updated during training.

[0036] like Figure 2 As shown in , the historical representation includes historical traffic state embedding information, historical time embedding information, historical space embedding information and historical location embedding information, and the future representation includes future time embedding information, future space embedding information and future location embedding information. That is, the historical representation of the historical time step t is The future representation of the future time step t is

[0037] Step S2: Apply the first BERT model to process the historical representation to obtain a first state code.

[0038] In order to capture the hidden temporal and spatial dependencies in traffic state data, a separate spatiotemporal attention mechanism is used to calculate temporal attention and spatial attention one by one. Figure 3 As shown in , the input is first passed to the temporal attention to capture temporal dependencies, and then passed to the spatial attention to capture spatial dependencies, obtaining the final output. It is worth noting that attention is calculated for each node in the road network, that is, each node can be used as a query vector. Figure 4 As shown, step S2 includes:

[0039] Step S21: Perform temporal attention calculation on the historical representation to obtain a first temporal attention code.

[0040] Decompose the historical representation into time steps t and node granularity v, and calculate the time input vector of the time attention of the current layer l at any time step t of any node v in the first BERT model based on the historical representation, wherein the time input vector includes the time query vector Time key vector and the time value vector The calculation formula is as follows:

[0041]

[0042]

[0043]

[0044] Where LN represents the layer normalization operation and a represents the ath attention head. Assume that the total number of attention heads is A and the dimension of attention heads is D h =÷A.

[0045] Apply the activation function to calculate the first time attention weight of the current layer l according to the time query q vector and the time key k vector

[0046]

[0047] Wherein, SM is an activation function, and the embodiment of the present invention preferably uses a softmax activation function.

[0048] Perform weighted summation on the first time attention weight of the current layer l and the time value vector And make a residual connection with the first time attention code of the previous layer to obtain the first time attention code of the current layer l The calculation formula is as follows:

[0049]

[0050]

[0051] Step S22: Encode the first time attention Perform spatial attention calculation and obtain the first spatial attention code

[0052] In this embodiment of the present invention, temporal attention encoding is the input for computing spatial attention. That is, the new spatial query vector, spatial key vector, and spatial value vector are composed of Obtained, still used here Indicates that no further details will be given.

[0053] In step S22, according to the first time attention code Calculate the spatial input vector of the spatial attention of the current layer l at any time step t of any node v in the first BERT model, where the spatial input vector includes the spatial query vector Spatial key vector and space-valued vectors The calculation formula is the same as that in step S21 and will not be repeated here.

[0054] According to the spatial query vector and the spatial key vector Apply the activation function to calculate the first spatial attention weight of the current layer l

[0055]

[0056] The first spatial attention weight of the current layer l With the space value vector Perform weighted summation And perform residual connection with the first time attention code of the current layer l to obtain the first spatial attention code of the current layer l The calculation formula is as follows:

[0057]

[0058]

[0059] Step S23: Encode the first spatial attention Perform layer normalization to obtain the first state encoding

[0060] Optionally, a feedforward network is first applied to encode the first spatial attention Then the first spatial attention code is superimposed on the output of the feedforward network to obtain the first state code The calculation formula is as follows:

[0061]

[0062] Step S3: Add the first state code to the future representation to obtain a predicted representation.

[0063] Of the two inputs, historical representation and future representation, the time, space, and location information of the future representation are known. Therefore, a secondary input is required to incorporate future information into the model and complete the transition from history to the future. The first state is encoded as a first number P of time step dimensions, and the future representation is encoded as a second number Q of time steps. P≠Q, that is, when the future representation is inconsistent with the first state encoding dimension, if the first number is less than the second number, the first state encoding of the first number of time steps is padded with random numbers and extended to the second number of time steps. If the first number is greater than the second number, the future representation of the second number of time steps is padded with zeros and extended to the first number of time steps. In this way, the dimensions of the historical representation and the future representation can be guaranteed to be consistent.

[0064] Step S4: Apply the second BERT model to process the predicted representation to obtain the predicted traffic status.

[0065] The structure of the TPBERT model is as follows Figure 5As shown in Figure 1, the entire TPBERT model is composed of two l layers stacked together. The first l layers are the number of operation layers of the first BERT model, and the last l layers are the number of operation layers of the second BERT model. The structure of the second BERT model is exactly the same as the first BERT model. The first l layers have extracted the abstract information of the historical representation, and the last l layers can be combined with the future representation to make corresponding predictions. The historical representation method is expressed as The future representation is expressed as E h is fed into the first L layers, producing an output When P = Q, or by expanding the dimension output H L and future representation E f When the dimensions of H are the same, L and future representations f Add together and get E f ' is fed into the second L layer, producing an output That is, the predicted representation E p To get the final prediction E p Will enter a fully connected neural network.

[0066] In step S4, a temporal attention calculation is performed on the predicted representation to obtain a second temporal attention code. A spatial attention calculation is performed on the second temporal attention code to obtain a second spatial attention code. Layer normalization is then performed on the second spatial attention code to obtain a second state code, which represents the predicted traffic state. The above analysis and calculation process is the same as in step 2, except that the historical representation data in the original formula is replaced with the predicted representation data.

[0067] The following experiments tested the deep learning-based traffic prediction method of the present invention. As shown in Table 1, two real-world public datasets, METR-LA and PeMS-BAY, were used to evaluate the TPBERT model of the present invention. Both datasets have a 5-minute time step, with 3, 6, and 12 time steps representing short-term, medium-term, and long-term predictions, respectively. METR-LA and PeMS-BAY are traffic datasets of different sizes, and the mean absolute error (MAE), root mean square error (RMSE), and mean absolute percentage error (MAPE) are used to measure model performance.

[0068] Table 1 Test comparison results

[0069]

[0070]

[0071] In Table 1, HA, ARIMA, SVR, FNN, FC-LSTM, DCRNN, STGCN, MRA-BGCN, Graph WaveNet, STAWnet, MTGNN, and GMAN are other different types of prediction models. HA represents a prediction model that uses the weighted average of historical time series as the prediction result; ARIMA and Kalman filter are statistical prediction models for predicting and analyzing time series; SVR is a model that regards traffic prediction as a regression task and makes predictions with the help of support vector machines; FNN is a prediction model composed of two dense layers and L2 regularization; FC-LSTM is an encoder-decoder prediction model; DCRNN is a prediction model that uses bidirectional graph random walks and RNN to capture spatial and temporal correlations; STGCN is a prediction model based on spatial-temporal convolution blocks that integrates graph convolution and gated temporal convolution; MRA-BGCN is a prediction model that introduces two-component graph convolution and multi-range attention mechanism to integrate traffic information from different neighbors; Graph WaveNet is a prediction model that uses adaptive dependency matrix and one-dimensional convolution to learn long sequence information; Wnet is a prediction model that uses self-learned node embeddings to represent latent spatial relationships; MTGNN is a multivariate time series prediction model composed of graph structure learning, graph convolution, and temporal convolution; GMAN is an encoder-decoder architecture prediction model equipped with multiple attention mechanisms, such as spatial attention, temporal attention, and transformation attention. Experimental results show that the deep learning-based traffic prediction method of the embodiment of the present invention has improved accuracy in traffic prediction. In short-term prediction, MRA-BGCN performed best on two datasets. In medium-term and long-term prediction, TPBERT outperformed other models on both datasets. From the perspective of different datasets, the prediction error of METR-LA is larger than that of PeMS-BAY, indicating that the traffic conditions in METR-LA are more complex than those in the BAY area. TPBERT performs well on the more challenging METR-LA, indicating that TPBERT has important modeling capabilities for complex traffic data.

[0072] The embodiment of the present invention obtains a historical representation of traffic status information and spatiotemporal information representing a first number of historical time steps and a future representation of spatiotemporal information representing a second number of future time steps; applies a first BERT model to process the historical representation to obtain a first state code; adds the first state code to the future representation to obtain a predicted representation; applies a second BERT model to process the predicted representation to obtain a predicted traffic status, thereby improving the accuracy of long-term predictions and facilitating the capture of hidden spatiotemporal dependencies in traffic data.

[0073] Based on the same inventive concept, an embodiment of the present invention also provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and runnable on the processor, and is characterized in that when the processor executes the program, it implements the method described in any one of the above.

[0074] Figure 6 6 shows a more specific hardware structure diagram of an electronic device provided in this embodiment. The device may include: a processor 601, a memory 602, an input / output interface 603, a communication interface 604, and a bus 605. The processor 601, the memory 602, the input / output interface 603, and the communication interface 604 are connected to each other within the device via the bus 605.

[0075] The processor 601 can be implemented using a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided by the embodiments of the present invention.

[0076] The memory 602 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage devices, dynamic storage devices, etc. The memory 602 can store an operating system and other application programs. When the technical solutions provided by the embodiments of the present invention are implemented through software or firmware, the relevant program codes are stored in the memory 602 and called and executed by the processor 601.

[0077] The input / output interface 603 is used to connect to input / output modules to implement information input and output. The input / output modules can be configured as components in the device (not shown in the figure) or can be externally connected to the device to provide corresponding functions. Input devices may include a keyboard, mouse, touch screen, microphone, various sensors, etc., and output devices may include a display, speaker, vibrator, indicator light, etc.

[0078] The communication interface 604 is used to connect to a communication module (not shown) to enable communication between the device and other devices. The communication module can communicate via a wired method (such as USB, network cable, etc.) or a wireless method (such as mobile network, WIFI, Bluetooth, etc.).

[0079] The bus 605 comprises a pathway for transmitting information between the various components of the device (eg, the processor 601 , the memory 602 , the input / output interface 603 , and the communication interface 604 ).

[0080] It should be noted that although the above device only shows the processor 601, the memory 602, the input / output interface 603, the communication interface 604, and the bus 605, in a specific implementation, the device may also include other components necessary for normal operation. In addition, those skilled in the art will understand that the above device may only include the components necessary to implement the embodiments of the present invention, and does not necessarily include all the components shown in the figure.

[0081] Based on the same inventive concept, an embodiment of the present invention further provides a computer storage medium, which stores at least one executable instruction, and the executable instruction enables a processor to execute any one of the methods described above.

[0082] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present disclosure (including the claims) is limited to these examples. Within the scope of the present disclosure, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present invention as described above, which are not provided in detail for the sake of simplicity.

[0083] The embodiments of the present invention are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present invention should be included in the scope of protection of this disclosure.

Claims

1. A traffic prediction method based on deep learning, characterized by: The traffic prediction method based on deep learning includes: Obtaining a historical representation of traffic state information and spatiotemporal information representing a first number of historical time steps and a future representation of spatiotemporal information representing a second number of future time steps; Applying a first BERT model to process the historical representation to obtain a first state code; Adding the first state code to the future representation to obtain a predicted representation; A second BERT model is applied to process the predicted representation to obtain a predicted traffic state. Applying the first BERT model to process the historical representation to obtain a first state code includes: Performing temporal attention calculation on the historical representation to obtain a first temporal attention code; Performing spatial attention calculation on the first temporal attention code to obtain a first spatial attention code; Performing layer normalization on the first spatial attention code to obtain a first state code; The performing temporal attention calculation on the historical representation to obtain a first temporal attention code includes: Decomposing the historical representation into time step and node granularity, and calculating a time input vector of the temporal attention of the current layer at any time step of any node in the first BERT model based on the historical representation, the time input vector including a time query vector, a time key vector, and a time value vector; Applying an activation function to calculate a first time attention weight of a current layer according to the time query vector and the time key vector; The first time attention weight of the current layer and the time value vector are weightedly summed, and a residual connection is performed with the first time attention code of the previous layer to obtain the first time attention code of the current layer.

2. The traffic prediction method based on deep learning according to claim 1, characterized in that: The performing spatial attention calculation on the first temporal attention code to obtain the first spatial attention code includes: Calculating a spatial input vector of the spatial attention of a current layer at any time step of any node in the first BERT model according to the first temporal attention encoding, wherein the spatial input vector includes a spatial query vector, a spatial key vector, and a spatial value vector; Calculate a first spatial attention weight of a current layer by applying an activation function according to the spatial query vector and the spatial key vector; The first spatial attention weight of the current layer and the spatial value vector are weightedly summed, and a residual connection is performed with the first temporal attention code of the current layer to obtain the first spatial attention code of the current layer.

3. The traffic prediction method based on deep learning according to claim 1, characterized in that: The performing layer normalization processing on the first spatial attention code to obtain a first state code includes: Applying a feedforward network to process the first spatial attention encoding; The first spatial attention code is superimposed on the output of the feedforward network to obtain a first state code.

4. The traffic prediction method based on deep learning according to claim 1, characterized in that: The adding the first state code to the future representation to obtain a predicted representation includes: If the first number is smaller than the second number, performing random number padding on the first state code of the first number of time steps to extend it to the second number of time steps; If the first number is greater than the second number, the future representation of the second number of time steps is padded with zeros to extend it to the first number of time steps.

5. The traffic prediction method based on deep learning according to claim 1, characterized in that: Applying the second BERT model to process the predicted representation to obtain the predicted traffic state includes: Performing temporal attention calculation on the predicted representation to obtain a second temporal attention code; Perform spatial attention calculation on the second temporal attention code to obtain a second spatial attention code, and perform layer normalization on the second spatial attention code to obtain a second state code, where the second state code is the predicted traffic state.

6. The traffic prediction method based on deep learning according to claim 5, characterized in that: The performing temporal attention calculation on the predicted representation to obtain a second temporal attention code includes: Decomposing the prediction representation into time step and node granularity, and calculating a time input vector of the temporal attention of the current layer at any time step of any node in the second BERT model based on the prediction representation, the time input vector including a time query vector, a time key vector, and a time value vector; Calculate a second temporal attention weight of the current layer by applying an activation function according to the temporal query vector and the temporal key vector; The second time attention weight of the current layer is weightedly summed with the time value vector, and a residual connection is performed with the second time attention code of the previous layer to obtain the second time attention code of the current layer.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method according to any one of claims 1 to 6 is implemented.

8. A computer storage medium, characterized in that: The storage medium stores at least one executable instruction, and the executable instruction enables the processor to execute the method according to any one of claims 1 to 6.

Images