Flow and trajectory combined urban spatio-temporal data dynamic representation learning model and method

Abstract

The invention discloses an urban spatio-temporal data dynamic representation learning model and method combining flow and trajectory, and the model comprises a traffic flow-based road network dynamic representation learning module which is used for coding a road network and a traffic flow sequence to obtain a first road network dynamic representation matrix; the track dynamic representation learning module is used for coding the road network and the dynamic transition probability matrix to obtain a second road network dynamic representation matrix; the cooperative attention Transform encoder is used for exchanging space-time information in the first road network dynamic representation matrix and the second road network dynamic representation matrix; and respectively inputting the exchanged representation matrix into a road network dynamic representation learning module and a track dynamic representation learning module based on the traffic flow to generate a road network dynamic representation matrix and a track dynamic representation vector. The model can provide more accurate spatio-temporal data representation for downstream tasks.

Description

Technical Field

[0001] The present invention relates to the fields of artificial intelligence, deep learning, smart cities, urban computing and computer applications, and more specifically to a learning model and method for dynamic representation of urban spatiotemporal data combining traffic and trajectory. Background Art

[0002] Driven by the wave of informatization and the rapid development of data science, smart cities have become a strategic choice for achieving sustainable urban development and improving people's quality of life in the future. Smart transportation is the core of smart city construction. The development of smart transportation is conducive to reducing traffic congestion, improving traffic safety, optimizing resource allocation and improving environmental quality. How to make good use of massive spatiotemporal data and serve urban planning and operation and the lives of urban residents is the key to realizing smart transportation.

[0003] Spatiotemporal data representation learning is a key technology in the field of spatiotemporal data mining. Its main goal is to map spatiotemporal data into a vector representation space so that downstream tasks such as prediction and clustering can be performed more easily. Spatiotemporal data representation learning usually involves the use of deep learning models to capture patterns and associations in spatiotemporal data, help understand various complex spatiotemporal relationships, improve prediction and decision-making capabilities, and provide key technical support for research in smart city infrastructure, spatiotemporal large models, and other fields. Among them, vehicle flow and trajectory are the two most common types of spatiotemporal data in traffic scenarios.

[0004] The goal of the road network representation learning method based on traffic flow is to convert the road segments in the road network into a general, low-dimensional representation vector based on traffic flow data. The difficulty of this technology lies in how to model the dynamic spatiotemporal correlation between road segments. From the perspective of spatial correlation, the traffic conditions of a road segment are highly correlated with the traffic conditions of other road segments. For example, a road segment with a short distance has a greater direct impact on the traffic conditions than a road segment with a long distance, and the traffic conditions between upstream and downstream sections will affect each other, etc. From the perspective of temporal correlation, the traffic conditions of a road segment will show a certain trend in the short term, and since traffic data is generated by human daily activities, there are certain periodic laws. In addition, the traffic conditions of the road network are constantly changing. For example, there are obvious differences in the traffic conditions of the road network during peak hours and non-peak hours. At the same time, various factors such as weather changes and traffic accidents also cause the spatiotemporal correlation between road segments to be not fixed, but highly dynamic over time.

[0005] The goal of trajectory representation learning is to convert trajectory data into a universal, low-dimensional representation vector. The difficulty of this technology is mainly reflected in two aspects: first, the low-dimensional representation vector of the trajectory needs to retain the spatiotemporal characteristics of the original trajectory as much as possible, and should be dynamic, that is, the trajectory representation vectors of the same path at different times are different; second, the low-dimensional representation vector of the trajectory needs to have good versatility and robustness so that it can be applied to various trajectory-related downstream tasks, such as trajectory travel time prediction, trajectory next location prediction, and most similar trajectory search.

[0006] In addition, spatiotemporal data are usually multi-source heterogeneous and difficult to integrate effectively, which also leads to the fact that existing spatiotemporal data representation learning work often only targets a single type of spatiotemporal data. However, many spatiotemporal data are not independent of each other, but have certain spatiotemporal correlations. Taking traffic flow and trajectory as an example, traffic flow is the macro-emergence result of a large number of individual trajectories, and at the same time, traffic flow also affects the trajectory selection of individual travel. Joint modeling of different spatiotemporal data can enrich the spatiotemporal information learned by the model, thereby learning more effective spatiotemporal data representation vectors, but no work has yet conducted relevant research.

[0007] In the related work of traffic flow prediction, some methods have tried to introduce trajectory data. The MDTP model uses multi-source trajectory data to count the traffic data of different travel modes for prediction, but does not use the road segment transfer pattern information contained in the trajectory. Models such as T-wave and TrajNet capture the multi-hop spatial correlation of road segments by propagating information along the real trajectory. The TrGNN model counts the road segment transfer probability in different time periods based on the road segment trajectory data, and integrates it into the deep learning framework based on the spatiotemporal graph. However, these works only focus on the single task of traffic flow prediction, and none of them can obtain a universal representation of road networks and trajectories.

[0008] In the work of trajectory representation learning, the Trembr model and the TMRN model use the Road2Vec model to learn the road network representation matrix. The Toast model extends the hop model by introducing an auxiliary context prediction task, so that the road network representation matrix can encode semantic information such as traffic patterns. Models such as CSTTE and CSTRM initialize a road segment embedding matrix randomly or based on road segment features, and synchronously train the road network representation matrix through trajectory self-supervised learning tasks. In addition, contrastive trajectory representation learning models such as PIM, WSCCL, and JCLRNT all introduce the comparison between the road segment trajectory representation vector and the road segment representation vector. However, the road network representation matrices obtained by the above works are all static, and none of them consider the impact of dynamic changes in traffic flow on the road network representation matrix.

[0009] Therefore, in the field of smart transportation, effective modeling and representation of spatiotemporal data is crucial. However, existing research focuses on a single type of data, such as traffic flow or trajectory data, and often ignores the impact of spatiotemporal dynamics. Future research needs to pay more attention to the joint modeling of different types of data, capturing the complex spatiotemporal dependencies between traffic flow and individual trajectories to achieve more accurate predictions and decisions, which is an urgent problem that technicians in this field need to solve. Summary of the invention

[0010] In view of this, the present invention provides a dynamic representation learning model and method for urban spatiotemporal data combining traffic and trajectory, which can at least partially solve the above technical problems.

[0011] In order to achieve the above object, the present invention adopts the following technical solution:

[0012] The embodiment of the present invention provides a dynamic representation learning model of urban spatiotemporal data combining traffic and trajectory, including:

[0013] A road network dynamic representation learning module based on vehicle flow is used to encode the road network and vehicle flow sequence to obtain a first road network dynamic representation matrix;

[0014] A trajectory dynamic representation learning module is used to encode the road network and the dynamic transition probability matrix to obtain a second road network dynamic representation matrix;

[0015] A collaborative attention Transformer encoder for exchanging spatiotemporal information between the first road network dynamics representation matrix and the second road network dynamics representation matrix;

[0016] The first road network dynamic representation matrix obtained after exchanging spatiotemporal information is input into the road network dynamic representation learning module based on vehicle flow, and a third road network dynamic representation matrix is ​​output;

[0017] The second road network dynamic representation matrix obtained after exchanging spatiotemporal information is input into the trajectory dynamic representation learning module, and combined with the road segment sequence and the time sequence, a trajectory dynamic representation vector is output.

[0018] It can be seen from the above technical solution that compared with the prior art, the present invention has the following advantages:

[0019] (1) The present invention proposes a road network dynamic representation learning module DRNRL based on traffic flow. Through the road network dynamic encoder, the complex dynamic spatiotemporal correlation between road sections is captured to obtain a dynamic representation matrix of the road network. Pre-training can be performed through two autoregressive self-supervised learning to mine the intrinsic patterns in traffic flow data.

[0020] (2) The present invention proposes a trajectory dynamic representation learning module DTRL, which perceives the static geographic and dynamic semantic spatial correlations between road segments through a dynamic transfer pattern-aware graph attention network, extracts temporal features in the trajectory through a temporal embedding layer, and then captures the global dependencies visited in the trajectory through a trajectory encoder to obtain a dynamic representation vector of the trajectory. Pre-training can be performed jointly through generative and contrastive self-supervised learning tasks.

[0021] (3) The present invention proposes a spatiotemporal data dynamic representation learning model DRNTRL that combines vehicle flow and trajectory. It models vehicle flow and road section trajectory respectively through two parallel streams, and uses a collaborative attention Transformer encoder to exchange the spatiotemporal information in these data to simultaneously obtain the dynamic representation matrix of the road network and the dynamic representation vector of the trajectory.

[0022] (4) The present invention designs a contrastive learning pre-training task, namely, the Trajectory-RoadNetwork Dynamic Matching (TRNDM) task, which maximizes the consistency between the trajectory dynamic representation vector and the road network dynamic representation vector based on traffic flow. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0024] Figure 1 The present invention is provided with an overall structural diagram of a dynamic representation learning model of urban spatiotemporal data combining traffic and trajectory. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0026] Reference Figure 1 As shown, the embodiment of the present invention discloses a dynamic representation learning model of urban spatiotemporal data combining traffic and trajectory, including:

[0027] The road network dynamic representation learning module based on traffic flow is used to encode the road network and traffic flow sequence to obtain the first road network dynamic representation matrix. It models the dynamic spatial correlation of long and short distances between road sections through the multi-head graph attention layer, and models the dynamic temporal correlation of long and short distances on the road section through the temporal self-attention mechanism.

[0028] The trajectory dynamic representation learning module is used to encode the road network and the dynamic transition probability matrix to obtain the second road network dynamic representation matrix. It obtains the dynamic representation vector of the trajectory by modeling the long-distance dependencies visited in the trajectory and identifying the global semantics of the trajectory.

[0029] The collaborative attention Transformer encoder is used to exchange the spatiotemporal information in the first road network dynamic representation matrix and the second road network dynamic representation matrix. It is responsible for exchanging information in the road network dynamic representation matrices of different perspectives.

[0030] The first road network dynamic representation matrix obtained after exchanging spatiotemporal information is input into the road network dynamic representation learning module based on vehicle flow, and a third road network dynamic representation matrix is ​​output;

[0031] The second road network dynamic representation matrix obtained after exchanging spatiotemporal information is input into the trajectory dynamic representation learning module, and combined with the road segment sequence and the time sequence, a trajectory dynamic representation vector is output.

[0032] The present invention proposes a spatiotemporal data dynamic representation learning model DRNTRL that combines vehicle flow and trajectory, which consists of a road network dynamic representation learning module DRNRL based on vehicle flow, a trajectory dynamic representation learning module DTRL, and a collaborative attention Transformer encoder. It can provide more accurate spatiotemporal data representation for downstream tasks.

[0033] The DRNRL module encodes the road network and traffic flow to obtain the first road network dynamic representation matrix from a macro perspective. The road network and dynamic transition probability matrix will be passed through the DTRL module to obtain the second road network dynamic representation matrix from a micro perspective. The collaborative attention Transformer encoder is responsible for exchanging information in the road network dynamic representation matrices from different perspectives. The exchanged representation matrices are then input into the DRNRL module and the DTRL module respectively to generate the road network dynamic representation matrix and trajectory dynamic representation vector.

[0034] The technical solution of the present invention is described in detail below:

[0035] 1. Road Network Dynamic Representation Learning Module DRNRL based on Traffic Flow

[0036] 1.1 Data Embedding Layer

[0037] The data embedding layer maps road network information and historical traffic flow sequences into a representation space, which not only retains the original information of the data but also adapts to the input requirements of deep neural networks.

[0038] The input of this module is road network data G and historical traffic flow sequence S t , the output is the spatiotemporal data embedding vector R is the road network; T is the time step, N is the number of spatial nodes, d x is the embedding dimension.

[0039] 1.2 Road Network Dynamic Encoder

[0040] The road network dynamic encoder is the core module of the DRNRL model. Its main function is to model the complex dynamic spatiotemporal correlation in the road network based on the intermediate representation output by the data embedding layer. The spatiotemporal encoder consists of a spatial encoder and a temporal encoder. The spatial encoder is used to model the dynamic spatial correlation of long and short distances, while the temporal encoder is used to model the dynamic temporal correlation of long and short distances.

[0041] 1.2.1 Spatial Encoder

[0042] The present invention introduces a global adjacency matrix that is aware of geographic neighbors and semantic neighbors From the perspective of short-distance modeling, this embodiment stipulates that if the distance between two road sections is less than a threshold E, they are geographical neighbors (GN); from the perspective of long-distance modeling, this embodiment proposes to use time series similarity algorithms such as cosine similarity or dynamic time warping to calculate the similarity of traffic flow sequences of any two road sections, and select the K most similar road sections for each road section as its semantic neighbors (SN). The geographical neighbors and semantic neighbors of each road section in the road network together constitute the global adjacency matrix Formally, the output of the spatial encoder in layer l is It can be calculated as follows:

[0043]

[0044] Among them, || represents the concatenation operation, GAT(·,·) represents the computational operation of the multi-head graph attention layer, From X (l)The embedding matrix of the time period ti extracted from . Since the spatial correlation between road sections changes dynamically over time, for example, the spatial correlation between peak and non-peak sections is completely different. In addition, various factors such as weather changes and traffic accidents will also cause the spatial correlation between road sections to be not fixed, but highly dynamic over time. Therefore, this model uses the graph attention layer to separately calculate the attention weights between road sections in different time periods for feature aggregation, that is, it realizes the modeling of the dynamic spatial correlation of long and short distances between road sections. It should be noted that considering that the number of road sections in the road network may be very large, directly using an ordinary graph attention layer may lead to insufficient video memory during training. Therefore, in specific operations, a sparse version of the graph attention layer is used to reduce the video memory required for model training.

[0045] 1.2.2 Time Encoder

[0046] The temporal encoder of the present invention uses a temporal self-attention layer to model the long-term and short-term dynamic temporal correlations on the road sections in the traffic flow data. The output l of the temporal encoder in the layer can be calculated as follows:

[0047]

[0048] Among them, SelfAttention(·) represents the computational operation of the temporal self-attention mechanism, From X (l) The road segment v extracted from j The traffic conditions of a road section will show a certain trend in the short term, that is, the traffic conditions of the road section in the current time period are correlated with the traffic conditions in the previous time periods. However, this correlation is affected by many factors, resulting in it not being fixed but changing dynamically over time.

[0049] Therefore, this model uses the temporal self-attention mechanism to calculate the attention weights of the current time period of each road section and its past T-1 time periods to achieve modeling of long-term and short-term dynamic temporal correlations on the road section. The temporal self-attention mechanism of this model adopts a multi-head attention mechanism, the goal of which is to learn a richer and more comprehensive feature representation and enhance the model's expressive power. j The input is (To make the formula neater, (l) is omitted below.) For the kth attention head, we first pass through three different mapping matrices to obtain the query matrix Key Matrix Sum Matrix

[0050]

[0051] in, is a learnable parameter. Then, based on the query matrix and the key matrix, the road segment v can be calculated j The time correlation between all time periods above is used to obtain the attention score matrix

[0052]

[0053] Based on the attention score matrix and the value matrix, we can further get the output of the kth attention head in the temporal encoder:

[0054]

[0055] Among them, softmax(·) represents the calculation operation of soft maximization. Finally, the multi-head attention mechanism will splice the outputs of H attention heads calculated in parallel, and then use a mapping matrix to fuse the information in these attention heads:

[0056]

[0057] in, are learnable parameters.

[0058] Taking the spatiotemporal encoder of the lth layer as an example, it will concatenate the outputs of the temporal encoder and the spatial encoder to form a fused representation

[0059]

[0060] The fused representation is residually connected with the input of the road network dynamic encoder layer and is subjected to layer normalization calculation operations:

[0061] Z1 (l) =LayerNorm(X (l) +Z (l) ),

[0062] Among them, LayerNorm(·) represents the calculation operation of layer normalization. Then, It will pass through a feedforward network (FFN) consisting of two layers of linear transformation and activation function:

[0063]

[0064] Wherein, GeLU(·) represents the computational operation of Gaussian Error Linear Unit (GELU); Respectively represent the weight parameters of the lth layer; They represent the bias term of the lth layer respectively; subscripts 1 and 2 represent different parameters. Finally, the output of the feedforward network is residually connected with the output of the previous layer, and the output of the road network dynamic encoder layer of the lth layer is obtained by layer normalization.

[0065] Z (l+1) =LayerNorm(Z1 (l) +FFN(Z1 (l) )).

[0066] 1.2.3 Output Layer

[0067] Based on the results of the road network dynamic encoder, the output layer is responsible for outputting the final generated road network dynamic representation matrix. This model uses a 1×1 convolutional jump connection layer to obtain the final representation of the road network in time period t

[0068] 1.3 Pre-training Tasks

[0069] The present invention uses self-supervised learning tasks to pre-train the DRNRL model and designs two autoregressive self-supervised tasks so that the generated road network representation matrix can learn spatiotemporal context information based on traffic flow data, thereby reflecting the dynamic changes of traffic conditions on the road network.

[0070] 1.3.1 Traffic flow reconstruction

[0071] The first task is Traffic Flow Reconstruction (TFR), which aims to reconstruct the original complete data from the noisy input data. First, a part of the traffic flow data in the traffic flow sequence is randomly masked to 0, and then the DRNRL model is used to rely only on the remaining data to understand the spatiotemporal context of the input and map the observed data to a latent representation. Finally, the decoder uses the latent representation and masked tokens to reconstruct the masked traffic flow data. However, the masking strategy of spatiotemporal data is different from that in the field of computer vision or natural language processing. The information density of traffic flow data is relatively low and it is redundant in the time dimension, that is, it is easy to restore missing values ​​through the data of the context time period, resulting in the model being unable to capture the changing trend of the traffic flow sequence.

[0072] Therefore, the present invention randomly assigns each road segment v in the road network G to i The traffic flow sequence S t,:i: The length of the middle section is T M The continuous subsequence of traffic flow is masked to 0, and the road network representation X generated from time period tT to time period t-1 based on the DRNRL model is required t skip, reconstruct the masked traffic subsequence. In addition, T M You need to choose a suitable ratio (for example, T M =T / 2). If T M If T is too small, it will reduce the difficulty of model learning, resulting in the model being unable to fully utilize spatiotemporal data and learn spatiotemporal context information. M If it is too large, it will result in too little effective information, making it too difficult to learn the model and unable to obtain effective spatiotemporal data representation. In terms of the decoder, the DRNRL model uses a simple fully connected layer to convert the potential representation into reconstructed traffic flow data. The corresponding loss function is defined as follows:

[0073]

[0074] Among them, M t,i It is road segment v i The historical traffic flow sequence S t,:i: The time period set of the masked traffic flow subsequence in , Yes X t skip Middle section v i The representation vector at time segment j, It is road segment v i Traffic flow data in time period j.

[0075] 1.3.2 Traffic flow prediction

[0076] The second task is traffic flow prediction (TFP). The present invention uses the road network representation matrix for time period t generated based on the DRNRL model. To predict the traffic flow data for time period t On the decoder side, a simple fully connected layer FC(·) is used: s →C ts , to convert the road network representation into predicted traffic flow data. The corresponding loss function is defined as follows:

[0077]

[0078] in, Yes H t Middle section v i In the time period t, It is road segment v i Traffic flow data at time period t.

[0079] 2. Trajectory Dynamic Representation Learning Module DTRL

[0080] 2.1 Dynamic Transfer Pattern-Aware Graph Attention Network

[0081] The present invention proposes a dynamic transfer pattern-aware graph attention network for road segment trajectories to obtain a dynamic representation matrix of the road network, which is used to encode the road segment sequence of the road segment trajectory to obtain the road network dynamic representation matrix.

[0082] Road segment track is a time series of adjacent road segments, recording the individual's movement behavior within the road network G, where m is the number of road segments, v i ∈V is the road segment visited for the i-th time, t i is the corresponding access timestamp;

[0083] Specifically, this work divides a week into 7×48=336 time slices with an interval of 30 minutes, and then introduces the dynamic transition probability p t,i,j To describe the road segment v in different time slices i and road segment v j Dynamic transfer mode between. Dynamic transfer probability p t,i, j is calculated based on the segment trajectory data as follows:

[0084] p t,i,j = count(v j →v i |t) / count(v i |t),

[0085] Among them, count(v j →v i |t) and count(v i |t) are the paths (v j ,v i ) and segment v i Next, this embodiment will dynamically transfer the probability p t,i,j The information is integrated into the standard L g In the layer-by-layer graph attention network. For time slice t, in the layer-l graph attention layer, the road segment v i The hidden state feature is The output features are The superscript d l represents the feature dimension of the lth layer, d l+1 Represents the feature dimension of the l+1th layer. i and road segment v j The correlation score between t,i,j and the dynamic normalized attention weight α t,i,j It is calculated as follows:

[0086] e t,i,j =(h t ' ,iW1+h t ' ,j W2+p t,i,j W3)W4,

[0087]

[0088] in, and are all learnable parameters, N i It is road segment v i The set of geographical neighboring road segments, LeakyReLU(·) represents the calculation operation of the leaky rectified linear unit. Then, the road segment v i Output characteristics is based on the road segment v i The attention weights of its neighboring road segments are aggregated to obtain the features of its neighboring road segments:

[0089]

[0090] in, is a learnable parameter, and ELU(·) represents the computational operation of the exponential linear unit. In addition, the graph attention network used in this study is a multi-head attention mechanism. Finally, GAT will concatenate the output results of each attention head to obtain the final output of the l-th layer GAT

[0091] 2.2 Temporal Embedding Layer

[0092] The time series of the trajectory is the original timestamp information, and the time characteristics of the trajectory are also very important for the dynamic representation vector of the trajectory. For example, the representation vector of the trajectory through the same path should be different in different time periods. Therefore, the present invention introduces a time embedding layer to convert the original timestamp in the trajectory into a representation vector.

[0093] First, human activities are cyclical, and the time embedding layer uses the embedding vector of the week within the week and intraday minute embedding vector To characterize the periodic time information in the timestamp:

[0094]

[0095] The weekly(·) function first converts t i Convert to a weekday index (i.e. an integer from 0 to 6), and then embed the discrete index into d t dimensional vector; the daily(·) function first converts t i Convert to intraday minute index (i.e. integer from 0 to 1439), and then embed the discrete index into d t dimensional vector.

[0096] In addition, since the trajectory encoder of the DTRL model is implemented based on the self-attention mechanism, the position order information visited in the trajectory is lost during the calculation process. To solve this problem, this module imitates the traditional Transformer model and uses sine and cosine functions of different frequencies to obtain the positional encoding (PE) embedding vector To reflect the location information of the access in the track:

[0097]

[0098] Finally, in order to retain as much timestamp information as possible, j is the feature dimension number, that is, an integer from 0 to dt / 2-1; this model also introduces a time difference embedding vector To reflect the time difference between the access timestamp and the departure timestamp:

[0099]

[0100] Among them, FC(·):1→d t Represents the computational operation of the fully connected layer, and t0 represents the timestamp of the first access.

[0101] 2.3 Track encoder

[0102] Based on the graph attention network and temporal embedding layer that perceives dynamic trajectory transfer patterns, the present invention can convert the original trajectory T into trajectory feature embedding The segment trajectory can be divided into segment sequence and time series. For the i-th visited segment v i , the dynamic road network representation H can be obtained from the graph attention network aware of dynamic trajectory transfer patterns t The corresponding road segment dynamic representation h is obtained t,i ; At the same time, for the timestamp t of the i-th access i , which can be input into the time embedding layer to get the embedding of the week Intraday minute embed Positional Encoding Embedding Vector and the time difference embedding vector Then, the overall feature embedding vector for the i-th visit in the trajectory can be derived

[0103]

[0104] In order to capture the long-distance dependencies visited in the trajectory and identify the global semantics of the trajectory, the present invention represents the trajectory as a sequence It is further input into the trajectory encoder composed of L layers of gated attention units (GAU). The output of the trajectory encoder is a sequence of representations accessed one by one in the trajectory. Therefore, it is necessary to perform a pooling operation on the sequence representation to obtain the representation vector of the entire trajectory. This invention imitates the BERT model and inserts a placeholder at the front of the trajectory. In the trajectory encoder, the placeholder will fully interact with all accesses of the entire trajectory, so the output representation vector of the placeholder can be used as the representation vector of the entire trajectory.

[0105] l T =GAU(ph,l1,...,l m )[0],

[0106] Where GAU(·) represents the computational operation of multi-layer gated attention units, is the embedding vector of the placeholder.

[0107] Each layer of gated attention unit consists of root mean square normalization (Root Mean Square Layer Normalization) and gated attention unit blocks. The input of this layer is normalized by root mean square and gated attention unit blocks, and then added with residual connection as the output of this layer, that is, the input of the next layer.

[0108] The gated attention unit block combines the gated linear unit (GLU) with the attention mechanism. It uses the gating mechanism to dynamically adjust the attention level of the input features, so that the model can more effectively capture long-distance dependencies while reducing noise interference, aiming to improve the model's ability to process sequence data. Assume that the input of the l-th layer gated attention unit is (To make the formula neater, (l) is omitted in the following.) X first passes through a dense layer and an affine transformation to obtain the query matrix and key matrix Then get the attention weight based on Q and K

[0109] Z=SiLU(XW Z ),

[0110] Q=Aff Q (Z),K=Aff K (Z),

[0111]

[0112] Among them, SiLU(·) represents the calculation operation of Sigmoid weighted linear unit, is a learnable parameter, Aff Q (·) and Aff K (·) represents the affine transformation applied to the query matrix and the key matrix respectively, and softmax(·) represents the soft maximization operation. Then, the gated attention unit performs attention weighting on the value matrix V in the gated linear unit based on the attention weight A to obtain the output of the gated attention unit block

[0113] U=SiLU(XW U ),

[0114] V=SiLU(XW V ),

[0115] O=(U⊙AV)W O ,

[0116] in, and is a learnable parameter and ⊙ represents the calculation operation of the Hadamard product.

[0117] 2.4 Pre-training Tasks

[0118] The present invention uses generative and contrastive self-supervised learning tasks to jointly pre-train the DTRL model, ensuring that the trajectory representation vector takes into account both local information such as the road segment topology and transfer pattern, as well as global information such as the overall travel semantics of the trajectory.

[0119] 2.4.1 Mask trajectory prediction

[0120] The generative self-supervised learning task of the present invention adopts a reconstruction-based generative task called Masked Trajectory Prediction (MTP), which randomly masks some continuous sub-segments and corresponding timestamps in the segment trajectory, and requires the model to predict the masked segments and timestamps.

[0121] 2.4.2 Contrastive trajectory learning task

[0122] The contrastive self-supervised learning task of the present invention adopts the contrastive trajectory learning task (CTL). The overall idea is to apply the contrastive learning strategy to generate multiple samples of trajectories from different views, and make semantically similar samples closer in the representation space, while dispersing dissimilar samples. The core of this task is how to construct a positive trajectory with similar semantics to the anchor trajectory. The present invention uses two data enhancement strategies for trajectories:

[0123] (1) Trajectory clipping: This strategy generates positive trajectories by clipping a small sub-trajectory at the beginning or end of the trajectory. The starting point of this strategy is that the travel semantics of trajectories that have only slight differences in the beginning or end are similar.

[0124] (2) Temporal perturbation: This strategy randomly selects the access timestamps of a small number of road segments for perturbation. Since traffic scenes have large uncertainties, this strategy can make the trajectory representation vector robust to slight temporal perturbations.

[0125] 3. Co-Attention Transformer Encoder

[0126] The traffic flow-based road network dynamic representation module DRNRL and the trajectory dynamic representation module DTRL can obtain the road network dynamic representation matrices under the macro perspective and the micro perspective respectively. If the information exchange between the two perspectives is further realized, a richer dynamic representation of the road network can be learned. The principle behind this is that the dynamic characteristics of the road network include both the dynamic changes of the traffic flow data on the road section and the dynamic transfer patterns between sections in the section trajectory data. To this end, the present invention proposes a collaborative attention Transformer encoder to capture the information exchange from multiple perspectives, and interacts with the two parallel streams through the collaborative attention Transformer encoder.

[0127] From the microscopic perspective of the road segment trajectory data, the present invention uses to represent the road segment v generated by the dynamic transfer mode-aware graph attention network i The road segment in time period t is represented (t is omitted in the following to make the formula more tidy). In order to model the information of the macro perspective of traffic flow data Influence, this embodiment uses a cross-view graph attention network. Inspired by the urban gravity model, the present invention assumes that the influence between two road segments decreases with the distance between them. Finally, the microscopic view of the road segment v designed in this embodiment i and road segment v j The correlation score between and the normalized attention weights between The calculation is as follows:

[0128]

[0129] in, and is a learnable parameter, represents the road segment v generated by the dynamic representation model of the road network based on traffic flow jIn the representation vector of time period t, the function deter(·) represents the distance decay function (such as negative power function, etc.), the function dist(·) represents the geographically defined distance function between two road segments, and LeakyReLU(·) represents the calculation operation of the leaky rectified linear unit. Cross-view graph attention network uses normalized attention weights To aggregate the features of the neighborhood road segments from a macro perspective, in order to improve the feature representation of the target road segment from a micro perspective. The rest of the collaborative attention Transformer encoder is the same as the traditional Transformer model.

[0130] Similarly, from the macro perspective of traffic flow data, in order to model the information of the micro perspective of road trajectory data, In the cross-view graph attention network, the road segment v i and road segment v j The correlation score between and road segment v i and road segment v j ∈R i The normalized attention weights between The calculation is as follows:

[0131]

[0132] in, and is a learnable parameter. Cross-view graph attention network uses normalized attention weights To aggregate the features of the neighborhood road segments from a micro perspective, so as to improve the feature representation of the target road segment from a macro perspective.

[0133] Based on the collaborative attention Transformer encoder, the micro-representation of a road segment can aggregate the features of its neighboring road segments from a macro perspective, so that when performing trajectory dynamic representation learning, dynamic features such as the traffic conditions of the road segment can be obtained; similarly, the macro-representation of a road segment can aggregate the features of its neighboring road segments from a micro perspective, so that when performing road network dynamic representation learning based on traffic flow, information such as the dynamic transfer patterns between road segments can be obtained.

[0134] 4. Trajectory-road network dynamic matching task

[0135] In terms of training tasks, based on the road network dynamic representation learning pre-training tasks and trajectory dynamic representation learning pre-training tasks, the present invention also designs a comparative learning pre-training task, namely the Trajectory-RoadNetwork Dynamic Matching (TRNDM) task, which maximizes the consistency between the trajectory dynamic representation vector and the road network dynamic representation vector based on traffic flow. Specifically, given a trajectory representation vector and a road network representation matrix, it is necessary to predict whether the two are aligned.

[0136] For the segment trajectory It represents the vector It can be obtained through the trajectory dynamic representation learning module in the DRNTRL model. At the same time, the corresponding segment sequence of the segment trajectory The representation matrix It can be obtained from the traffic flow-based road network dynamic representation learning module in the DRNTRL model:

[0137]

[0138] in, Indicates from The road segment v obtained in i The representation vector at timestamp t1. Then, this embodiment can be The first dimension of performs an average pooling operation to obtain the representation vector of the road segment trajectory directly obtained from the road network representation matrix

[0139] In the process of comparative learning, the present invention will B t The spatiotemporal data of a set of time periods Θ is regarded as a training batch. For each trajectory T in the time period t∈Θ, this embodiment will In order to construct negative sample pairs, this example obtains the road network representation matrix based on traffic flow from different time periods t′ in the time period set Θ And in the same way we get A negative sample pair can be constructed as That is, the representation vectors of the same road section sequence at different times are regarded as negative samples. Formally, for the positive sample Normalized Temperature-Scaled Cross Entropy Based on Contrastive Learning is defined as:

[0140]

[0141] Where τ represents the temperature parameter and sim(·) is the calculation operation of the vector similarity function (this work uses cosine similarity as the vector similarity function). The overall loss L for the entire batch in contrastive learning is Match It is obtained by averaging the loss functions of all positive sample pairs in the batch.

[0142] The DRNTRL model is jointly trained using the pre-training tasks of the DRNRL module, the pre-training tasks of the DTRL module, and the trajectory-road network dynamic matching task. The overall loss function L is defined as follows:

[0143] L=λ Traj L Traj +λ Traf L Traf +λ Match L Match ,

[0144] Among them, λ Traj , Traf and λ Match They are the hyperparameters that control the impact of each loss function on the joint training of the DRNTRL model.

[0145] The spatiotemporal data dynamic representation learning model of joint traffic and trajectory provided by the present invention can be used to pre-train the model in the self-supervised learning tasks of vehicle flow reconstruction, vehicle flow prediction, mask trajectory prediction and comparative trajectory learning, and a new self-supervised learning task is proposed to align the spatiotemporal information contained in the trajectory and the road network.

[0146] The specific execution steps of the above joint flow and trajectory spatiotemporal data dynamic representation learning model are as follows:

[0147] S1: The road network and traffic flow sequence are first converted into a high-dimensional spatiotemporal representation vector through the data embedding layer of the DRNRL module;

[0148] S2: The high-dimensional spatiotemporal representation vector is input into the spatial encoder and temporal encoder of the road network dynamic encoder respectively to capture the spatiotemporal dependencies of the traffic flow and convert it into an intermediate representation.

[0149] S3: The intermediate representation is passed through the output layer to obtain the final generated road network dynamic representation matrix from a macro perspective.

[0150] S4: The road network and dynamic transition probability matrix will pass through the dynamic transfer pattern-aware graph attention layer of the DTRL module to obtain the road network dynamic representation matrix from a microscopic perspective.

[0151] S5: The road network dynamic representation matrix from a macro perspective obtained by the DRNRL module and the road network dynamic representation matrix from a micro perspective obtained by the DTRL module are input into the collaborative attention Transformer encoder to exchange information in the road network dynamic representation matrices from different perspectives.

[0152] S6: The road network dynamic representation matrix from a macro perspective output by the collaborative attention Transformer encoder will be passed through the road network dynamic encoder of the DRNRL module to obtain the final generated road network dynamic representation matrix.

[0153] S7: The road network dynamic representation matrix is ​​used for two road network dynamic representation self-supervised learning tasks, and the loss L of the road network dynamic representation learning module based on traffic flow is calculated. Traf .

[0154] S8: The road network dynamics representation matrix at the microscopic perspective output by the co-attention Transformer encoder is used to process the road segment sequence of the road segment trajectory.

[0155] S9: The time series of the road segment trajectories processed by the time embedding layer of the DTRL module.

[0156] S10: The segment sequence and time series obtained in steps S8 and S9 are input into the trajectory encoder together, and the trajectory dynamic representation vector is generated through multiple layers of gated attention units.

[0157] S11: The trajectory dynamics representation vector is used for two trajectory dynamics representation self-supervised learning tasks, and the loss L of the trajectory dynamics representation learning module is calculated. Traj .

[0158] S12: The road network dynamic representation matrix obtained in step S6 and the trajectory dynamic representation vector obtained in step S10 are used for the trajectory-road network dynamic matching task to calculate L Match .

[0159] S13: Using the losses obtained in steps S7, S11, and S12, calculate the overall loss and optimize the model.

[0160] For example, using the taxi datasets of Xi'an and Chengdu in October and November 2018, after data preprocessing, a traffic flow speed dataset and a road segment trajectory dataset were constructed. After pre-training on the two datasets, experiments were conducted on four downstream tasks: traffic flow sequence prediction, traffic flow sequence missing value filling, travel time prediction, and trajectory next position prediction. The performance of the model proposed in this invention exceeds that of several existing methods, achieving accurate urban spatiotemporal prediction.

[0161] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0162] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A dynamic representation learning model for urban spatiotemporal data combining traffic and trajectory, characterized by: include: A road network dynamic representation learning module based on vehicle flow is used to encode the road network and vehicle flow sequence to obtain a first road network dynamic representation matrix; A trajectory dynamic representation learning module is used to encode the road network and the dynamic transition probability matrix to obtain a second road network dynamic representation matrix; A collaborative attention Transformer encoder for exchanging spatiotemporal information between the first road network dynamics representation matrix and the second road network dynamics representation matrix; The first road network dynamic representation matrix obtained after exchanging spatiotemporal information is input into the road network dynamic representation learning module based on vehicle flow, and a third road network dynamic representation matrix is ​​output; The second road network dynamic representation matrix obtained after exchanging spatiotemporal information is input into the trajectory dynamic representation learning module, and combined with the road segment sequence and the time sequence, a trajectory dynamic representation vector is output.

2. According to claim 1, a dynamic representation learning model of urban spatiotemporal data combining traffic and trajectory is characterized in that: The road network dynamic representation learning module based on traffic flow includes: The data embedding layer is used to map the road network information and historical traffic flow series into the representation space and output the spatiotemporal data embedding vector; The road network dynamic encoder is used to model the complex dynamic spatiotemporal correlation in the road network according to the spatiotemporal data embedding vector output by the data embedding layer to obtain a first road network dynamic representation matrix; it includes: The spatial encoder models the dynamic spatial correlation of long and short distances between road segments through a multi-head graph attention layer; Temporal Encoder: Modeling long-term and short-term dynamic temporal dependencies on road segments through temporal self-attention mechanism.

3. According to claim 2, a dynamic representation learning model of urban spatiotemporal data combining traffic and trajectory is characterized in that: In the road network dynamic encoder, the output of the spatial encoder It is expressed as follows: Where l represents the output layer; || represents the concatenation operation, T represents the length of the time series considered by the model; GAT(·,·) represents the computational operation of the multi-head graph attention layer, From X (l) The embedding matrix of the extracted time segment ti; R represents the road network; N represents the number of nodes in the road segment; d x represents the embedding dimension; A global adjacency matrix consisting of the geographical and semantic neighbors of each road segment in the road network; In the road network dynamic encoder, the output of the time encoder It is expressed as follows: Among them, SelfAttention(·) represents the computational operation of the temporal self-attention mechanism, From X (l) The road segment v extracted from j The embedding matrix of The overall output of the road network dynamic encoder includes: In the spatiotemporal encoder of layer l, the outputs of the temporal encoder and the spatial encoder are concatenated to form a fused representation This fused representation will be combined with the input X of the road network dynamics encoder layer (l) Perform residual connection and layer normalization calculation operations: After a feed-forward network FFN consisting of two layers of linear transformation and activation function: Where GeLU(·) represents the computational operation of the Gaussian error linear unit, Respectively represent the weight parameters of the lth layer; They represent the bias items of the lth layer respectively; subscripts 1 and 2 represent different parameters; The output of the feedforward network is residually connected to the output of the previous layer and normalized by layers to obtain the output of the road network dynamic encoder layer at layer l Among them, LayerNorm(·) represents the calculation operation of layer normalization, and the output Z (l+1) As the first road network dynamic representation matrix.

4. According to claim 1, a dynamic representation learning model of urban spatiotemporal data combining traffic and trajectory is characterized in that: Trajectory dynamic representation learning module, including: Dynamic transition pattern-aware graph attention network, which is used to fuse the input road network and dynamic transition probability matrix into the graph attention network; The time embedding layer converts timestamps into representation vectors, including day of week, minute of day, position encoding, and time difference embeddings; Trajectory encoder, which is used to capture the long-range dependencies and global semantics visited in the trajectory through multiple layers of gated attention units and outputs a trajectory dynamic representation vector.

5. According to claim 4, a dynamic representation learning model of urban spatiotemporal data combining traffic and trajectory is characterized in that: The dynamic transfer pattern-aware graph attention network comprises: Road segment track is a time series of adjacent road segments, recording the individual's movement behavior within the road network G, where m is the number of road segments, v i ∈V is the road segment visited for the i-th time, t i is the corresponding access timestamp; Determine the division of multiple time slices and introduce the dynamic transfer probability matrix p t,i ,j Describe the road segments v in different time slices i and road segment v j Dynamic transfer mode between; dynamic transfer probability p t,i ,j The calculation based on the segment trajectory data is as follows: p t,i,j =count(v j →v i |t) / count(v i |t), Among them, count(v j →v i |t) and count(v i |t) respectively represent the path (v j ,v i ) and segment v i Number of occurrences; The dynamic transition probability p t,i,j Fusion to standard L g In the layer graph attention network; for time slice t, in the lth layer graph attention layer, road segment v i The hidden state feature is The output features are The superscript d l represents the feature dimension of the lth layer, d l+1 represents the feature dimension of the l+1th layer; road segment v i and road segment v j The correlation score between t,i,j and the dynamic normalized attention weight α t,i,j It is calculated as follows: e t,i,j =(h′ t,i W1+h′ t,j W2+p t,i,j W3)W4, in, and Both represent learnable parameters, N i Represents road segment v i The set of geographical neighboring road segments, LeakyReLU(·) represents the calculation operation of the leaky rectified linear unit; the road segment v i Output characteristics is based on the road segment v i The attention weights of its neighboring road segments are aggregated to obtain the features of its neighboring road segments: in, represents a learnable parameter, and ELU(·) represents the computational operation of the exponential linear unit; The graph attention network adopts a multi-head attention mechanism. The graph attention layer concatenates the output results of each attention head to obtain the final output of the lth graph attention layer.

6. The urban spatiotemporal data dynamic representation learning model combining traffic and trajectory according to claim 5 is characterized in that: The time embedding layer comprises: Get the time series based on the trajectory and embed the vector using the day of the week and intraday minute embedding vector To characterize the periodic time information in the timestamp: Among them, the weekly(·) function converts t i Convert to a week-of-week index, and then embed the discrete index into d t dimensional vector; the daily(·) function converts t i Convert to intraday minute index, and then embed the discrete index into d t dimensional vector; Use sine and cosine functions of different frequencies to obtain the positional encoding embedding vector To reflect the location information of the access in the track: Among them, j is the feature dimension number, that is, an integer from 0 to dt / 2-1; a time difference embedding vector is introduced To reflect the time difference between the access timestamp and the departure timestamp: Among them, FC(·):1→d t Represents the computational operation of the fully connected layer, and t0 represents the timestamp of the first access.

7. The urban spatiotemporal data dynamic representation learning model combining traffic and trajectory according to claim 6 is characterized in that: The track encoder comprises: The dynamic representation h of the corresponding road segment in the graph attention network based on the dynamic transfer pattern perception t,i , and the week of the week, minute of the day, position encoding, and time difference embedding of the time embedding layer to derive the overall feature embedding vector for the i-th visit in the trajectory The trajectory represents the sequence Input is sent to the L-layer gated attention unit, and the output is a sequence of representations visited one by one in the trajectory Insert a placeholder at the front of the trajectory. The placeholder will fully interact with all the accesses of the entire trajectory. Finally, the output representation vector of the placeholder is used as the dynamic representation vector of the entire trajectory. l T =GAU(ph,l1,...,l m )[0], Where GAU(·) represents the computational operation of multi-layer gated attention units, Embedding vector representing the placeholder.

8. According to claim 4, a dynamic representation learning model of urban spatiotemporal data combining traffic and trajectory is characterized in that: The collaborative attention Transformer encoder comprises: (1) Based on the microscopic perspective of the road segment trajectory data, use to represent the road segment v generated by the dynamic transfer mode-aware graph attention network i The road segment in time period t is represented; The road section v under the microscopic perspective of design i and road segment v j The correlation score between and the normalized attention weights between The calculation is as follows: in, and is a learnable parameter, represents the road segment v generated by the dynamic representation model of the road network based on traffic flow j In the representation vector of time period t, the function deter(·) represents the distance decay function, the function dist(·) represents the geographically defined distance function between two road segments, and LeakyReLU(·) represents the calculation operation of the leaky rectified linear unit; Cross-view graph attention networks using normalized attention weights To aggregate the features of the neighborhood road segments from a macro perspective, so as to improve the feature representation of the target road segment from a micro perspective; (2) Based on the macroscopic perspective of traffic flow data, in the cross-perspective graph attention network, the road segment v i and road segment v j The correlation score between and road segment v i and road segment v j ∈R i The normalized attention weights between The calculation is as follows: in, and is a learnable parameter; the cross-view graph attention network uses normalized attention weights To aggregate the features of the neighborhood road segments from a micro perspective, so as to improve the feature representation of the target road segment from a macro perspective.

9. A method for learning dynamic representation of urban spatiotemporal data combining traffic and trajectory, characterized in that: Based on the urban spatiotemporal data dynamic representation learning model combining vehicle flow and trajectory as described in any one of claims 1 to 8, the method comprises: S1: The road network and traffic flow sequence are converted into high-dimensional spatiotemporal representation vectors through the data embedding layer of the road network dynamic representation learning module based on traffic flow; S2: inputting the high-dimensional spatiotemporal representation vector into the spatial encoder and the temporal encoder of the road network dynamic encoder respectively, capturing the spatiotemporal dependency of the traffic flow, and converting it into an intermediate representation; S3: The intermediate representation is passed through an output layer to obtain a final generated road network dynamic representation matrix from a macroscopic perspective as a first road network dynamic representation matrix; S4: The road network and dynamic transition probability matrix are passed through the graph attention network perceived by the dynamic transition pattern of the trajectory dynamic representation learning module to obtain the road network dynamic representation matrix from a microscopic perspective as the second road network dynamic representation matrix; S5: Input the first road network dynamic representation matrix and the second road network dynamic representation matrix into the collaborative attention Transformer encoder to exchange information in the road network dynamic representation matrices from different perspectives; S6: The road network dynamic representation matrix under the macroscopic perspective output by the collaborative attention Transformer encoder will be further passed through the road network dynamic representation encoder based on the traffic flow-based road network dynamic representation learning module to obtain the final generated road network dynamic representation matrix as the third road network dynamic representation matrix; S7: The third road network dynamic representation matrix is ​​used for two road network dynamic representation self-supervised learning tasks, and the loss L of the road network dynamic representation learning module based on traffic flow is calculated. Traf ; S8: The road network dynamic representation matrix at the microscopic perspective output by the cooperative attention Transformer encoder is used to process the road segment sequence of the road segment trajectory; S9: The time series of the road segment trajectory is processed by the time embedding layer of the trajectory dynamic representation learning module to obtain the time series; S10: The segment sequence and time sequence obtained in steps S8 and S9 are input into the trajectory encoder together, and the trajectory dynamic representation vector is generated through multiple layers of gated attention units; S11: The trajectory dynamics representation vector is used for two trajectory dynamics representation self-supervised learning tasks, and the loss L of the trajectory dynamics representation learning module is calculated. Traj ; S12: The road network dynamic representation matrix obtained in step S6 and the trajectory dynamic representation vector obtained in step S10 are used for the trajectory-road network dynamic matching task to calculate the loss L Match ; S13: Using the losses obtained in steps S7, S11, and S12, calculate the overall loss and optimize the model.

Images