Track data representation method based on deep self-supervised learning

Abstract

The invention discloses a trajectory data representation method based on deep self-supervised learning, and belongs to the technical field of traffic trajectory spatio-temporal data processing. The method solves the problems that the existing method needs to respectively design features for different application scenes, cannot fully mine the potential relationship between the time sequence features and the road network topology structure information, and is difficult to fully cover the application scenes. According to the method, a double-branch architecture of an online network and a target network is adopted, optimization targets of a contrast learning task and a mask task are deeply fused, and the model can capture global semantic relevance and local context features of trajectory data at the same time. According to the method, a cooperative working mechanism of the graph attention network and the time coding module is constructed, the graph attention network can analyze a spatial constraint relation of road topology, the time coding module can capture a motion time sequence rule of a moving object, and organic fusion of multi-modal information is realized through feature cascading and gradient sharing of the graph attention network and the time coding module. The method can be used for extracting representation of traffic track data.

Description

Technical Field

[0001] The present invention belongs to the technical field of traffic trajectory spatiotemporal data processing, and specifically relates to a trajectory data representation method based on deep self-supervised learning. Background Art

[0002] With the rapid development of the Internet of Things (IoT) and the Internet of Vehicles (IoV) technologies, trajectory data capable of describing the diversity and attributes of moving objects has exploded. Against the backdrop of accelerating urbanization, the contradiction between the rapid growth of transportation demand and the insufficient supply of transportation infrastructure has become increasingly prominent. Vehicle trajectory data on urban road networks has become an important research foundation for traffic prediction and management. It not only reflects the changes in vehicle positions over time but also contains high-level semantic information such as vehicle travel behavior and driving characteristics. Deep mining of trajectory data can support research and applications in multiple traffic-related fields, such as smart cities, intelligent transportation, and location-based services. In particular, within intelligent transportation systems (ITS), trajectory data analysis and management are crucial for improving traffic signal control strategies, vehicle routing, short-term traffic state prediction, and travel time estimation.

[0003] Traditional research on large-scale trajectory data has primarily focused on trajectory similarity metrics (such as dynamic time warping (DTW), longest common subsequence (LCSS), and edit distance real sequence (EDR)), trajectory clustering, trajectory anomaly detection, and path reachability. However, as trajectory data continues to grow in size and complexity, traditional methods face numerous challenges in processing high-dimensional and diverse data. In recent years, deep representation learning techniques have been increasingly applied to trajectory data analysis, transforming raw trajectory point sequences into low-dimensional, dense, and universal vector representations. However, existing supervised trajectory representation methods based on deep learning typically rely on large amounts of manually annotated data for model training, which is not only costly but also struggles to cover diverse or scarce application scenarios. Furthermore, current research often focuses on independent modeling for a single task, requiring repeated training and parameter tuning, resulting in low efficiency. For traffic trajectory data with irregular spatiotemporal distribution and significant non-Euclidean characteristics, traditional time series models or convolutional models struggle to fully capture its dynamic variations, further limiting their effectiveness.

[0004] In summary, the existing methods still have the following shortcomings:

[0005] (1) Insufficient model versatility: It needs to rely on expert knowledge to design features, lacks scalability, and requires redesigning features for different application scenarios (such as travel time estimation, vehicle type classification (Jiang Peng, Yao Zhihan, Liu Jun, Hu Hua, Xu Huan. Vehicle classification method based on short-term GPS trajectory data: China, CN112948715A[P]. 2021-06-11.), etc.), which increases the difficulty of application;

[0006] (2) Insufficient utilization of time series data and graph network topology information: The patent application with publication number CN116956224A failed to fully explore the potential relationship between the time series features in trajectory data and the topological structure information of the road network, which affected the model's expressive power and prediction accuracy. Summary of the Invention

[0007] The purpose of this invention is to solve the problems that existing methods need to rely on expert knowledge to design features for different application scenarios, cannot fully explore the potential relationship between time series features and road network topology information, and have difficulty in comprehensively covering application scenarios. A trajectory data representation method based on deep self-supervised learning is proposed.

[0008] The technical solution adopted by the present invention to solve the above technical problems is: a trajectory data representation method based on deep self-supervised learning, the method specifically comprising the following steps:

[0009] Step 1: Obtain the GPS trajectory data of the vehicle and the road network structure of the area covered by the GPS trajectory data, and then obtain the feature vector of each node in the road network based on the road network structure;

[0010] Step 2: Use the GPS trajectory data and the feature vectors of each node as input to the online network, and output the trajectory representation of the GPS trajectory data through the online network;

[0011] The working process of the online network is:

[0012] Step 21: Using GPS trajectory data and feature vectors of each node vector as inputs of the first encoder;

[0013] Step 22: using the output of the first encoder as the input of the first projection head;

[0014] Step 2 and step 3: Use the output of the first projection head as the input of the prediction head, and use the output of the prediction head as the trajectory representation of the GPS trajectory data.

[0015] Furthermore, the characteristic vector of each node in the road network is obtained according to the road network structure, and the specific process is as follows:

[0016] Each road section in the road network structure is regarded as a node. The feature vector of each node includes the road type, section length, number of lanes, maximum speed limit, in-degree and out-degree of the section corresponding to the current node.

[0017] Furthermore, the working process of the first encoder is:

[0018] The GPS trajectory data is used as the input of the convolution module, and the feature vector of each node is used as the input of the graph attention network;

[0019] The output of the convolution module is added to the output of the graph attention network, and the addition result is used as the input of the encoding unit, and the output of the encoding unit is used as the output of the first encoder.

[0020] Furthermore, the working process of the graph attention network is as follows:

[0021] Step 1: The characteristic vector of the i-th node obtained according to the road network structure is recorded as The eigenvector of the jth node is recorded as

[0022] Step 2: Initialize the number of layers l = 1;

[0023] Step 3: The k-th attention head calculates the edge weight between the i-th node and the j-th node

[0024]

[0025] in, and are the weight parameters of the k-th attention head in the l-th layer, express The transpose of

[0026] Step 4: The kth attention head in the lth layer calculates the attention weight between the i-th node and the j-th node

[0027]

[0028] Among them, N i represents the set of all nodes connected to the i-th node in the road network structure, represents the edge weight between the i-th node and the p-th node calculated by the k-th attention head in the l-th layer;

[0029] Step 5: The updated feature of the i-th node is:

[0030]

[0031] Among them, H1 represents the number of heads of multi-head attention, ELU(·) represents the activation function, Represents the concatenation of the activation function outputs of the H1 head attention, Represents the feature update result of the l-1th layer for the pth node, represents the weight parameter;

[0032] Step 6: Determine whether l=L, where L is the total number of layers in the graph attention network.

[0033] If l=L, then the feature of the i-th node This is the output of the graph attention network;

[0034] If l=L is not satisfied, set l=l+1 and return to step 3.

[0035] Furthermore, the working process of the convolution module is:

[0036] Get the timestamp X corresponding to the i-th node according to the GPS trajectory data i , and timestamp X i Transformed into an embedding vector X through the embedding layer t :

[0037] X t =Embedding(X i )

[0038] Among them, Embedding(·) represents the embedding layer;

[0039] Then the output Z of the convolution module is i for:

[0040]

[0041] Among them, X t is the vector representation of the timestamp, w s is the wavelet convolution kernel of the sth scale, Conv1D(·) represents the output of the convolution layer, ReLU(·) represents the activation function, and W s is the mapping matrix.

[0042] Furthermore, the output of the convolutional module is added to the output of the graph attention network, specifically:

[0043]

[0044] in, Indicates the addition result corresponding to the i-th node;

[0045] According to the order of each node in the GPS trajectory data, the corresponding addition results of each node are Connect and get the connection result

[0046] The encoding unit includes N encoding modules, which will connect the results As the input of the encoding unit, the working process of the encoding unit is:

[0047] Will As the input of the first encoding module, in the first encoding module, input First, after multi-head self-attention, the output of the mth self-attention head is recorded as A m After concatenating the outputs of multiple self-attention heads, the concatenated result is mapped to X':

[0048]

[0049] in, Indicates the concatenation of the outputs of multiple self-attention heads, W o represents a learnable parameter;

[0050] Will and Perform residual connection and use the residual connection result as the input of the first normalization layer in the first encoding module;

[0051] The output of the first normalization layer is used as the input of the fully connected layer in the first encoding module, the output of the fully connected layer is residually connected to the output of the first normalization layer, and the residual connection result is used as the input of the second normalization layer in the first encoding module;

[0052] The output of the second normalization layer is used as the output of the first encoding module;

[0053] Then the output of the first encoding module is used as the input of the second encoding module. Similarly, the output of the N-1th encoding module is used as the input of the Nth encoding module, and the output of the Nth encoding module is used as the output of the encoding unit.

[0054] Furthermore, the training process of the online network is:

[0055] Step S1: Obtain a GPS trajectory data set and a road network structure of the area covered by each GPS trajectory data in the data set;

[0056] And divide each track data in the obtained GPS track data set into Q batches;

[0057] Step S2, initialize the batch number q=1;

[0058] Step S3: Randomly mask the trajectory data of the qth batch, and use the road network structure corresponding to the trajectory data of the qth batch and the randomly masked trajectory data as inputs to the first encoder in the online network;

[0059] The output of the first encoder is used as the input of the first projection head in the online network, and the output of the first projection head is used as the input of the prediction head in the online network;

[0060] Perform a random transformation on the trajectory data of the qth batch, and use the road network structure corresponding to the trajectory data of the qth batch and the randomly transformed trajectory data as the input of the second encoder in the target network;

[0061] Pass the output of the second encoder as the input of the second projection head in the target network;

[0062] Step S4: Calculate the mask loss based on the output of the first encoder Compute the contrastive loss L based on the output of the second projection head in the target network and the output of the prediction head in the online network. MSE ;

[0063] Then according to the mask loss and contrast loss L MSE Calculate the total loss L pre :

[0064]

[0065] Among them, λ and β are hyperparameters;

[0066] Step S5: According to the total loss L pre Back propagation to update the parameters θ of the online network:

[0067]

[0068] in, Indicates L pre The gradient of , η represents the learning rate;

[0069] Momentum updates the parameters of the target model:

[0070] ξ←τ·ξ+(1-τ)·θ

[0071] Among them, ξ represents the parameters of the target model, τ represents the momentum coefficient;

[0072] Step S6: determine whether q=Q is satisfied;

[0073] If q=Q, the training ends and the trained online network is obtained;

[0074] If q=Q is not satisfied, set q=q+1 and return to step S3.

[0075] Furthermore, the contrast loss L MSE for:

[0076]

[0077] Among them, P online,k' Z represents the output of the prediction head in the online network after the k'th trajectory data in the qth batch is input into the online network; target,k' represents the output of the second projection head in the target network after the k'th trajectory data in the qth batch is input into the target network; ‖·‖ represents the calculation of the 2-norm; |x q | represents the total number of trajectory data in the qth batch.

[0078] Furthermore, the first projection head includes a linear layer, a LayerNormalization layer, a ReLU layer and a linear layer in sequence.

[0079] Furthermore, the prediction head includes a linear layer, a LayerNormalization layer, a ReLU layer and a linear layer in sequence.

[0080] The beneficial effects of the present invention are:

[0081] The model of the present invention adopts a dual-branch architecture of an online network and a momentum target network, deeply integrates the optimization objectives of the contrast learning task and the mask task, and enhances the stability of the model through a parameter asynchronous update strategy. The model can simultaneously capture the global semantic relevance and local context features of the trajectory data, significantly improving the generalization ability of the features. The time periodic coding module based on wavelet transform adaptively extracts the time-varying periodic pattern of traffic flow through a multi-scale one-dimensional convolution kernel, and dynamically adjusts the time feature weights in combination with an embedding mechanism to solve the technical defect that traditional methods cannot flexibly adapt to changes in traffic patterns in different time periods. The present invention constructs a collaborative working mechanism between the graph attention network and the time coding module. The graph attention network can analyze the spatial constraint relationship of the road topology, and the time coding module can capture the temporal laws of the movement of mobile objects. It not only enhances the ability to capture time periodicity, but also dynamically embeds time information into the position code, thereby effectively responding to the challenges brought by trajectory data. The two realize the organic fusion of multimodal information through feature cascading and gradient sharing, and can fully explore the potential relationship between time series features and road network topology structure information.

[0082] Furthermore, the proposed method achieves comprehensive coverage of application scenarios, eliminating the need to design features specifically for each scenario, thereby improving the accuracy and robustness of trajectory data representation. Experimental results demonstrate that the proposed method demonstrates good scalability and effectiveness in downstream tasks across multiple datasets. Compared to existing state-of-the-art models, the proposed method achieves an average performance improvement of 6.8%, fully demonstrating its robustness and generalization capabilities in self-supervised trajectory representation learning. BRIEF DESCRIPTION OF THE DRAWINGS

[0083] Figure 1 is a flow chart of the trajectory data representation method of the present invention for downstream tasks;

[0084] Figure 2 It is a structural diagram of the trajectory representation learning model of the present invention;

[0085] Figure 3 It is a structural diagram of the prediction head (PredictionHead) and the projection head (ProjectionHead);

[0086] Figure 4 This is a structural diagram of the encoder (RepresentationEncoder). DETAILED DESCRIPTION

[0087] Specific implementation method 1: Combination Figure 2 This embodiment describes a trajectory data representation method based on deep self-supervised learning, which specifically includes the following steps:

[0088] Step 1: Obtain the GPS trajectory data of the vehicle and the road network structure of the area covered by the GPS trajectory data, and then obtain the feature vector of each node in the road network based on the road network structure;

[0089] Step 2: Use the GPS trajectory data and the feature vectors of each node as input to the online network, and output the trajectory representation of the GPS trajectory data through the online network;

[0090] The working process of the online network is:

[0091] Step 21: Using GPS trajectory data and feature vectors of each node vector as inputs of the first encoder;

[0092] Step 22: using the output of the first encoder as the input of the first projection head;

[0093] Step 2 and step 3: Use the output of the first projection head as the input of the prediction head, and use the output of the prediction head as the trajectory representation of the GPS trajectory data.

[0094] To ensure temporal accuracy, this method converts trajectory timestamps to the local time zone before inputting them into the network. This ensures that trajectory patterns align with actual urban traffic flows, enabling meaningful comparative analysis. Road network structure data and GPS data are used as input. The road network structure data captures spatial constraints and road topology information, while the GPS data provides highly accurate spatiotemporal trajectory point information. By fusing multi-source data, the method can more comprehensively reflect the actual motion characteristics of mobile objects, providing more accurate and robust trajectory representations for downstream tasks.

[0095] Specific embodiment 2: This embodiment differs from specific embodiment 1 in that the characteristic vector of each node in the road network is obtained according to the road network structure. The specific process is as follows:

[0096] Each road section in the road network structure (i.e., the road section between two adjacent intersections) is regarded as a node. The feature vector of each node includes the road type, section length, number of lanes, maximum speed limit, in-degree and out-degree of the section corresponding to the current node.

[0097] Other steps and parameters are the same as those in the first embodiment.

[0098] The present invention assigns different type labels to different road types. The type labels are digital labels. Therefore, the feature vector of the node corresponding to the road section can be composed of the road type label, section length, number of lanes in the section, maximum speed limit of the section, in-degree and out-degree of the section.

[0099] Specific implementation method three: Combination Figure 4 The difference between this embodiment and the first or second embodiment is that the working process of the first encoder is as follows:

[0100] The GPS trajectory data is used as the input of the convolution module, and the feature vector of each node is used as the input of the graph attention network;

[0101] The output of the convolution module is added to the output of the graph attention network, and the addition result is used as the input of the encoding unit, and the output of the encoding unit is used as the output of the first encoder.

[0102] Other steps and parameters are the same as those in the first or second embodiment.

[0103] Specific embodiment 4: This embodiment differs from any one of specific embodiments 1 to 3 in that the working process of the graph attention network is as follows:

[0104] Step 1: The characteristic vector of the i-th node obtained according to the road network structure is recorded as The eigenvector of the jth node is recorded as

[0105] Step 2: Initialize the number of layers l = 1;

[0106] Step 3: The k-th attention head calculates the edge weight between the i-th node and the j-th node

[0107]

[0108] in, and are all learnable weight parameters of the k-th attention head in the l-th layer, express The transpose of

[0109] Step 4: The kth attention head in the lth layer calculates the attention weight between the i-th node and the j-th node

[0110]

[0111] Among them, N i represents the set of all nodes connected to the i-th node in the road network structure (in the road network structure, if two road sections have a common intersection, then there is an edge between the two nodes corresponding to the two sections, that is, the two nodes are connected). represents the edge weight between the i-th node and the p-th node calculated by the k-th attention head in the l-th layer;

[0112] Step 5: The updated feature of the i-th node is:

[0113]

[0114] Among them, H1 represents the number of heads of multi-head attention, ELU(·) represents the activation function, Represents the concatenation of the activation function outputs of the H1 head attention, Represents the feature update result of the l-1th layer for the pth node, represents the learnable weight parameters,

[0115] Step 6: Determine whether l=L, where L is the total number of layers in the graph attention network.

[0116] If l=L, then the feature of the i-th node This is the output of the graph attention network;

[0117] If l=L is not satisfied, set l=l+1 and return to step 3.

[0118] The other steps and parameters are the same as those in the first to third embodiments.

[0119] The present invention can capture the complex spatial dependencies between trajectory points through a multi-head attention mechanism.

[0120] Specific embodiment 5: This embodiment differs from any one of specific embodiments 1 to 4 in that the working process of the convolution module is as follows:

[0121] Get the timestamp X corresponding to the i-th node according to the GPS trajectory data i , that is, there is a road section corresponding to the i-th node in the GPS trajectory data (including the time point and the position data corresponding to the time point), and the time series corresponding to this road section in the GPS trajectory data is obtained. This time series is the timestamp X i , and timestamp X i Transformed into an embedding vector X through the embedding layer t :

[0122] X t =Embedding(X i )

[0123] Among them, Embedding(·) represents the embedding layer;

[0124] Then the output Z of the convolution module is i for:

[0125]

[0126] Among them, X t is the vector representation of the timestamp, w s is the wavelet convolution kernel of the sth scale (convolution kernels of different scales are used to capture feature patterns in different time ranges), Conv1D(·) represents the output of the convolution layer, ReLU(·) represents the activation function, and W s is a learnable mapping matrix.

[0127] The other steps and parameters are the same as those in the first to fourth embodiments.

[0128] Specific implementation method six: combination Figure 4 This embodiment differs from any one of the first to fifth embodiments in that the output of the convolution module is added to the output of the graph attention network, specifically:

[0129]

[0130] in, Indicates the addition result corresponding to the i-th node, the addition result Realize the joint representation of temporal and spatial features;

[0131] According to the order of each node in the GPS trajectory data, the corresponding addition results of each node are Connect and get the connection result

[0132] The coding unit includes N coding modules (in the present invention, the value of N is set to 3), and the connection result As the input of the encoding unit, the working process of the encoding unit is:

[0133] Will As the input of the first encoding module, in the first encoding module, input First, after multi-head self-attention (multi-head self-attention includes H2 parallel self-attention heads, As the input of the 1st, 2nd, ..., H2th self-attention head respectively), the output of the mth self-attention head is recorded as A m After concatenating the outputs of multiple self-attention heads, the concatenated result is mapped to X':

[0134]

[0135] in, Indicates the concatenation of the outputs of multiple self-attention heads, W o represents a learnable parameter;

[0136] X' and Perform residual connection and use the residual connection result as the input of the first normalization layer in the first encoding module;

[0137] The output of the first normalization layer is used as the input of the fully connected layer (feedforward layer) in the first encoding module. The output of the fully connected layer is residually connected with the output of the first normalization layer, and the residual connection result is used as the input of the second normalization layer in the first encoding module.

[0138] The output of the second normalization layer is used as the output of the first encoding module;

[0139] Then the output of the first encoding module is used as the input of the second encoding module. Similarly, the output of the N-1th encoding module is used as the input of the Nth encoding module, and the output of the Nth encoding module is used as the output of the encoding unit. The output of the encoding unit is the output of the encoder.

[0140] The other steps and parameters are the same as those in the first to fifth embodiments.

[0141] Through multiple encoding modules, the present invention can gradually extract the multi-scale and multimodal features of trajectories, improve the expressiveness of trajectory representation, and ultimately generate a highly expressive trajectory representation. The working process of the second to the Nth encoding modules is the same as that of the first encoding module. The encoding modules contain multi-head self-attention and feedforward neural networks, and introduce residual connections and normalization (Add & Norm) to ensure numerical stability and alleviate the gradient vanishing problem.

[0142] The working process of a single self-attention head is:

[0143] Will Mapped to query matrix Q, key matrix K and value matrix V respectively, the output of the self-attention head is:

[0144]

[0145] Specific implementation method seven: combination Figure 2 This embodiment differs from any one of the first to sixth embodiments in that the online network training process is as follows:

[0146] Step S1: Obtain a GPS trajectory data set and a road network structure of the area covered by each GPS trajectory data in the data set;

[0147] And divide each track data in the obtained GPS track data set into Q batches;

[0148] Step S2, initialize the batch number q=1;

[0149] Step S3: Randomly mask the trajectory data of the qth batch, and use the road network structure corresponding to the trajectory data of the qth batch and the randomly masked trajectory data as inputs to the first encoder in the online network;

[0150]

[0151] Among them, θ represents the parameters of the online network, Represents the trajectory data of the qth batch after random masking, R online Represents the output of the first encoder, RepresentationEncoder θ represents the first encoder;

[0152] The output of the first encoder is used as the input of the first projection head in the online network, and the output of the first projection head is used as the input of the prediction head in the online network;

[0153] Z online ←ProjectionHead θ (R online )

[0154] P online ←Predictor θ 4Z online )

[0155] Among them, Z online represents the output of the first projection layer, P online Represents the output of the prediction head, ProjectionHead θ represents the first projection layer, and Predictor represents the prediction head;

[0156] Perform a random transformation on the qth batch of trajectory data (i.e., randomly replace a small portion of one trajectory data with a portion of another trajectory data. In the present invention, the masking and replacement ratio is very small, for example, it can be set to 5%). The road network structure corresponding to the qth batch of trajectory data and the randomly transformed trajectory data are used as the input of the second encoder in the target network;

[0157]

[0158] Among them, ξ represents the parameters of the target network, R target Represents the output of the second encoder, RepresentationEncoder ξ represents the second encoder;

[0159] Pass the output of the second encoder as the input of the second projection head in the target network;

[0160] Z target ←ProjectionHead ξ (R target )

[0161] Among them, Z target Represents the output of the second projection layer, ProjectionHead ξ Represents the second projection layer;

[0162] Step S4: Calculate the mask loss based on the output of the first encoder Compute the contrastive loss L based on the output of the second projection head in the target network and the output of the prediction head in the online network. MSE ;

[0163] Then according to the mask loss and contrast loss L MSE Calculate the total loss L pre :

[0164]

[0165] Among them, λ and β are hyperparameters for balancing the two tasks, λ∈(0,1), β∈(0,1), and the specific values of λ and β can be set according to actual conditions;

[0166] The joint optimization strategy of this paper combines contrastive loss and mask loss. The contrastive loss is used to maximize the similarity of representations of the same trajectory across different views, achieving maximum consistency in enhanced view features, thereby learning a more discriminative global representation while distinguishing different trajectories. The mask loss is used to learn fine-grained contextual information about the trajectory, enhancing the model's ability to capture fine-grained local features. This design achieves the synergistic learning of local information and global features, accelerating model convergence and enhancing the generalization ability of trajectory representation learning. By integrating global information with local details, the model improves learning effectiveness.

[0167] Step S5: According to the total loss L pre Back propagation to update the parameters θ of the online network:

[0168]

[0169] in, Indicates L pre The gradient of , η represents the learning rate;

[0170] Momentum updates the parameters of the target model:

[0171] ξ←τ·ξ+(1-τ)·θ

[0172] Among them, ξ represents the parameters of the target model, τ represents the momentum coefficient;

[0173] Step S6: determine whether q=Q is satisfied;

[0174] If q=Q, the training ends and the trained online network is obtained;

[0175] If q=Q is not satisfied, set q=q+1 and return to step S3.

[0176] The other steps and parameters are the same as those in the first to sixth embodiments.

[0177] Specific embodiment eight: This embodiment differs from any one of specific embodiments one to seven in that the contrast loss L MSE for:

[0178]

[0179] Among them, P online,k' Z represents the output of the prediction head in the online network after the k'th trajectory data in the qth batch is input into the online network; target,k'represents the output of the second projection head in the target network after the k'th trajectory data in the qth batch is input into the target network; ‖·‖ represents the calculation of the 2-norm; |x q | represents the total number of trajectory data in the qth batch.

[0180] The other steps and parameters are the same as those in the first to seventh embodiments.

[0181] Specific implementation method nine: combination Figure 3 This embodiment is described below. The difference between this embodiment and any one of the specific embodiments 1 to 8 is that the first projection head includes a linear layer, a LayerNormalization layer, a ReLU layer, and a linear layer in sequence.

[0182] The other steps and parameters are the same as those in Specific Embodiments 1 to 8.

[0183] The projection head of the present invention maps input features to a low-dimensional space to adapt to the comparison learning task, and the structure of the second projection head is the same as that of the first projection head.

[0184] Specific implementation method ten: Combination Figure 3 This embodiment is described below. The difference between this embodiment and any one of the specific embodiments 1 to 9 is that the prediction head includes a linear layer, a LayerNormalization layer, a ReLU layer, and a linear layer in sequence.

[0185] The other steps and parameters are the same as those in Specific Embodiments 1 to 9.

[0186] The prediction head can further adjust the output features of the online network to align them with the representation of the target network. This design eliminates the reliance on negative samples, improving the efficiency of contrastive learning and the ability to represent trajectory features.

[0187] Experimental part

[0188] The trajectory representation learning method based on deep self-supervised learning proposed in this paper is called AttentionBERT Contrastive (ABC). Figure 2As shown, the method of the present invention efficiently captures the contextual semantics and global structural features of trajectory data by integrating contrastive learning and masking tasks, significantly improving the generalization ability of the model in downstream tasks. In the masking task, the present invention learns the local contextual information of the trajectory by randomly masking some labels in the trajectory sequence and predicting their true values, thereby helping the model to understand the semantic structure of the trajectory more deeply. In the contrastive learning task, the present invention adopts a dual network design including an online network and a target network. The online network predicts the representation of the target network through different enhanced views of the input trajectory, capturing the global structural similarity of the trajectory data. The parameters of the target network are generated by the momentum update strategy of the online network, which ensures the stability of the target network representation and effectively avoids divergence during training.

[0189] During pre-training, some trajectory segments are randomly masked and reconstructed by the model to learn the contextual representation of the trajectory. The mask setting is fixed to ensure that the model can capture the global pattern. Figure 1 As shown, the trajectory representation of the present invention can be adapted to a variety of downstream tasks, including trajectory travel time estimation and trajectory classification.

[0190] Trajectory travel time estimation: In the travel time estimation task, a single-layer fully connected layer regression model is used to map the trajectory representation to the predicted travel time, and the input representation p i Mapped to predicted travel time

[0191]

[0192] The optimization objective is the mean square error (MSE), which is defined as:

[0193]

[0194] Among them, y i is the actual travel time, is the predicted value, and N is the number of trajectories.

[0195] By optimizing the error between the predicted value and the true value, the estimation accuracy of the model is improved.

[0196] Trajectory classification: In trajectory classification tasks, trajectories are classified according to specific labels (such as whether they carry passengers, driver identity, etc.). The classification model consists of a single fully connected layer and a softmax activation function, and its predicted probability distribution is:

[0197]

[0198] The optimization objective of the classification task is cross entropy loss:

[0199]

[0200] Among them, y i (c) represents the true label of category c, represents the predicted category probability, and C is the number of categories.

[0201] Improve classification performance by optimizing the matching degree between classification probability distribution and true label.

[0202] On the taxi trajectory dataset, the performance of the model (ABC) of the present invention in the travel time estimation task is significantly improved compared with the latest model, showing higher prediction accuracy and error control ability. In the trajectory classification task, although the START model is slightly better in some indicators, the model (ABC) of the present invention still shows strong competitiveness. On the taxi trajectory dataset of City B, the model (ABC) of the present invention is comprehensively ahead in travel time estimation and multi-classification tasks. The present invention does not need to pre-calculate trajectory transition probabilities, which can reduce dependence on prior knowledge and reduce computational complexity, while avoiding complex pre-processing operations and adapting to dynamic environments and diverse datasets. The model can balance local and global feature learning by combining mask tasks and contrastive learning tasks. The time periodic coding method based on multi-scale wavelet transform can effectively capture trajectory time characteristics and significantly improve travel time estimation and trajectory classification performance.

[0203] Table 1 Performance comparison of various baselines on the City A taxi trajectory dataset

[0204]

[0205]

[0206] Table 2 Performance comparison of various baselines on the City B taxi trajectory dataset

[0207]

[0208]

[0209] On a dataset of taxi trajectories from City A, the optimal MAE for travel time estimation reached 7.211 (a 21.05% improvement over the optimal baseline, START), and the RMSE was reduced to 22.04 (a 37.74% improvement in error control capability). On a multi-classification task on a dataset of taxi trajectories from City B, the Micro-F1 score reached 0.095 (surpassing the state-of-the-art model by 6.74%), and the Recall@5 score exceeded 0.250 (a 2.46% improvement in coverage accuracy).

[0210] The above examples are merely illustrative of the calculation model and process of the present invention and are not intended to limit the embodiments of the present invention. Persons skilled in the art will readily appreciate that other variations or modifications based on the above description are possible. This list of embodiments is not exhaustive; however, any obvious variations or modifications derived from the technical solution of the present invention remain within the scope of protection of the present invention.

Claims

1. A trajectory data representation method based on deep self-supervised learning, characterized in that: The method specifically comprises the following steps: Step 1: Obtain the GPS trajectory data of the vehicle and the road network structure of the area covered by the GPS trajectory data, and then obtain the feature vector of each node in the road network based on the road network structure; Step 2: Use the GPS trajectory data and the feature vectors of each node as input to the online network, and output the trajectory representation of the GPS trajectory data through the online network; The working process of the online network is: Step 21: Using GPS trajectory data and feature vectors of each node vector as inputs of the first encoder; Step 22: using the output of the first encoder as the input of the first projection head; Step 2 and step 3: Use the output of the first projection head as the input of the prediction head, and use the output of the prediction head as the trajectory representation of the GPS trajectory data.

2. A trajectory data representation method based on deep self-supervised learning according to claim 1, characterized in that: The specific process of obtaining the characteristic vector of each node in the road network according to the road network structure is as follows: Each road section in the road network structure is regarded as a node. The feature vector of each node includes the road type, section length, number of lanes, maximum speed limit, in-degree and out-degree of the section corresponding to the current node.

3. The trajectory data representation method based on deep self-supervised learning according to claim 1, characterized in that: The working process of the first encoder is: The GPS trajectory data is used as the input of the convolution module, and the feature vector of each node is used as the input of the graph attention network; The output of the convolution module is added to the output of the graph attention network, and the addition result is used as the input of the encoding unit, and the output of the encoding unit is used as the output of the first encoder.

4. The trajectory data representation method based on deep self-supervised learning according to claim 3 is characterized in that: The working process of the graph attention network is as follows: Step 1: The characteristic vector of the i-th node obtained according to the road network structure is recorded as The eigenvector of the jth node is recorded as Step 2: Initialize the number of layers l = 1; Step 3: The k-th attention head calculates the edge weight between the i-th node and the j-th node in, and are the weight parameters of the k-th attention head in the l-th layer, express The transpose of Step 4: The kth attention head in the lth layer calculates the attention weight between the i-th node and the j-th node Among them, N i represents the set of all nodes connected to the i-th node in the road network structure, represents the edge weight between the i-th node and the p-th node calculated by the k-th attention head in the l-th layer; Step 5: The updated feature of the i-th node is: Among them, H1 represents the number of heads of multi-head attention, ELU(·) represents the activation function, Represents the concatenation of the activation function outputs of the H1 head attention, Represents the feature update result of the l-1th layer for the pth node, represents the weight parameter; Step 6: Determine whether l=L, where L is the total number of layers in the graph attention network. If l=L, then the feature of the i-th node This is the output of the graph attention network; If l=L is not satisfied, set l=l+1 and return to step 3.

5. The trajectory data representation method based on deep self-supervised learning according to claim 4 is characterized in that: The working process of the convolution module is as follows: Get the timestamp X corresponding to the i-th node according to the GPS trajectory data i , and timestamp X i Transformed into an embedding vector X through the embedding layer t : X t =Embedding(X i ) Among them, Embedding(·) represents the embedding layer; Then the output Z of the convolution module is i for: Among them, X t is the vector representation of the timestamp, w s is the wavelet convolution kernel of the sth scale, Conv1D(·) represents the output of the convolution layer, ReLU(·) represents the activation function, and W s is the mapping matrix.

6. A trajectory data representation method based on deep self-supervised learning according to claim 5, characterized in that: The output of the convolutional module is added to the output of the graph attention network, specifically: in, Indicates the addition result corresponding to the i-th node; According to the order of each node in the GPS trajectory data, the corresponding addition results of each node are Connect and get the connection result The encoding unit includes N encoding modules, which will connect the results As the input of the encoding unit, the working process of the encoding unit is: Will As the input of the first encoding module, in the first encoding module, input First, after multi-head self-attention, the output of the mth self-attention head is recorded as A m After concatenating the outputs of multiple self-attention heads, the concatenated result is mapped to X': in, Indicates the concatenation of the outputs of multiple self-attention heads, W o represents a learnable parameter; X' and Perform residual connection and use the residual connection result as the input of the first normalization layer in the first encoding module; The output of the first normalization layer is used as the input of the fully connected layer in the first encoding module, the output of the fully connected layer is residually connected to the output of the first normalization layer, and the residual connection result is used as the input of the second normalization layer in the first encoding module; The output of the second normalization layer is used as the output of the first encoding module; Then the output of the first encoding module is used as the input of the second encoding module. Similarly, the output of the N-1th encoding module is used as the input of the Nth encoding module, and the output of the Nth encoding module is used as the output of the encoding unit.

7. The trajectory data representation method based on deep self-supervised learning according to claim 6, characterized in that: The training process of the online network is: Step S1: Obtain a GPS trajectory data set and a road network structure of the area covered by each GPS trajectory data in the data set; And divide each track data in the obtained GPS track data set into Q batches; Step S2, initialize the batch number q=1; Step S3: Randomly mask the trajectory data of the qth batch, and use the road network structure corresponding to the trajectory data of the qth batch and the randomly masked trajectory data as inputs to the first encoder in the online network; The output of the first encoder is used as the input of the first projection head in the online network, and the output of the first projection head is used as the input of the prediction head in the online network; Perform a random transformation on the trajectory data of the qth batch, and use the road network structure corresponding to the trajectory data of the qth batch and the randomly transformed trajectory data as the input of the second encoder in the target network; Pass the output of the second encoder as the input of the second projection head in the target network; Step S4: Calculate the mask loss based on the output of the first encoder Compute the contrastive loss L based on the output of the second projection head in the target network and the output of the prediction head in the online network. MSE ; Then according to the mask loss and contrast loss L MSE Calculate the total loss L pre : Among them, λ and β are hyperparameters; Step S5: According to the total loss L pre Back propagation to update the parameters θ of the online network: in, Indicates L pre The gradient of , η represents the learning rate; Momentum updates the parameters of the target model: ξ←τ·ξ+(1-τ)·θ Among them, ξ represents the parameters of the target model, τ represents the momentum coefficient; Step S6: determine whether q=Q is satisfied; If q=Q, the training ends and the trained online network is obtained; If q=Q is not satisfied, set q=q+1 and return to step S3.

8. The trajectory data representation method based on deep self-supervised learning according to claim 7, characterized in that: The contrast loss l MSE for: Among them, P online,k' Z represents the output of the prediction head in the online network after the k'th trajectory data in the qth batch is input into the online network; target,k' represents the output of the second projection head in the target network after the k'th trajectory data in the qth batch is input into the target network; ‖·‖ represents the calculation of the 2-norm; Indicates the total number of trajectory data in the qth batch.

9. The trajectory data representation method based on deep self-supervised learning according to claim 1, characterized in that: The first projection head includes a linear layer, a LayerNormalization layer, a ReLU layer and a linear layer in sequence.

10. The trajectory data representation method based on deep self-supervised learning according to claim 1, characterized in that: The prediction head includes a linear layer, a LayerNormalization layer, a ReLU layer and a linear layer in sequence.

Images