Sparse trajectory pattern learning method based on mutual learning semantic distillation

Abstract

The invention discloses a sparse trajectory pattern learning method based on mutual learning semantic distillation, and particularly relates to the field of trajectory analysis. Comprising the following steps: acquiring spatial characteristics of a road section to which a track belongs and spatial-temporal characteristics of each track point, and determining an embedding vector of each track point according to the spatial-temporal characteristics and spatial characteristics of the track points; constructing track features based on semantics and semantic features based on tracks; determining a track-based semantic vector according to the semantic-based track features, and determining a semantic enhanced track vector according to the track-based semantic features; fusing and interacting the semantic vector based on the track and the semantic enhanced track vector to obtain an embedded vector of the track; and inputting the embedding vector of the track, the initial occurrence time, the position of the interest point and the cue word into a pre-trained LLM text encoder to obtain a track mode. On the basis of the method, mode missing detection caused by data sparsity can be avoided, and then the accuracy of trajectory analysis is improved.

Description

Technical Field

[0001] The present application relates to the field of trajectory analysis, and in particular to a sparse trajectory pattern learning method based on mutual learning semantic distillation. Background Art

[0002] Spatiotemporal trajectory data analysis has become a key technology for understanding the motion patterns of objects across space and time, with profound implications for practical applications such as urban traffic optimization, human mobility analysis, and autonomous driving navigation. The ability to accurately extract features from trajectory data and perform diverse tasks is crucial for progress in these fields. In recent years, a significant amount of research has focused on trajectory representation learning, which can be roughly categorized into the following two categories. Traditional deep learning methods, such as autoencoders, variational autoencoders, and contrastive learning-based methods, as well as methods based on large language models (LLMs), have significantly improved trajectory representation learning.

[0003] Traditional methods mainly rely on self-supervised paradigms, such as sequence-to-sequence architectures and contrastive learning frameworks, which can effectively capture local trajectory patterns. However, these methods face significant challenges in obtaining comprehensive semantic information to represent diverse traffic scenarios. Recent advances in large language models (LLMs) have demonstrated the strong potential of cross-domain semantic learning. In the field of spatiotemporal trajectory mining, LLM-based methods have significantly improved trajectory analysis performance through semantic knowledge fusion. However, these models still have difficulties in adapting to sparse trajectory scenarios, which are often caused by low traffic volume (such as rural roads) or geographical restrictions (such as mountainous terrain). Dense areas naturally have rich trajectory patterns (such as straight driving, fast movement, and afternoon hours), while sparse areas have incomplete trajectory pattern extraction due to limited trajectory data, resulting in reduced accuracy of trajectory analysis. Summary of the Invention

[0004] The main purpose of this application is to provide a sparse trajectory pattern learning method based on mutual learning semantic distillation, aiming to solve the problem of low trajectory analysis accuracy in existing trajectory analysis methods.

[0005] To achieve the above objectives, the present application provides a sparse trajectory pattern learning method based on mutual learning semantic distillation, including: obtaining a trajectory, determining the starting time, trajectory points and interest point locations of the trajectory; obtaining the spatial features of the road section to which the trajectory belongs, and mapping the spatial features of the road section into shared parameters; determining the spatial feature embedding of each trajectory point based on the shared parameters; obtaining the spatiotemporal features of each trajectory point, and determining the embedding vector of each trajectory point based on the spatiotemporal features and spatial feature embedding of the trajectory point; constructing semantic-based trajectory features and trajectory-based semantic features based on the embedding vector of the trajectory point; mapping the semantic-based trajectory features to the semantic space to obtain a trajectory-based semantic vector, and mapping the trajectory-based semantic features to the trajectory space to obtain a semantically enhanced trajectory vector; fusing the trajectory-based semantic vector and the semantically enhanced trajectory vector, and interacting the fused vector with the embedding vector of each trajectory point to obtain the embedding vector of the trajectory; inputting the embedding vector, starting time and interest point location of the trajectory, and the prompt word into a pre-trained LLM text encoder to obtain a trajectory pattern.

[0006] Optionally, the spatial features of the road section are mapped to shared parameters; based on the shared parameters, the spatial feature embedding of each trajectory point is determined, including: converting the spatial features of the road section into spatial feature embedding; mapping the spatial feature embedding of the road section to shared parameters; using a Gaussian distribution model in combination with the shared parameters to determine the spatial feature probability distribution of the road section; and determining the spatial feature embedding of each trajectory point based on the spatial feature probability distribution of the road section.

[0007] Optionally, the embedding vector of each trajectory point is determined based on the spatiotemporal features and spatial feature embeddings of the trajectory point, including: converting the spatiotemporal features of the trajectory point into the spatiotemporal feature embeddings of the trajectory point; wherein the spatiotemporal features include the position, velocity, acceleration and direction of the trajectory point; and determining the embedding vector of each trajectory point based on the sum of the spatiotemporal feature embeddings and the spatial feature embeddings of the trajectory point.

[0008] Optionally, a semantic-based trajectory feature is constructed based on the embedding vector of the trajectory point, including: using the embedding vectors of all trajectory points to establish a trajectory feature set, and randomly generating a semantic vector set; determining the attention weight of each semantic vector in the semantic vector set to the embedding vectors of all trajectory points through a learnable projection matrix, and using it as the first attention weight; determining the semantic-based trajectory feature corresponding to each semantic vector based on each first attention weight and the embedding vector of the corresponding trajectory point.

[0009] Optionally, constructing trajectory-based semantic features based on the embedding vectors of the trajectory points, including: using the embedding vectors of all trajectory points to establish a trajectory feature set, and randomly generating a semantic vector set; using a learnable projection matrix to determine the attention weight of the embedding vector of each trajectory point and all semantic vectors in the trajectory feature set, and using it as the second attention weight; determining the trajectory-based semantic features based on each second attention weight and the corresponding semantic vector.

[0010] Optionally, the trajectory-based semantic vector and the semantically enhanced trajectory vector are fused, and the fused vector is interacted with the embedding vector of each trajectory point to obtain the embedding vector of the trajectory, including: element-by-element multiplication of the trajectory-based semantic representation and the semantically enhanced trajectory feature to obtain a fused vector; the first fused vector is interacted with the embedding vector of each trajectory point through product projection to obtain a transformed embedding vector of each trajectory point; the transformed embedding vectors of all trajectory points are residually connected with the embedding vectors of all trajectory points, and are processed through a feedforward neural network to generate the embedding vector of the trajectory.

[0011] Optionally, the spatial characteristics of the road segment include road type, number of lanes, one-way street signs, spatial smoothness, and road shape.

[0012] Optionally, the spatial features of the road section are converted into spatial feature embeddings, including: using an index retrieval embedding module to convert the road type, number of lanes, and one-way street sign into road type feature embeddings, lane number feature embeddings, and one-way street sign feature embeddings, respectively; obtaining a set of road vertices, using the road vertices as nodes, and using the nodes and the road sections connected to each node to establish a road network graph; performing multiple random walks on each node in the road network to generate a node sequence; using a Word2Vec model to convert the node sequence into an embedding vector of the road section, and using the embedding vector of the road section as a spatial smoothness and road shape feature embedding; splicing the road type feature embedding, lane number feature embedding, one-way street sign feature embedding, and spatial smoothness and road shape feature embedding to obtain a spatial feature embedding.

[0013] Optionally, converting the spatiotemporal features of the trajectory points into spatiotemporal feature embeddings of the trajectory points includes: performing convolution on the spatiotemporal features of each trajectory point to obtain the spatiotemporal feature embeddings of the trajectory points.

[0014] Optionally, the method for determining the pre-trained LLM text encoder is: adding a low-rank matrix to the parameters of the attention layer of the LLM text encoder, and freezing the parameters except the attention layer during training to obtain the pre-trained LLM text encoder.

[0015] Compared with the prior art, the present invention has the following advantages: The sparse trajectory pattern learning method based on mutual learning semantic distillation of the present invention uses the trajectories of road sections in sparse areas and road sections in dense areas to obtain a multi-layer perceptron, so that the traffic pattern learned in the dense area is implicitly encoded into the multi-layer perceptron through shared layer parameters. During learning, the spatial feature embedding of the road section is mapped to the shared parameters through the multi-layer perceptron, and the spatial feature embedding of the trajectory point is determined according to the shared parameters, so that the spatial feature embedding of the trajectory point contains the common pattern migrated from the dense area; the mutual semantic distillation method is adopted through the bidirectional attention mechanism and the knowledge transfer strategy, so that the semantic pattern learned in the dense area can be migrated to the sparse area, while retaining the local features of the sparse area, which can avoid the pattern omission caused by data sparsity, thereby improving the accuracy of trajectory analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 This is a flowchart of the sparse trajectory pattern learning method based on mutual learning semantic distillation in this application; Figure 2 This is the application flowchart of the sparse trajectory pattern learning method based on mutual learning semantic distillation in this application; Figure 3 This is the knowledge transfer scenario diagram of the sparse trajectory pattern learning method based on mutual learning semantic distillation in this application; Figure 4 This is the structural diagram of the LLM text encoder in the sparse trajectory pattern learning method based on mutual learning semantic distillation in this application; Figure 5 This is a graph showing the impact of key hyperparameters on the sparse trajectory pattern learning method in Example 1.

[0017] The realization of the objectives, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0018] To make the objectives, technical solutions, and advantages of this application more clear, the technical solutions of this application will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments of this application, all other embodiments obtained by ordinary technicians in this field without making any creative efforts are within the scope of protection of this application.

[0019] The present invention provides a sparse trajectory pattern learning method based on mutual learning semantic distillation, such as Figure 1-2 As shown, the specific steps include: Step S10: Acquire the trajectory, determine the start time of the trajectory, the trajectory points, and the location of the points of interest; obtain the spatial features of the road section to which the trajectory belongs, and map the spatial features of the road section into shared parameters; determine the spatial feature embedding of each trajectory point based on the shared parameters; Specifically, step S11, obtain the trajectory, the trajectory It includes multiple trajectory points, which are a time-ordered sequence of sampling points collected, recorded as: , in is the trajectory length. Each sampling point is represented as a tuple ,in: The geographical coordinates (longitude and latitude) of the point; is the sampling timestamp. Determine the starting time of the trajectory and record it as the time of the first trajectory point t 1. A point of interest (POI) is a geographical location that has a specific function or attracts individual attention, such as an airport, shopping mall, tourist attraction, or hospital. POIs are formally represented as a triple: ,in is the geographic coordinate, For the name, For the detailed address.

[0020] Step S12, obtaining the spatial features of the road section to which the trajectory belongs, and converting the spatial features of the road section into spatial feature embedding; Specifically, in step S121 , the spatial characteristics of the road section to which the trajectory belongs are obtained. The spatial characteristics of the road section include road type, number of lanes, one-way street signs, spatial smoothness (sharp turn or gentle slope), and road shape.

[0021] Step S122: Using the index retrieval embedding module, the road type, number of lanes, and one-way street sign are converted into a road type feature embedding, a lane number feature embedding, and a one-way street sign feature embedding, respectively. Furthermore, in step S1221, the road type feature is input into the first index retrieval embedding module obtained in advance , output road segment Road type feature embedding ,in, is the number of road type values, Represents the embedding dimension of the feature; the number of lanes is input into the pre-trained second index retrieval embedding module , output road segment Lane number feature embedding ,in, is the number of lane values, d （b）Represents the embedding dimension of the feature; the one-way street sign feature is input into the pre-trained third index retrieval embedding module , output road segment One-way street sign feature embedding ,in, is the number of values of one-way street signs, d （c） Represents the embedding dimension of the feature; Step S1222: Use the DeepWalk algorithm to obtain spatial smoothness and road shape feature embedding. DeepWalk is a network representation learning method based on random walks that can simultaneously capture local and global structural information of network nodes. Through the DeepWalk algorithm, each road segment is given a low-dimensional embedding representation that reflects the spatial proximity and connectivity between road segments. Specifically, a set of road vertices is obtained, and the road vertices are used as nodes. The road network graph is constructed using the nodes and the road segments connected to each node. ,in represents a set of road vertices (such as intersections), Represent the road segments connecting these vertices; perform multiple random walks on each node in the road network to generate a node sequence; use the Word2Vec model to convert the node sequence into a road segment Embedding vector of , the embedding vector of the road segment is used as the spatial smoothness and road shape features; Step S123: The road type feature embedding, lane number feature embedding, one-way street sign feature embedding, spatial smoothness and road shape feature embedding are spliced to obtain the road segment Spatial feature embedding .

[0022] Step S13, embedding and mapping the spatial features of the road section into shared parameters; In this embodiment, considering the data sparsity, that is, the trajectory data is unevenly distributed on different road sections, the data in urban centers and other areas are rich, while the data in remote areas is scarce, a statistical sharing method is used to overcome the data sparsity problem. , maps the space of road segments to a set of shared parameters, thereby sharing statistical information between different road segments. This allows the knowledge learned from frequently visited road segments to be applied to road segments with sparse data. Specifically, the spatial feature embedding is input into two multi-layer perceptrons (MLPs) with shared input and hidden layers to obtain shared parameters. and It is worth noting that the MLP is pre-trained through trajectories of road sections in sparse areas and road sections in dense areas. The MLP adopts a knowledge transfer mechanism of shared parameters and a shared input layer and hidden layer to force road sections in different areas (dense / sparse) to share the same parameter space during the feature extraction stage. This architectural constraint allows the traffic patterns learned in dense areas (such as turning probabilities and temporal associations) to be implicitly encoded into the model through shared layer parameters, and the road sections in sparse areas automatically inherit the common knowledge in these shared parameters during mapping. During the training process, a hierarchical parameter update mechanism is adopted, and the shared layer parameters are updated jointly by all regional data. At the same time, the parameters of specific output layers are updated by regional groups to retain regional characteristics.

[0023] Step S14, using the Gaussian distribution model and the shared parameters, determine the spatial feature probability distribution of the road segment; based on the spatial feature probability distribution of the road segment, the spatial feature embedding of each trajectory point is determined by sampling from the distribution through the reparameterization technique. The calculation formula is as follows:

[0024] Where, Represents the spatial feature embedding of each trajectory point.

[0025] Step S20, obtaining the spatiotemporal features of each trajectory point, and determining the embedding vector of each trajectory point based on the spatiotemporal features and spatial features of the trajectory point; Specifically, in step S21, the spatiotemporal features of the trajectory points are converted into spatiotemporal feature embeddings of the trajectory points; wherein the spatiotemporal features are multiple continuous features, which may include the position, speed, acceleration and direction of the trajectory points, and the spatiotemporal features are expressed as ; Step S22, convolve the spatiotemporal features of each trajectory point to obtain the spatiotemporal feature embedding of the trajectory point; specifically, a one-dimensional convolution operation is performed on the spatiotemporal features of each trajectory point to obtain the spatiotemporal feature embedding ; Step S23: Determine the embedding vector of each trajectory point based on the sum of the spatiotemporal feature embedding and the spatial feature embedding of the trajectory point. ; .

[0026] To convert the embedding vectors of the trajectory points obtained above into semantic vectors, this embodiment employs a mutual semantic distillation method for trajectory learning. Through a bidirectional attention mechanism and a knowledge transfer strategy, semantic patterns learned in dense regions (such as "smooth right turn" and "speed changes during morning and evening rush hours") can be transferred to sparse regions while preserving the local features of the sparse regions. The mutual semantic distillation module comprises two complementary attention subnetworks: the trajectory → semantic attention subnetwork and the semantic → trajectory attention subnetwork. These two subnetworks work together to locate and extract motion patterns from trajectory data, enabling the learning of semantic-based trajectory features and trajectory-based semantic representations.

[0027] Step S30: construct semantic-based trajectory features and trajectory-based semantic features based on the embedding vectors of the trajectory points; map the semantic-based trajectory features to the semantic space to obtain a trajectory-based semantic vector; map the trajectory-based semantic features to the trajectory space to obtain a semantically enhanced trajectory vector; fuse the trajectory-based semantic vector and the semantically enhanced trajectory vector, and interact the fused vector with the embedding vector of each trajectory point to obtain the trajectory embedding vector.

[0028] Specifically, first, a trajectory→semantic attention subnet is constructed, and the working principle of this attention subnet is as shown in step S31; then a semantic→trajectory attention subnet is constructed, and the working principle of this attention subnet is as shown in step S32; finally, the complementary semantic information is captured and integrated by adaptively fusing the perspectives of the two attention subnets, see S33.

[0029] Step S31: Use the embedding vectors of all trajectory points to create a trajectory feature set , and randomly generate a set of semantic vectors ,in, is a hyperparameter that defines the number of virtual semantics, the semantic vector As a learnable pattern detector, it realizes semantic sharing through learnable semantic vectors As a universal pattern detector across regions, it captures traffic semantics that are independent of the region. The attention weight of each semantic vector in the set of semantic vectors to the embedding vectors of all trajectory points is determined through a learnable projection matrix and used as the first attention weight. The calculation formula for the first attention weight is as follows:

[0030] Where, is the learnable projection matrix, For the Semantic pair The attention weight of the trajectory point, that is, the first attention weight, reflects the The trajectory point and The degree of semantic matching (i.e. confidence). Figure 3 As shown, for sparse trajectory points , if its local features (such as speed fluctuation) are consistent with a semantic vector (learning from dense regions) matching, then The confidence level is 0.9, which is significantly increased according to the above formula. The green trajectory in the sparse area has a semantic confidence level of 0.9, which conforms to the “smooth right turn in the morning” pattern. The trajectory points can be associated with the semantic categories defined in the dense area to achieve implicit knowledge transfer and avoid missed detection of patterns due to data sparsity.

[0031] According to each first attention weight and the embedding vector of the corresponding trajectory point, the semantic-based trajectory feature corresponding to each semantic vector is determined; specifically, all the first attention weights constitute the first attention weight set , semantic-based trajectory features can be extracted for each semantic, then semantically related trajectory features The expression is:

[0032] in, Capture the semantic features that best represent the motion pattern of each trajectory point. (Such as a sharp turn), it will give the The semantics are assigned a higher positive confidence score. Thus, the semantic-based trajectory feature set corresponding to each trajectory point is obtained. .

[0033] Through the mapping function Map each semantic-based trajectory feature to the semantic space to obtain a trajectory-based semantic representation , the formula is as follows:

[0034] in, is the weight matrix.

[0035] Step S32: Determine the trajectory feature set using the learnable projection matrix In , the attention weight of the embedding vector of each trajectory point and all semantic vectors is used as the second attention weight;

[0036] Where, is the learnable projection matrix, For the The trajectory point and The attention weight of each semantic, i.e. the second attention weight, indicates the importance of each semantic for a given trajectory point; All the second attention weights constitute the second attention weight set , the trajectory-based semantic features can be extracted for each trajectory point, and the trajectory-based semantic features can be determined according to each second attention weight and the corresponding semantic vector , the calculation formula is as follows:

[0037] is the trajectory eigenvector The aligned trajectory semantic representation, therefore, utilizes the mapping function Based on the semantic features of the trajectory Mapping to the trajectory space, the sparse trajectory points are supplemented with semantic descriptions migrated from dense areas (such as "evening rush hour congestion pattern") to make up for the lack of local features and obtain the semantically enhanced trajectory vector, which is expressed as follows;

[0038] Where, is the weight matrix, output Indicates the first The trajectory embedding of each trajectory point is the semantically enhanced trajectory vector.

[0039] Due to the trajectory-based semantic representation and trajectory sequence length Unaligned, so it is necessary to align before fusion. The dimension is expanded to correspond to the trajectory length Dimension, this embodiment expands by averaging Obtain a sequence-level representation that reflects global features. This process dynamically assigns the first The confidence scores of the semantic categories.

[0040] Step S33: Fusing the trajectory-based semantic vector and the semantically enhanced trajectory vector, and interacting the fused vector with the embedding vector of each trajectory point to obtain the trajectory embedding vector; Specifically, the trajectory-based semantic representation and the semantically enhanced trajectory features are multiplied element-by-element to obtain a fusion vector; the first fusion vector is interacted with the embedding vector of each trajectory point through product projection to obtain a transformed embedding vector of each trajectory point; the transformed embedding vectors of all trajectory points are residually connected with the embedding vectors of all trajectory points, and processed through a two-layer feedforward neural network to generate the embedding vector of the trajectory.

[0041] In this embodiment, the fusion vector is used to construct a trajectory semantic space to coordinate the outputs of the two sub-networks. The transformed embedding vector of the trajectory point is obtained by mapping the embedding vector of the original trajectory point to the trajectory semantic space. The final embedding vector of the trajectory output through the residual connection and feedforward neural network retains the spatiotemporal semantic features and pattern information of the trajectory.

[0042] In step S40, the trajectory embedding vector, the start time, the location of the point of interest, and the prompt word are input into a pre-trained LLM text encoder to obtain a trajectory pattern. The pre-trained LLM text encoder is determined by adding a low-rank matrix to the parameters of the attention layer of the LLM text encoder and freezing the parameters except for the attention layer during training to obtain the pre-trained LLM text encoder.

[0043] Specifically, in the embedding layer of the pre-trained LLM text encoder, the prompt word is tokenized into an embedding sequence ,in represents the sequence length, is the embedding dimension. The prompt word structure follows:

[0044] In the attention layer, the parameters of the attention layer are injected with a trainable low-rank matrix, taking query projection as an example:

[0045] in is the rank hyperparameter. Key projection and value projection adopt the same paradigm and use independent low-rank matrices and . And freeze the parameters except the attention layer during training, and only train the query projection, key projection and value projection.

[0046] The encoding process outputs the hidden state , whose updates are constrained to a small subset of model parameters. For downstream tasks, the final hidden state is used Perform prediction head computations while maintaining parameter efficiency by freezing the backbone network and task-specific adapters.

[0047] In this embodiment, a pre-trained LLM text encoder is obtained through low-rank adaptation (LoRA). This lightweight method can achieve trajectory-specific adaptation with minimal computational overhead while retaining pre-training knowledge.

[0048] like Figure 4As shown in the figure, the LLM text encoder works as follows: the input first passes through the word embedding layer, converting it into a sequence of word-unit vectors. These word-unit vectors then enter the Transformer block. A multi-head self-attention mechanism calculates the attention score between each word-unit and the others. These scores are weighted and summed across the word-unit vectors to produce a representation that incorporates contextual information. The vectors processed by the multi-head self-attention mechanism then enter a feedforward neural network for nonlinear transformations to further extract features. Layer normalization is performed after each Transformer block. Multiple Transformer blocks are stacked sequentially to extract and abstract features layer by layer from the input text, ultimately outputting a vector representation rich in semantics and contextual information.

[0049] Example Experimental Setup Datasets All methods are evaluated on two trajectory datasets. The key statistical features of the datasets are shown in Table 1.

[0050] Table 1: Dataset statistics

[0051] Nine trajectory learning methods are selected for comparison as follows: • Traj2vec: A sliding window based feature extractor for sequence representation learning via an autoregressive pre-training task.

[0052] • T2vec: A denoising autoencoder architecture that reconstructs the original spatial distribution from downsampled trajectories.

[0053] • TremBR: An RNN-based sequence-to-sequence model that jointly recovers the segment and temporal properties of the input trajectory.

[0054] • CTLE: A bidirectional Transformer model pre-trained on the position and timestamp dual mask prediction task.

[0055] • Toast: A context-aware node embedding framework combining masked language modeling and sequence discrimination objectives.

[0056] • TrajCL: A bimodal self-attention encoder based on InfoNCE contrastive learning to achieve feature alignment across trajectory scales.

[0057] • START: A time-aware encoder integrating graph attention networks, optimized via mask prediction.

[0058] • LightPath: A sparse path coding architecture for path reconstruction and cross-view comparison tasks.

[0059] • TrajCogn: A framework based on a large language model that encodes spatiotemporal features through a semantic embedder and uses dynamic prompts to achieve multi-task adaptation.

[0060] (3) Settings This model is built on the GPT-2 architecture, with a hidden state dimension of 768. 、 、 and The embedding dimensions are set to 64, 32, 16, and 128. By employing Low-Rank Adaptation (LoRA), the model only fine-tunes less than one parameter during downstream adaptation, significantly reducing computational cost. The training process consists of two phases: first, 20 epochs of cross-modal reconstruction pre-training are performed to obtain a universal trajectory representation, followed by fine-tuning based on validation set performance (early stopping patience = 10) to prevent overfitting. The trajectory data is strictly split into training, validation, and test sets in an 8:1:1 ratio in chronological order to ensure temporal causality - the model is trained only on historical data and evaluated on future data segments.

[0061] This example uses the Adam optimizer, with an initial learning rate of , while the baseline method is 0.001. Key hyperparameters in Table 2 include LoRA rank, convolution kernel size, number of virtual attributes, and number of POIs. Experiments were conducted on a server with an AMD EPYC9754 CPU (18vCPUs), an NVIDIA RTX 4090D GPU (24GB VRAM), and 512GB DDR5 RAM, using PyTorch 2.3.0 and Python 3.12 with mixed precision acceleration.

[0062] Table 2: Key hyperparameter settings

[0063] (4) Performance indicators: The following indicators are used for evaluation: Travel time estimation: Three regression metrics were used – mean absolute error (MAE) for absolute deviation, root mean square error (RMSE) for error dispersion to indicate error dispersion, and mean absolute percentage error (MAPE) for relative error analysis.

[0064] Destination Prediction: The hit rate of top predictions is evaluated by Top-1 / 5 Accuracy (ACC@1 / 5), and supplemented with Macro-F1 score to balance the inter-class classification performance.

[0065] Similar trajectory search: Two metrics are used to evaluate this task, including average rank (the average position of correct matches) and ACC@1 / 5 (the hit rate in the top-N results).

[0066] 2. Effectiveness Evaluation Table 3 compares the performance of all models across three downstream tasks. This evaluation of the effectiveness of task-adaptive trajectory learning methods reveals that our method significantly outperforms existing methods across all tasks. For example, the average performance improvement across the three tasks reaches 66.57%. RNN-based models (Traj2vec, T2vec, and TremBR) underperform in the travel time estimation task, primarily due to their difficulty integrating spatiotemporal attributes and trip destination characteristics. Transformer-based methods (CTLE and Toast) underperform in the destination prediction task, demonstrating their limited ability to extract dynamic patterns of continuous motion. Contrastive learning models (TrajCL, START, and LightPath) exhibit low accuracy in the similar trajectory search task, reflecting their limited ability to capture trajectory functional semantics. While the state-of-the-art large language model augmentation method, TrajCogn, has achieved significant improvements, our model still maintains an advantage. For example, in the time estimation task on the Chengdu dataset, our model achieves an 8.17% lower RMSE than TrajCogn.

[0067] Table 3: Performance comparison of different methods in downstream tasks

[0068] 3. Ablation Experiment To evaluate the impact of the core components in the framework, we designed four model variants for ablation experiments to accurately analyze the contribution of each component to the overall performance of the model: w / o : Remove the spatiotemporal feature embedding component vector

[0069] w / o POI: Omit point of interest (POI) data from track prompts w / o Conv: Replace the convolution operator in spatiotemporal feature embedding with a simple fully connected layer w / o MSD: Completely remove the Mutual Semantic Distillation module Table 4: Ablation experiments

[0070] Table 4 shows the performance impact of each variant on Dataset 1. The main findings are as follows: Removing the spatiotemporal feature embedding component The results show that the time estimation performance is significantly reduced (RMSE increases by 16.0% from 106.9 to 123.9), confirming its key role in motion dynamic encoding; removing POI data significantly reduces the accuracy of similar trajectory search (ACC@1 decreases by 5.8%), highlighting its value for semantic understanding; replacing convolution operations with fully connected layers slightly reduces the destination prediction recall rate, verifying the effectiveness of the convolution structure; canceling the MSD module leads to a decrease in similar search accuracy (ACC@1 decreases by 1.7%), proving its role in promoting semantic learning.

[0071] Hyperparameter Learning We studied key hyperparameters 、 、 and Impact on model performance. Figure 5 As shown in Figure 1 (based on the ACC@1 and Recall metrics of the destination prediction task in Dataset 1), the main findings are as follows: like Figure 5 As shown in (a), the increase The value can increase the model capacity but may cause overfitting. This reduces the model complexity, resulting in insufficient fitting of the large language model to the trajectory data. After balancing considerations, we finally set To achieve the best model performance. Figure 5 (b) shows that when Smaller convolution kernels may lead to inaccurate recognition of trajectory point motion patterns, while too large kernel sizes will produce overly smoothed features. Figure 5 (c) Display and set the number of virtual semantics The best performance can be achieved. Increasing this parameter will increase memory and computation requirements, while too small a value may result in information loss. Figure 5 As shown in (d), a small number of POIs may lead to errors in the identification of the starting and ending points, while too many POIs will introduce noise. To ensure optimal model performance The above are only preferred embodiments of the present application and do not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A sparse trajectory pattern learning method based on mutual learning semantic distillation, characterized by: include: Obtaining a trajectory, and determining the starting time, trajectory points, and locations of points of interest of the trajectory; Obtaining spatial features of the road section to which the trajectory belongs, mapping the spatial features of the road section into shared parameters; determining the spatial feature embedding of each trajectory point based on the shared parameters; Obtaining the spatiotemporal features of each trajectory point, and determining an embedding vector for each trajectory point based on the spatiotemporal features and spatial features of the trajectory point; Constructing a semantic-based trajectory feature and a trajectory-based semantic feature according to the embedding vector of the trajectory point; Mapping the semantic-based trajectory features to a semantic space to obtain a trajectory-based semantic vector, and mapping the trajectory-based semantic features to the trajectory space to obtain a semantically enhanced trajectory vector; Fusing the trajectory-based semantic vector and the semantically enhanced trajectory vector, and interacting the fused vector with the embedding vector of each trajectory point to obtain the embedding vector of the trajectory; The embedding vector, starting time, location of the point of interest, and prompt word of the trajectory are input into the pre-trained LLM text encoder to obtain the trajectory pattern.

2. The sparse trajectory pattern learning method based on mutual learning semantic distillation according to claim 1 is characterized in that Mapping the spatial features of the road section into shared parameters; and determining the spatial feature embedding of each trajectory point based on the shared parameters, including: Convert the spatial features of the road section into spatial feature embedding; Embedding and mapping the spatial features of the road segment into shared parameters; Determine the spatial characteristic probability distribution of the road section by using a Gaussian distribution model in combination with the shared parameters; The spatial feature embedding of each trajectory point is determined according to the spatial feature probability distribution of the road section.

3. The sparse trajectory pattern learning method based on mutual learning semantic distillation according to claim 1 is characterized in that Determining the embedding vector of each trajectory point according to the spatiotemporal features and spatial features of the trajectory point includes: Converting the spatiotemporal features of the trajectory points into spatiotemporal feature embedding of the trajectory points; wherein the spatiotemporal features include the position, velocity, acceleration and direction of the trajectory points; An embedding vector for each trajectory point is determined according to the sum of the spatiotemporal feature embedding and the spatial feature embedding of the trajectory point.

4. The sparse trajectory pattern learning method based on mutual learning semantic distillation according to claim 1 is characterized in that The step of constructing a semantic-based trajectory feature according to the embedding vector of the trajectory point includes: Use the embedding vectors of all trajectory points to establish a trajectory feature set and randomly generate a semantic vector set; Determine, by means of a learnable projection matrix, the attention weight of each semantic vector in the set of semantic vectors to the embedding vectors of all trajectory points, and use it as the first attention weight; Determine a semantic-based trajectory feature corresponding to each semantic vector based on each of the first attention weights and the embedding vector of the corresponding trajectory point.

5. The sparse trajectory pattern learning method based on mutual learning semantic distillation according to claim 1 is characterized in that The step of constructing trajectory-based semantic features according to the embedding vectors of the trajectory points includes: Use the embedding vectors of all trajectory points to establish a trajectory feature set and randomly generate a semantic vector set; Using a learnable projection matrix to determine the attention weight of the embedding vector of each trajectory point and all semantic vectors in the trajectory feature set, and using it as the second attention weight; A trajectory-based semantic feature is determined according to each of the second attention weights and the corresponding semantic vector.

6. The sparse trajectory pattern learning method based on mutual learning semantic distillation according to claim 1 is characterized in that The step of fusing the trajectory-based semantic vector and the semantically enhanced trajectory vector, and interacting the fused vector with the embedding vector of each trajectory point to obtain the embedding vector of the trajectory includes: Multiplying the trajectory-based semantic representation and the semantically enhanced trajectory feature element-by-element to obtain a fusion vector; Interact the first fusion vector with the embedding vector of each trajectory point through product projection to obtain a transformed embedding vector of each trajectory point; The transformed embedding vectors of all trajectory points are residually connected with the embedding vectors of all trajectory points and processed by a feedforward neural network to generate the embedding vector of the trajectory.

7. The sparse trajectory pattern learning method based on mutual learning semantic distillation according to claim 1 is characterized in that The spatial characteristics of the road segment include road type, number of lanes, one-way street signs, spatial smoothness and road shape.

8. The sparse trajectory pattern learning method based on mutual learning semantic distillation according to claim 7 is characterized in that The spatial features of the road segment are converted into spatial feature embedding, including: Using an index retrieval embedding module, the road type, number of lanes, and one-way street sign are converted into a road type feature embedding, a lane number feature embedding, and a one-way street sign feature embedding respectively; Obtain a set of road vertices, use the road vertices as nodes, and use the nodes and the road sections connected by each node to build a road network graph; Performing multiple random walks on each node in the road network to generate a node sequence; The node sequence is converted into an embedding vector of a road segment using a Word2Vec model, and the embedding vector of the road segment is embedded as a spatial smoothness and road shape feature; The road type feature embedding, lane number feature embedding, one-way street sign feature embedding, and spatial smoothness and road shape feature embedding are spliced to obtain a spatial feature embedding.

9. The sparse trajectory pattern learning method based on mutual learning semantic distillation according to claim 1 is characterized in that The converting the spatiotemporal features of the trajectory points into spatiotemporal feature embedding of the trajectory points comprises: The spatiotemporal features of each trajectory point are convolved to obtain the spatiotemporal feature embedding of the trajectory point.

10. The sparse trajectory pattern learning method based on mutual learning semantic distillation according to claim 1 is characterized in that: The method for determining the pre-trained LLM text encoder is: A low-rank matrix is added to the parameters of the attention layer of the LLM text encoder, and the parameters except the attention layer are frozen during training to obtain the pre-trained LLM text encoder.

Images