A multi-modal trajectory representation learning method, device, equipment and medium

Abstract

The application discloses a multi-mode trajectory representation learning method, device and equipment and a medium, and relates to the field of traffic big data analysis. The method comprises the following steps: encoding each node in a GPS point pair sequence in target trajectory data and each node in a road network section index sequence into a vector to obtain initial embedding of a target downstream task, and inputting the initial embedding into a pre-trained trajectory encoding module to obtain trajectory embedding of the target downstream task; inputting the trajectory embedding into a downstream task prediction model to obtain a prediction result of the target downstream task; wherein the pre-trained trajectory encoding module is obtained by training a weight-sharing fusion encoder based on an attention mechanism using first training data constructed based on unlabeled multi-mode trajectory data; and the downstream task prediction model is obtained by training a downstream task prediction head based on a multi-layer perception using second training data. The application can improve the quality of trajectory representation and facilitate application.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of traffic big data analysis, and in particular to a multi-mode trajectory representation learning method, device and equipment and medium. BACKGROUND

[0002] With the increasing popularity of location-based services such as map platforms and location-based social networks, the generation and analysis of trajectory data have become more and more extensive. These data provide valuable insights into the movement patterns of individuals and vehicles. Trajectory data has rich spatio-temporal information and is widely used, including location-based social networks, intelligent transportation systems and urban computing.

[0003] Representation learning of trajectory data converts original trajectories into low-dimensional vectors, which can be used for various downstream tasks such as travel time estimation, trajectory similarity measurement and road label classification. For example, GPS trajectories contain spatio-temporal information, which indirectly represents the state of the transportation network. A common approach is to convert GPS trajectories into adjacent road segment indices as paths, and then convert road paths in the physical world into general representation vectors in network space for various downstream tasks, i.e. so-called trajectory representation learning.

[0004] Currently, research on trajectory representation learning reveals a positive trend of learning trajectory representation by fusing multi-mode trajectory data, i.e. combining two trajectory data formats, including GPS coordinates and trajectory adjacent road segment indices, can produce better representation than learning single-mode trajectory alone. However, related research needs to strictly align different modes of input and representation when fusing, or manually design rule-based or domain-specific contrastive objectives. Although the above "hard constraint" design has achieved certain results, it usually relies on labeled data for aligning multi-mode data and manual design, resulting in limited performance improvement and high annotation cost, affecting the quality of trajectory representation and increasing the difficulty of technical application. SUMMARY

[0005] The purpose of the present application is to provide a multi-mode trajectory representation learning method, device, equipment and medium, which can improve the quality of trajectory representation and facilitate application.

[0006] To achieve the above purpose, the present application provides the following solutions.

[0007] In a first aspect, the present application provides a multi-mode trajectory representation learning method, comprising the following steps.

[0008] Obtaining target trajectory data; the target trajectory data includes multi-mode trajectory data of a floating car in a target downstream task; the multi-mode trajectory data includes a sequence of GPS point pairs and a sequence of road network road segment indices.

[0009] Each node in the GPS point pair sequence and each node in the road network segment index sequence in the target trajectory data are encoded into a vector to obtain an initial embedding of the target downstream task.

[0010] The initial embedding of the target downstream task is input into the pre-trained trajectory encoding module to obtain a trajectory embedding of the target downstream task; the trajectory embedding is a representation learning result of the multi-mode trajectory; and the trajectory embedding of the target downstream task is used to input into a downstream task prediction model to obtain a prediction result of the target downstream task.

[0011] The pre-trained trajectory encoding module is obtained by training a weight-shared fusion encoder based on an attention mechanism using first training data; and the first training data is constructed based on unannotated multi-mode trajectory data.

[0012] The downstream task prediction model is obtained by training a downstream task prediction head based on a multi-layer perception using second training data; and the second training data includes a trajectory embedding of a downstream task for training output by the pre-trained trajectory encoding module and a real result of the downstream task for training.

[0013] In a second aspect, the present application provides a multi-mode trajectory representation learning device, including the following modules.

[0014] A data acquisition module is configured to acquire target trajectory data; the target trajectory data includes multi-mode trajectory data of a floating car in a target downstream task; and the multi-mode trajectory data includes a GPS point pair sequence and a road network segment index sequence.

[0015] An initial embedding determination module is configured to encode each node in the GPS point pair sequence and each node in the road network segment index sequence in the target trajectory data into a vector to obtain an initial embedding of the target downstream task.

[0016] A trajectory representation learning module is configured to input the target embedding vector into a pre-trained trajectory encoding module to obtain a trajectory embedding of the target downstream task; the trajectory embedding is a representation learning result of the multi-mode trajectory; and the trajectory embedding of the target downstream task is used to input into a downstream task prediction model to obtain a prediction result of the target downstream task.

[0017] The pre-trained trajectory encoding module is obtained by training a weight-shared fusion encoder based on an attention mechanism using first training data; and the first training data is constructed based on unannotated multi-mode trajectory data.

[0018] The downstream task prediction model is obtained by training a multi-layer perceptron-based downstream task prediction head using second training data; the second training data includes trajectory embeddings for a trained downstream task output by the pre-trained trajectory encoding module and real results for the trained downstream task.

[0019] In a third aspect, the present application provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor executes the computer program to implement the multi-modal trajectory representation learning method described above.

[0020] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, and the computer program is executed by a processor to implement the multi-modal trajectory representation learning method described above.

[0021] In a fifth aspect, the present application provides a computer program product comprising a computer program, and the computer program is executed by a processor to implement the multi-modal trajectory representation learning method described above.

[0022] According to the specific embodiments provided by the present application, the present application has the following technical effects: the present application provides a multi-modal trajectory representation learning method, device, equipment and medium, adopts unannotated multi-modal trajectory data and a fusion encoder based on attention mechanism and weight sharing to construct a pre-trained trajectory encoding module, realizes representation learning of multi-modal trajectory, constructs a downstream task prediction model based on a multi-layer perceptron-based downstream task prediction head to realize prediction of downstream tasks, the present application does not need annotated data and does not need domain-specific sampling or target learning strategy, can improve the quality of trajectory representation, and is convenient to apply. BRIEF DESCRIPTION OF DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained without creative labor on the basis of these drawings.

[0024] Figure 1 It is an application environment diagram of a multi-modal trajectory representation learning method in an embodiment of the present application.

[0025] Figure 2 It is a flowchart of a multi-modal trajectory representation learning method provided by an embodiment of the present application.

[0026] Figure 3 It is a training process diagram of a pre-trained trajectory encoding module provided by an embodiment of the present application.

[0027] Figure 4 A schematic diagram of the training process of a downstream task prediction model provided in one embodiment of the present application.

[0028] Figure 5 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0029] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0030] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0031] In this application, all actions to obtain signals, information or data are carried out in compliance with the relevant data protection laws and policies of the country where they are located and with the authorization given by the owner of the corresponding device.

[0032] The multi-modal trajectory representation learning method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown, the terminal 102 communicates with the server 104 via a network. The data storage system can store data that the server 104 needs to process. The data storage system can be set up separately, integrated on the server 104, or placed on the cloud or other servers. The terminal 102 can send target trajectory data to the server 104. After the server 104 receives the target trajectory data, the server 104 obtains the target trajectory data; encodes each node in the GPS point pair sequence and each node in the road network segment index sequence in the target trajectory data into a vector to obtain an initial embedding of the target downstream task; and inputs the initial embedding of the target downstream task into the pre-trained trajectory encoding module to obtain the trajectory embedding of the target downstream task. The server 104 can feed back the obtained prediction result of the target downstream task to the terminal 102. In addition, in some embodiments, the representation learning method of the multimodal trajectory can also be implemented independently by the server 104 or the terminal 102. For example, the terminal 102 can directly process the target trajectory data, or the server 104 can obtain the target trajectory data from the data storage system and process the target trajectory data.

[0033] The terminal 102 can be, but is not limited to, various desktop computers, notebook computers, smart phones, tablet computers, Internet of Things devices, and portable wearable devices. The Internet of Things device can be a smart speaker, a smart television, a smart air conditioner, a smart vehicle device, etc. The portable wearable device can be a smart watch, a smart bracelet, a head-mounted device, etc. The server 104 can be implemented by a single server or a server cluster composed of multiple servers, and can also be a cloud server.

[0034] In an exemplary embodiment, as shown in Figure 2 , a multi-modal trajectory representation learning method is provided, which is executed by a computer device, specifically, can be executed by a terminal or a server, or both. In the embodiments of the present application, the method is applied to the server 104 in Figure 1 , which includes the following steps 201 to 203. Wherein:

[0035] Step 201, obtaining target trajectory data; the target trajectory data includes: multi-modal trajectory data of a floating car at a target downstream task; the multi-modal trajectory data includes: a sequence of GPS point pairs and a sequence of road network link indexes.

[0036] Step 202, encoding each node in the sequence of GPS point pairs and each node in the sequence of road network link indexes in the target trajectory data into a vector to obtain an initial embedding of the target downstream task.

[0037] Step 203, inputting the initial embedding of the target downstream task into a pre-trained trajectory encoding module to obtain a trajectory embedding of the target downstream task.

[0038] Wherein, the trajectory embedding is the result of multi-modal trajectory representation learning; the trajectory embedding of the target downstream task is used to input into a downstream task prediction model to obtain a prediction result of the target downstream task.

[0039] Wherein, the pre-trained trajectory encoding module is obtained by training a weight-sharing fusion encoder based on attention mechanism using first training data; the first training data is constructed based on unlabeled multi-modal trajectory data.

[0040] The downstream task prediction model is obtained by training a multi-layer perception-based downstream task prediction head using second training data; the second training data includes: trajectory embedding of the downstream task for training output by the pre-trained trajectory encoding module and real result of the downstream task for training.

[0041] Implementing the above steps 201 to 203 can improve the quality of trajectory representation and facilitate application.

[0042] In another example embodiment of the present application, the target downstream task is a travel time estimation task, a trajectory similarity measurement task, or a road label classification task; and the prediction result of the target downstream task is a travel time estimation result, a trajectory similarity measurement result, or a road label classification result.

[0043] In another example embodiment of the present application, step 202 specifically comprises: (1) performing affine transformation on each node in the GPS point pair sequence in the target trajectory data to obtain GPS initial embedding of the target downstream task; (2) constructing a corresponding word embedding vector for each node in the road network link index sequence in the target trajectory data, and injecting the position information of each node into the corresponding word embedding vector to obtain road network link initial embedding of the target downstream task; and (3) determining the initial embedding of the target downstream task according to the GPS initial embedding of the target downstream task and the road network link initial embedding of the target downstream task.

[0044] In another example embodiment of the present application, the method for determining the pre-trained trajectory encoding module comprises the following steps.

[0045] (1) Obtain training trajectory data; the training trajectory data comprises multi-mode trajectory data of a floating car in a downstream task used for training.

[0046] (2) Perform data enhancement on the training trajectory data to generate multi-mode enhanced samples. Specifically:

[0047] For the GPS point pair sequence of the i-th trajectory in the training trajectory data, add a random value conforming to a Gaussian distribution to the GPS point pair sequence of the i-th trajectory to obtain a GPS positive sample trajectory sequence similar to the GPS point pair sequence of the i-th trajectory. i i For the GPS point pair sequence of the i-th trajectory in the training trajectory data, add a random value conforming to a Gaussian distribution to the GPS point pair sequence of the i-th trajectory to obtain a GPS positive sample trajectory sequence similar to the GPS point pair sequence of the i-th trajectory. i i For the GPS point pair sequence of the i-th trajectory in the training trajectory data, add a random value conforming to a Gaussian distribution to the GPS point pair sequence of the i-th trajectory to obtain a GPS positive sample trajectory sequence similar to the GPS point pair sequence of the i-th trajectory. i For the road network link index sequence of the i-th trajectory in the training trajectory data, determine a road network link positive sample trajectory sequence similar to the road network link index sequence of the i-th trajectory according to a set mask rate.

[0048] i For the road network link index sequence of the i-th trajectory in the training trajectory data, determine a road network link positive sample trajectory sequence similar to the road network link index sequence of the i-th trajectory according to a set mask rate. i i For the road network link index sequence of the i-th trajectory in the training trajectory data, determine a road network link positive sample trajectory sequence similar to the road network link index sequence of the i-th trajectory according to a set mask rate. i

[0049] ​​​​​The multi-mode enhanced samples are constructed according to the GPS point pair sequence, the GPS positive sample trajectory sequence, the GPS negative sample trajectory sequence, the road network link index sequence, the road network link positive sample trajectory sequence and the road network link negative sample trajectory sequence.

[0050] (3) Each node in the GPS point pair sequence in the multi-mode enhanced sample and each node in the road network link index sequence are encoded into vectors to obtain initial embeddings for training of a downstream task.

[0051] (4) First training data are determined according to the initial embeddings for training of the downstream task and mask training data; the mask training data include a partially masked road network link sequence and a true value of a masked link.

[0052] (5) The first training data are input into a weight-shared fusion encoder based on an attention mechanism, and training is performed with a contrastive learning loss function minimum as a target to obtain a pre-trained trajectory encoding module.

[0053] The weight-shared fusion encoder based on the attention mechanism includes a GPS trajectory encoder, a link trajectory encoder and a mask prediction encoder; the GPS trajectory encoder, the link trajectory encoder and the mask prediction encoder adopt a parameter sharing mechanism.

[0054] The GPS trajectory encoder is used for inputting GPS initial embeddings in the initial embeddings for training of the downstream task; the link trajectory encoder is used for inputting road network link initial embeddings in the initial embeddings for training of the downstream task; and the mask prediction encoder is used for inputting the mask training data.

[0055] The contrastive learning loss function is constructed according to a single mode loss function, a cross-mode loss function and a cross-entropy loss function between the partially masked road network link sequence and the true value of the masked link.

[0056] The determination method of the downstream task prediction model includes: (1) obtaining second training data; and (2) inputting the second training data into a downstream task prediction head based on a multi-layer perceptron to train a downstream task prediction model with a target loss function minimum as a target. The target loss function is determined according to a prediction result of the downstream task for training output by the downstream task prediction head based on the multi-layer perceptron and a true result of the downstream task for training.

[0057] A more detailed and complete training process of the pre-trained trajectory encoding module and the downstream task prediction model will be described below. Figure 3 Figure 4

[0058] ​​1) Obtain a multi-modal representation of trajectory data, including GPS point pair sequence form and road network road segment index sequence form, construct multi-modal trajectory data, and generate multi-modal enhanced samples as training trajectory data.

[0059] Specifically, assuming that there are N floating car trajectories in the obtained trajectory data, i.e. for the i-th trajectory data, its multi-modal trajectory data includes the following two parts: the GPS point pair sequence of the floating car , and the road network road segment index sequence .

[0060] Wherein, the multi-modal enhanced sample is generated, including the following sub-steps.

[0061] For the GPS point pair sequence , add a random value conforming to the (0, 1) Gaussian distribution to the value of each element to obtain a GPS positive sample trajectory sequence similar to the original GPS point pair sequence ; select the i+1-th GPS point pair sequence as the GPS negative sample trajectory sequence .

[0062] For the road network road segment index sequence , determine whether to mask each element value with a specific mask probability to obtain a road network road segment positive sample trajectory sequence similar to the original road network road segment index sequence ; select the i+1-th road network road segment index sequence as the road network road segment negative sample trajectory sequence .

[0063] 2) Convert each node in each sequence in the multi-modal enhanced sample into a vector to convert the training trajectory data into a compact vector, form an initial embedding for training downstream tasks, and input the initial embedding into the attention mechanism-based weight-shared fusion encoder.

[0064] Specifically, the initial embedding is formed, including the following sub-steps.

[0065] For the enhanced sample of the GPS point pair sequence , , , respectively, perform affine transformation on each node in each sample to obtain the corresponding GPS initial embedding , , .

[0066] For the enhanced sample of the road network road segment index sequence , , , respectively, for each node in each sample, and inject the relative position information of each node into the word embedding vector to obtain the corresponding initial embedding of the road network section 、 、 .

[0067] 3) Build a weight-shared fusion encoder based on the attention mechanism, and constrain the node vector representation of each sequence using weight sharing and the attention mechanism to obtain a sequence embedding vector.

[0068] As shown in Figure 3 , specifically, the initial embedding of the downstream task for training is input into the weight-shared fusion encoder based on the attention mechanism, and the trajectory embedding of different views is output. The weight-shared fusion encoder based on the attention mechanism includes a GPS trajectory encoder, a road section trajectory encoder, and a mask prediction encoder. The initial embedding represents a vector of a sequence composed of nodes, with a size of BxSxD, B representing the batch size, S representing the sequence length, and D representing the feature dimension of the initial embedding; the trajectory embedding represents a vector obtained by fusing all node information of the sequence, with a size of BxD', D' representing the feature dimension of the trajectory embedding. The trajectory embedding is a low-dimensional vector obtained by encoding the initial embedding.

[0069] Specifically, the weight-shared fusion encoder based on the attention mechanism is represented as follows.

[0070] .

[0071] wherein, represents an optional embedding vector in the initial embedding, including a GPS point pair sequence or a road network section index sequence; represents a fusion encoder composed of several layers of attention mechanisms, which shares weights for different modes of input; represents the trajectory embedding; represents the initial embedding.

[0072] Based on the initial embedding generated in step 2) 、 、 、 、 、 , step 3) uses weight sharing and the attention mechanism to constrain the node vector representation of each sequence to obtain a sequence embedding vector, thereby outputting the trajectory embedding 、 、 、 、 、 .

[0073] Based on the two-mode trajectory embedding of step 2), the contrastive sample combination in a single mode is a sequence of GPS points or a sequence of road segment indexes, denoted as (X, X) and (X, X) respectively. 、 , , The single-mode loss function (i.e., the contrastive loss component in a single mode) is used as a similarity measure for the above sample combinations. The single-mode loss function includes a first loss function L1 and a second loss function L2, which are expressed as follows.

[0074] .

[0075] .

[0076] The similarity between a sequence of GPS points and its corresponding positive GPS trajectory sequence and negative GPS trajectory sequence is quantified. The similarity between a sequence of road segment indexes and its corresponding positive road segment trajectory sequence and negative road segment trajectory sequence is quantified. N represents the number of floating car trajectories, W represents a weight matrix that adjusts the importance of negative sample items, T represents a temperature coefficient, COS represents the cosine similarity between and COS represents the cosine similarity between and COS represents the cosine similarity between and COS represents the cosine similarity between and COS represents the cosine similarity between and COS represents the cosine similarity between and COS represents the cosine similarity between and The first loss function corresponding to all floating car trajectories is represented as L1. The second loss function corresponding to all floating car trajectories is represented as L2. Since the two-mode trajectory embedding is a different representation of the same target trajectory, the two-mode trajectory embedding is further reorganized into a contrastive sample combination in a cross-mode (X, X) and (X, X).

[0077] 、 , , ​​​​​​), the contrast sample combination in the cross-mode, that is, the contrast sample combination formed by the combination of the GPS point pair sequence and the road network link index sequence, the above combination is the different modes of the GPS point pair sequence and the road network link index being the same track, therefore, the samples between the two modes should enjoy the similarity similar to the contrast sample combination in the single mode; the cross-mode loss function (the contrast loss component in the cross-mode) is the similarity measurement of the above sample combination, the cross-mode loss function includes a third loss function and a fourth loss function , and the specific expressions are as follows.

[0078] .

[0079] .

[0080] is used to quantify the similarity of the GPS point pair sequence and the road network link positive sample track sequence and the road network link negative sample track sequence represented by the road network link index sequence; is used to quantify the similarity of the road network link index sequence and the GPS positive sample track sequence and the GPS negative sample track sequence represented by the GPS point pair sequence. represents the cosine similarity of and ; represents the cosine similarity of and ; represents the cosine similarity of and ; represents the cosine similarity of and . The third loss function corresponding to all the probe vehicle tracks is represented as ; the fourth loss function corresponding to all the probe vehicle tracks is represented as .

[0081] The cross-mode loss function evaluates the similarity between the track embeddings of different views in the cross-mode.

[0082] The single-mode loss function and the cross-mode loss function are the similarity measurements between the target samples and their positive and negative samples, and the purpose of the similarity measurement is to make the similarity between the target sample and its positive sample as large as possible, and the similarity between the target sample and its negative sample as small as possible.

[0083] 4) Perform trajectory-segment overall contrast pre-training learning, and use an auxiliary masked segment completion task to enhance learning, learn path representation by maximizing the similarity prediction of four components of the contrast learning loss, and maximize the prediction accuracy of the masked segment completion to enhance the learning of the segment representation.

[0084] Specifically, in order to enhance the parameter update of the trajectory embedding and stabilize the effect of the pre-training process, a masked segment completion task is introduced, that is, to determine whether to mask each node in the partially masked road network segment sequence with a certain probability, and to calculate the cross-entropy loss between the predicted completion value of the masked segment and the true value of the masked segment. The cross-entropy loss function is represented as The cross-entropy loss function corresponding to all probe car trajectories is represented as .

[0085] Therefore, the expression of the trajectory-segment overall contrast learning loss function is as follows.

[0086] .

[0087] 5) Freeze the parameter gradients of the pre-training stage modules in step 4), construct a multi-layer perceptron-based downstream task prediction head for a specific downstream task, and fine-tune and optimize the task.

[0088] Specifically, for a specific trajectory / segment-related downstream task, a task-specific trajectory / segment dataset is constructed, and a multi-layer perceptron-based downstream task prediction head is constructed; the parameter gradient information of the modules in steps 2) and 3) is frozen.

[0089] According to step 2), it is converted into an initial embedding and input into the weight-shared fusion encoder based on the attention mechanism in step 3), and the trajectory embedding of the trajectory / segment is output. Further, it is input into the multi-layer perceptron-based downstream task prediction head to obtain the output for the downstream task; further, the prediction result of the downstream task is calculated with the true result of the downstream task (i.e. the task-specific trajectory data), and the target loss function is calculated. The multi-layer perceptron-based downstream task prediction head is fine-tuned, and the downstream task output is optimized.

[0090] The entire pre-training and fine-tuning process is shown in Figure 4 .

[0091] The existing trajectory contrast representation learning method usually relies on strictly aligned labeled data or problem-specific objective function / module design, resulting in limited performance improvement and high annotation cost. Based on this, the embodiment of the present application proposes a simplified contrast representation learning and fusion method based on probe car multi-mode trajectories, which predicts the same relationship between different / same mode trajectories without labeled data and without domain-specific sampling or target learning strategy.

[0092] Based on the same inventive concept, the embodiments of the present application also provide a multi-modal trajectory representation learning device for implementing the multi-modal trajectory representation learning method described above. The implementation scheme for solving problems provided by the device is similar to the implementation scheme described in the above method, so the specific limitations in one or more multi-modal trajectory representation learning device embodiments provided below can refer to the limitations of the multi-modal trajectory representation learning method in the above text, which will not be described here again.

[0093] In one exemplary embodiment, a multi-modal trajectory representation learning device is provided, which includes the following modules.

[0094] A data acquisition module is configured to acquire target trajectory data, wherein the target trajectory data includes multi-modal trajectory data of a floating car at a target downstream task, and the multi-modal trajectory data includes a sequence of GPS point pairs and a sequence of road network segment indexes.

[0095] An initial embedding determination module is configured to encode each node in the sequence of GPS point pairs and each node in the sequence of road network segment indexes in the target trajectory data into a vector to obtain an initial embedding of the target downstream task.

[0096] A trajectory representation learning module is configured to input the target embedding vector into a pre-trained trajectory encoding module to obtain a trajectory embedding of the target downstream task, wherein the trajectory embedding is a representation learning result of the multi-modal trajectory, and the trajectory embedding of the target downstream task is used to input a downstream task prediction model to obtain a prediction result of the target downstream task.

[0097] The pre-trained trajectory encoding module is obtained by training a weight-sharing fusion encoder based on an attention mechanism using first training data, and the first training data is constructed based on unlabeled multi-modal trajectory data.

[0098] The downstream task prediction model is obtained by training a downstream task prediction head based on a multi-layer perception using second training data, and the second training data includes a trajectory embedding of the downstream task for training output by the pre-trained trajectory encoding module and a real result of the downstream task for training.

[0099] The embodiments of the present application design a simplified comparative learning and fusion framework for multi-modal trajectories of floating cars by comparing the same relationship between the predicted trajectories of one mode and the trajectories of another mode, so as to solve the problem that the complex hand-specific design of the traditional trajectory representation learning deep learning framework leads to difficult application of the framework and suboptimal effect of downstream application.

[0100] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Figure 5 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O) and a communication interface. The processor, memory and input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device is used to store target trajectory data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a multi-modal trajectory representation learning device is implemented.

[0101] Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0102] In an exemplary embodiment, a computer device is further provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps in the above method embodiments when executing the computer program.

[0103] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0104] In an exemplary embodiment, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0105] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when the computer program is executed, the processes of the above-mentioned embodiments of the methods can be included. Any reference to memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration but not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0106] The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided in the present application can be a general processor, a central processor, a graphics processor, a digital signal processor, a programmable logic device, etc., without being limited thereto.

[0107] The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the technical features in the above embodiments are not described, however, as long as the combinations of the technical features do not exist contradictory, they should be considered as the scope of the present application.

[0108] The principles and implementation modes of the present application are described by applying specific examples in the present application. The above-mentioned embodiments are only used to help understand the method and its core idea of the present application; at the same time, for those skilled in the art, according to the idea of the present application, the specific implementation mode and application range will be changed. In conclusion, the content of the present application should not be understood as a limitation.

Claims

1. A multi-modal trajectory representation learning method, characterized in that, The representation learning method of the multi-mode trajectory comprises: obtaining target trajectory data; the target trajectory data comprises multi-mode trajectory data of a floating car in a target downstream task; the multi-mode trajectory data comprises a sequence of GPS point pairs and a sequence of road network link indexes; encoding each node in the sequence of GPS point pairs and each node in the sequence of road network link indexes in the target trajectory data into a vector to obtain initial embedding of the target downstream task; inputting the initial embedding of the target downstream task into a pre-trained trajectory encoding module to obtain trajectory embedding of the target downstream task; the trajectory embedding is a representation learning result of the multi-mode trajectory; the trajectory embedding of the target downstream task is used to input into a downstream task prediction model to obtain a prediction result of the target downstream task; wherein the pre-trained trajectory encoding module is obtained by training a weight-shared fusion encoder based on an attention mechanism using first training data; the first training data is constructed based on unannotated multi-mode trajectory data; the downstream task prediction model is obtained by training a downstream task prediction head based on a multi-layer perceptron using second training data; the second training data comprises trajectory embedding of the downstream task for training output by the pre-trained trajectory encoding module and real result of the downstream task for training. 2.The multi-modal trajectory representation learning method of claim 1, wherein, the target downstream task is a travel time estimation task, a trajectory similarity measurement task or a road label classification task; the prediction result of the target downstream task is a travel time estimation result, a trajectory similarity measurement result or a road label classification result. 3.The multi-modal trajectory representation learning method of claim 1, wherein, the determination method of the pre-trained trajectory encoding module comprises: obtaining training trajectory data; the training trajectory data comprises multi-mode trajectory data of a floating car in a downstream task for training; performing data enhancement on the training trajectory data to generate multi-mode enhanced samples; encoding each node in the sequence of GPS point pairs and each node in the sequence of road network link indexes in the multi-mode enhanced samples into a vector to obtain initial embedding of the downstream task for training; determining first training data according to the initial embedding of the downstream task for training and mask training data; the mask training data comprises a partially masked sequence of road network links and real values of the masked links; inputting the first training data into the weight-shared fusion encoder based on the attention mechanism and training with the objective of minimizing a contrastive learning loss function to obtain the pre-trained trajectory encoding module; wherein the contrastive learning loss function is constructed according to a single-mode loss function, a cross-mode loss function and a cross-entropy loss function between predicted completion values of the masked links and real values of the masked links.

4. The multi-modal trajectory representation learning method of claim 3, wherein, the determination method of the downstream task prediction model comprises: obtaining second training data; inputting the second training data into a downstream task prediction head based on a multi-layer perceptron to train the downstream task prediction model with the objective of minimizing a target loss function; wherein the target loss function is determined according to prediction results of the downstream task for training output by the downstream task prediction head based on the multi-layer perceptron and real results of the downstream task for training.

5. The multi-modal trajectory representation learning method of claim 3, wherein, The training trajectory data is data augmented to generate multi-mode augmented samples, specifically including: For the GPS point pair sequence of the first i GPS point pair sequence of the first i GPS point pair sequence of the first i GPS point pair sequence of the first i GPS point pair sequence of the first i GPS point pair sequence of the first For the training trajectory data i The road network segment index sequence of the trajectory is determined according to the set mask rate. i The positive sample trajectory sequence of the road network segment with similar road network segment index sequence of the trajectory is i The road network segment index sequence of the +1 trajectory is determined as i A sequence of negative sample trajectories of road network segments with trajectories; According to the GPS point pair sequence, the GPS positive sample trajectory sequence, the GPS negative sample trajectory sequence, the road network road segment index sequence, the road network road segment positive sample trajectory sequence and the road network road segment negative sample trajectory sequence, a multi-mode augmented sample is constructed.

6. The multi-modal trajectory representation learning method of claim 3, wherein, The attention mechanism-based weight sharing fusion encoder includes a GPS trajectory encoder, a road segment trajectory encoder and a mask prediction encoder; the GPS trajectory encoder, the road segment trajectory encoder and the mask prediction encoder adopt a parameter sharing mechanism; The GPS trajectory encoder is used for inputting a GPS initial embedding in initial embeddings for training of a downstream task; the road segment trajectory encoder is used for inputting a road network road segment initial embedding in initial embeddings for training of the downstream task; and the mask prediction encoder is used for inputting mask training data.

7. The multi-modal trajectory representation learning method according to claim 1, characterized in that, Each node in the GPS point pair sequence and each node in the road network road segment index sequence in the target trajectory data are encoded into vectors to obtain initial embeddings of the target downstream task, specifically including: Each node in the GPS point pair sequence in the target trajectory data is subjected to affine transformation to obtain GPS initial embeddings of the target downstream task; Each node in the road network road segment index sequence in the target trajectory data is constructed into a corresponding word embedding vector, and position information of each node is injected into the corresponding word embedding vector to obtain road network road segment initial embeddings of the target downstream task; The initial embeddings of the target downstream task are determined according to the GPS initial embeddings of the target downstream task and the road network road segment initial embeddings of the target downstream task.

8. A multimodal trajectory representation learning device, characterized in that: The multi-mode trajectory representation learning device includes: A data acquisition module is configured to acquire target trajectory data; the target trajectory data includes multi-mode trajectory data of a floating car in a target downstream task; the multi-mode trajectory data includes a GPS point pair sequence and a road network road segment index sequence; An initial embedding determination module is configured to encode each node in the GPS point pair sequence and each node in the road network road segment index sequence in the target trajectory data into vectors to obtain initial embeddings of the target downstream task; A trajectory representation learning module is configured to input the target embedding vectors into a pre-trained trajectory encoding module to obtain trajectory embeddings of the target downstream task; the trajectory embeddings are used as a representation learning result of the multi-mode trajectory; and the trajectory embeddings of the target downstream task are used as input into a downstream task prediction model to obtain a prediction result of the target downstream task. The pre-trained trajectory encoding module is obtained by training an attention mechanism-based weight sharing fusion encoder using first training data; the first training data is constructed based on unannotated multi-mode trajectory data; The downstream task prediction model is obtained by training a multi-layer perception-based downstream task prediction head using second training data; the second training data includes trajectory embeddings of the downstream task for training output by the pre-trained trajectory encoding module and real results of the downstream task for training.

9. A computer device comprising: A memory, a processor, and a computer program stored on the memory and runnable on the processor, characterized in that the processor executes the computer program to implement the multi-modal trajectory representation learning method of any one of claims 1-7.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the multi-modal trajectory representation learning method of any one of claims 1-7.

Images