Trajectory prediction method based on semantic fusion representation

Abstract

The present invention discloses a trajectory prediction method based on semantic fusion representation, including: using a trajectory data preprocessing method to remove error points and redundant points, and using a sliding window to divide the trajectory data to form a labeled trajectory sequence; combining the latitude and longitude information, time information, speed information and direction information of the vehicle to perform multi-dimensional trajectory fusion vector representation; using an autoencoder to learn the deep features of the trajectory sequence, combining the original features of the trajectory sequence, and jointly constructing the semantic representation of the trajectory sequence; a transformer-based trajectory prediction method, through a multi-head self-attention mechanism and a mask-based self-attention method, learns the correlation between trajectories, and then realizes trajectory prediction. The method can better predict the position and regional distribution of the car in the future, improve the accuracy of predicting the vehicle trajectory, and enable the driver to change the travel time or travel trajectory in advance to avoid traffic congestion.

Description

Technical Field

[0001] The invention relates to a trajectory prediction method based on semantic fusion representation. Background Art

[0002] With the rapid development of the Internet and the Internet of Things, a massive amount of trajectory data has been generated. Trajectory data includes traffic trajectory data, human activity trajectory data, and trajectory data of other movable objects. By mining trajectory data, the activity patterns of moving objects can be obtained.

[0003] However, traditional methods for predicting the location of vehicles are too simplistic and inaccurate. Specifically, time series models are widely used to predict vehicle trajectories, and more representative time series models are RNN, LSTM, etc. However, the current trajectory prediction model lacks semantic fusion representation of vehicle trajectory points and trajectory sequences, and traditional time series models are difficult to capture the correlation between trajectory points, so the overall prediction accuracy of the model is not high.

[0004] Therefore, in view of the fact that traditional methods lack semantic mining representation of multimodal trajectory data and acquisition of correlation information between trajectory points, the present invention proposes a trajectory prediction method based on semantic fusion representation. Summary of the invention

[0005] The purpose of the present invention is to provide a trajectory prediction method based on semantic fusion representation, which can better predict the location and regional distribution of cars in the future, improve the accuracy of predicting vehicle trajectories, and enable drivers to change travel time or travel trajectory in advance to avoid traffic congestion.

[0006] In order to achieve the above object, the present invention provides a trajectory prediction method based on semantic fusion representation, the method comprising:

[0007] Step 1: Use the trajectory data preprocessing method to remove error points and redundant points, and use the sliding window to divide the trajectory data to form a labeled trajectory sequence;

[0008] Step 2: Combine the latitude and longitude information, time information, speed information and direction information of the vehicle to perform multi-dimensional trajectory fusion vector representation;

[0009] Step 3: Use the autoencoder to learn the deep features of the trajectory sequence, and combine the original features of the trajectory sequence to jointly construct the semantic representation of the trajectory sequence;

[0010] Step 4: The transformer-based trajectory prediction method learns the correlation between trajectories through the multi-head self-attention mechanism and the mask-based self-attention method, thereby realizing trajectory prediction.

[0011] Preferably, the method further comprises step 5: verifying the trajectory prediction model.

[0012] Preferably, step 1 comprises:

[0013] Step 1.1: Error point removal;

[0014] Let the trajectory point p i and p j The time interval between them is Δt1, and the spatial distance is Δd1;

[0015] Let the trajectory point p j and p k The time interval between them is Δt2, and the spatial distance is Δd2;

[0016] Assume that the upper limit of vehicle speed on urban roads is V max ;

[0017] If the condition is met: Δd1>Δt1*V max And Δd2>Δt2*V max , then determine the trajectory point p j is the error point and should be removed;

[0018] Step 1.2: Remove redundant points;

[0019] Let the trajectory point p i 、p j 、p k 、p n The distances between them are Δd1, Δd2 and Δd n , set the circle radius threshold R = 20m, if the conditions Δd1<2*R, Δd2<2*R and Δd n <2*R, it is considered that the position of the vehicle remains basically unchanged during the time period, and the trajectory point p is used as the equivalent point of the redundant point in the area; the longitude and latitude of p are calculated according to the average value in the redundant area;

[0020] Step 1.3: Trajectory sequence formation;

[0021] The trajectory is divided based on a sliding window. A sliding window of fixed length is set at the beginning of the trajectory. The last trajectory point in the window is set as the position point to be predicted, and the remaining trajectory points are training features to form a training sample. The window slides forward one position in turn to form a new training sample, which is added to the training feature sequence and label sequence respectively. When the window reaches the last position of the trajectory sequence and extracts the trajectory point, the division ends. Specifically, in the time window T j , trajectory T = {p1, p2, ..., p n}, replace p1,p2,...,p j-1 As a training feature, pj as the next position label for the track.

[0022] Preferably, in step 2, the multimodal semantic trajectory is set as Traj(o i )={p1(o i ),p2(o i ),...,p n (o i )}, where p n (o i ) represents the object o i The trajectory semantic information of the nth position, that is, p n (o i )={L n ,T n ,D n ,I n}; where L n Represents object o i At the nth position, the longitude and latitude semantic information, T n Representing temporal semantic information, D n Represents the vehicle speed information, I n Characterize the direction information of the vehicle;

[0023] The multi-dimensional trajectory fusion vector representation includes:

[0024] Semantic representation of trajectory latitude and longitude features, including the use of grid division to represent the area formed within a certain latitude and longitude range, in order to capture the semantic information of latitude and longitude features;

[0025] Assuming the grid is divided into n segments, then L n It can be expressed as an n×1 dimensional vector; at the same time, the design dimension is D l ×n transformation vector E l , L n Convert to D l ×1 vector The formula is as follows:

[0026]

[0027] Semantic representation of trajectory temporal features, including characterizing time periods using a grid division method to capture the semantic information of temporal features;

[0028] Set the grid division to be based on hours as the period, and divide one hour into m segments, then T n It can be expressed as an m×1 dimensional vector; at the same time, the design dimension is D t ×m transformation vector E t , T n Convert to D t ×1 vector The formula is as follows:

[0029]

[0030] The semantic representation of vehicle speed information corresponds to the speed information contained in the trajectory information, and the number of discrete values ​​of vehicle speed is set to V. First, the speed information D n Encoded as a V×1 vector, V represents the number of discrete speed values ​​in the current data set; then, use the dimension D d ×V conversion matrix E d D n Convert to The formula is as follows:

[0031]

[0032] The semantic representation of vehicle direction information corresponds to the vehicle direction information contained in the trajectory. The number of discrete values ​​of vehicle direction information is set to Q, then I n The dimension vector is Q×1, and the dimension used is D i ×Q conversion matrix E i Will I n Convert to The formula is as follows:

[0033]

[0034] At this point, the multi-dimensional trajectory semantics p is calculated n (o i )The formula is as follows:

[0035]

[0036] Preferably, step 3 comprises:

[0037] Step 3.1: Encoding;

[0038] The encoder-decoder autoencoder is used to learn the deep features of the trajectory sequence, that is, the obtained multi-dimensional trajectory semantic sequence is input into the encoder part of the autoencoder, and the hidden layer is updated as shown in the following formula:

[0039] h i =f encoder (h i-1 ,b i )

[0040] Among them, f encoder represents the encoder function of the autoencoder, b i Represents the semantic input of trajectory points;

[0041] Step 3.2: Decoding;

[0042] The final output of the hidden layer of the Encoder in step 3.1 is h i will represent the entire trajectory sequence and will serve as the initial hidden layer of the decoder LSTM to obtain the output sequence {c1,c2,...,c i}, the decoder’s hidden layer is updated as follows:

[0043] h' i =f decoder (h' i-1 ,c i-1 )

[0044] Among them, f decoder represents the decoder function of the autoencoder, c i-1 represents the output of the decoder;

[0045] Step 3.3: The learning target of the decoder is the input of the encoder. The encoder and decoder pass the minimum error. The error adopts the following function:

[0046]

[0047] The output of the trained encoder can effectively represent the input data, that is, the output information contains the deep feature information of the trajectory sequence.

[0048] Preferably, step 4 includes trajectory prediction model encoding training and trajectory prediction model decoding training.

[0049] Preferably, in the trajectory prediction model encoding training, it is assumed that the representation vector of the trajectory sequence is T=(T1+T2+...+T N ), where N is the number of trajectory points, T i For the semantic representation of each trajectory point, the input of the trajectory sequence needs to add one at the end of the sequence, and the length of the sequence is set to F. For trajectory sequences that are less than F, the length is made F through Padding. Similarly, add <s>, add at the end< / s> , set the output prediction sequence length of the trajectory to M; including:

[0050] First, the training is carried out in batch mode. The batch size is set to B, and the dimension of the input trajectory sequence is (B, F). The dimensions of trajectory encoding and position encoding are set to be the same, both E. After trajectory embedding and position embedding, the vector dimension is (B, F, E).

[0051] Secondly, adjust the dimension of the vector after embedding from (B, F, E) to (B*F, E), and then use the vector to construct the query, key, and value matrices so that their dimensions are all (B*F, E); then re-transpose the dimensions of the query, key, and value matrices from (B*F, E) to (B, F, E); due to the use of a multi-head attention mechanism, the number of heads is set to N, and N*H=E, then query, key, and value are divided into (B, F, N, H), and the attention scores are calculated based on query, key, and value. The formula is as follows:

[0052]

[0053] Finally, the attention vector is calculated using the scores and value matrix;

[0054] After obtaining the attention vector, the input dimension is (B, F, E), which is input into the fully connected layer. The dimension of the intermediate hidden variable is set to D. The different words in the sequence are multiplied by the weight matrix (E, D), and then multiplied by another weight matrix (D, E), so the final dimension of the matrix is ​​(B, F, E); after passing through the remaining 5 layers of encoders, the final encoder output is (B, F, E).

[0055] Preferably, in the trajectory prediction model decoding training, the decoder of the training process receives the input sequence (B, T), performs trajectory encoding and position encoding on it, and the trajectory encoding and position encoding are the same set of weight matrices as those in the encoder; including:

[0056] First, after the input sequence is trajectory encoded and position encoded, the matrix dimension is (B, T, E), which is encoded using the masked multi-head attention mechanism; the masked multi-head attention has the same structure as the self-attention network, but adds a mask matrix to add the mask matrix to the complete sequence T of the input during training to mask future information;

[0057] Secondly, after the multi-head attention mechanism, the information encoded by the decoder needs to be calculated with the information encoded by the encoder, the key and value vectors are obtained from the information encoded by the encoder, the query vector is constructed from the internal sequence variables of the decoder, and then the attention operation is performed; the decoder repeats the same structure 5 times, and the final output sequence embedding is represented as (B, T, E);

[0058] Then, the (B, T, E) vector output by the decoder is connected to the fully connected layer to the logits space of the region table, becoming (B, T, Z); where Z is the size of the region table; the logits vector is connected to a layer of softmax to obtain the probability, and the predicted sequence is obtained based on the probability. The cross entropy loss function is used to calculate the loss function value for the predicted sequence and the target sequence, and then the parameter optimization process is started.

[0059] Preferably, step 5 comprises:

[0060] Input the trajectory sequence to be predicted into the encoder to obtain an encoder encoding vector (F, E), and then input the start marker into the decoder <s>A single element sequence is used to predict the next trajectory area T1, and then <s> It is concatenated with T1 as the input sequence of the decoder, and the next trajectory area is predicted until the predicted sequence has< / s> until.

[0061] According to the above technical scheme, the present invention pre-processes the data based on the analysis of existing vehicle trajectory data, eliminates error points and redundant points in the trajectory data, and forms trajectory sequence data. Then, a multimodal semantic fusion representation method is proposed to integrate the vehicle's position, time, speed and direction information. Finally, a Transformer-based trajectory prediction method is adopted to realize vehicle trajectory prediction. Compared with the existing method, the Transformer trajectory prediction method based on semantic fusion representation proposed in the present invention can integrate multimodal trajectory semantic representations, and effectively realize the trajectory prediction method by extracting the correlation between trajectories.

[0062] Other features and advantages of the present invention will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the following specific embodiments, they are used to explain the present invention but do not constitute a limitation of the present invention. In the accompanying drawings:

[0064] Figure 1 is a schematic diagram of deep features of trajectory learning by the automatic encoder in the present invention;

[0065] Figure 2 It is a flow chart of the transformer-based trajectory prediction method in the present invention. DETAILED DESCRIPTION

[0066] The specific implementation of the present invention is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described here is only used to illustrate and explain the present invention, and is not used to limit the present invention.

[0067] The present invention provides a trajectory prediction method based on semantic fusion representation, the method comprising:

[0068] Step 1: Use the trajectory data preprocessing method to remove error points and redundant points, and use the sliding window to divide the trajectory data to form a labeled trajectory sequence;

[0069] Step 2: Combine the latitude and longitude information, time information, speed information and direction information of the vehicle to perform multi-dimensional trajectory fusion vector representation;

[0070] Step 3: Use the autoencoder to learn the deep features of the trajectory sequence, and combine the original features of the trajectory sequence to jointly construct the semantic representation of the trajectory sequence;

[0071] Step 4: Transformer-based trajectory prediction method (such as Figure 2 As shown in the figure, through the multi-head self-attention mechanism and the mask-based self-attention method, the correlation between trajectories is learned to achieve trajectory prediction.

[0072] Wherein, step 1 comprises:

[0073] Step 1.1: Error point removal;

[0074] Let the trajectory point p i and p j The time interval between them is Δt1, and the spatial distance is Δd1;

[0075] Let the trajectory point p j and p k The time interval between them is Δt2, and the spatial distance is Δd2;

[0076] Assume that the upper limit of vehicle speed on urban roads is V max ;

[0077] If the condition is met: Δd1>Δt1*V max And Δd2>Δt2*V max , then determine the trajectory point p j is the error point and should be removed;

[0078] Step 1.2: Remove redundant points;

[0079] Let the trajectory point p i 、p j 、p k 、p n The distances between them are Δd1, Δd2 and Δd n , set the circle radius threshold R = 20m, if the conditions Δd1<2*R, Δd2<2*R and Δd n <2*R, it is considered that the position of the vehicle remains basically unchanged during the time period, and the trajectory point p is used as the equivalent point of the redundant point in the area; the longitude and latitude of p are calculated according to the average value in the redundant area;

[0080] Step 1.3: Trajectory sequence formation;

[0081] The research goal of the present invention is to predict the next position of the trajectory. Directly using the last position of the trajectory as a label is not conducive to the prediction result of the trajectory. Therefore, the present invention divides the trajectory based on a sliding window. A sliding window of fixed length is set at the beginning of the trajectory. The last trajectory point in the window is set as the position point to be predicted, and the remaining trajectory points are training features to form a training sample. The window slides forward one position in turn to form a new training sample, which is added to the training feature sequence and the label sequence respectively. When the window reaches the last position of the trajectory sequence and extracts the trajectory point, the division ends. Specifically, in the time window T j , trajectory T = {p1, p2, ..., p n }, replace p1,p2,...,p j-1 As a training feature, p j as the next position label for the track.

[0082] In step 2, set the multimodal semantic trajectory as Traj(o i )={p1(o i ),p2(o i ),...,p n (o i )}, where p n (o i ) represents the object o i The trajectory semantic information of the nth position, that is, p n (o i )={L n ,T n ,D n ,I n }; where L n Represents object o i At the nth position, the longitude and latitude semantic information, T n Representing temporal semantic information, D n Represents the vehicle speed information, I n Characterize the direction information of the vehicle;

[0083] The multi-dimensional trajectory fusion vector representation includes:

[0084] Semantic representation of trajectory latitude and longitude features, including the use of grid division to represent the area formed within a certain latitude and longitude range, in order to capture the semantic information of latitude and longitude features;

[0085] Assuming the grid is divided into n segments, then L n It can be expressed as an n×1 dimensional vector; at the same time, the design dimension is D l ×n transformation vector E l , L n Convert to D l ×1 vector The formula is as follows:

[0086]

[0087] Semantic representation of trajectory temporal features, including characterizing time periods using a grid division method to capture the semantic information of temporal features;

[0088] Set the grid division to be based on hours as the period, and divide one hour into m segments, then T n It can be expressed as an m×1 dimensional vector; at the same time, the design dimension is D t ×m transformation vector E t , T n Convert to D t ×1 vector The formula is as follows:

[0089]

[0090] The semantic representation of vehicle speed information corresponds to the speed information contained in the trajectory information, and the number of discrete values ​​of vehicle speed is set to V. First, the speed information D n Encoded as a V×1 vector, V represents the number of discrete speed values ​​in the current data set; then, use the dimension D d ×V conversion matrix E d D n Convert to The formula is as follows:

[0091]

[0092] The semantic representation of vehicle direction information corresponds to the vehicle direction information contained in the trajectory. The number of discrete values ​​of vehicle direction information is set to Q, then I n The dimension vector is Q×1, and the dimension used is D i ×Q conversion matrix E i Will I n Convert to The formula is as follows:

[0093]

[0094] At this point, the multi-dimensional trajectory semantics p is calculated n (o i )The formula is as follows:

[0095]

[0096] Step 3 includes:

[0097] Step 3.1: Encoding;

[0098] The encoder-decoder autoencoder is used to learn the deep features of the trajectory sequence. The autoencoder structure is as follows Figure 1 The obtained multi-dimensional trajectory semantic sequence is input into the encoder part of the autoencoder, and the hidden layer is updated as shown in the following formula:

[0099] h i =f encoder (h i-1 ,b i )

[0100] Among them, f encoder represents the encoder function of the autoencoder, b i Represents the semantic input of trajectory points;

[0101] Step 3.2: Decoding;

[0102] The final output of the hidden layer of the Encoder in step 3.1 is h i will represent the entire trajectory sequence and will serve as the initial hidden layer of the decoder LSTM to obtain the output sequence {c1,c2,...,c i }, the decoder’s hidden layer is updated as follows:

[0103] h' i =f decoder (h' i-1 ,c i-1 )

[0104] Among them, f decoder represents the decoder function of the autoencoder, c i-1 represents the output of the decoder;

[0105] Step 3.3: The learning target of the decoder is the input of the encoder. The encoder and decoder pass the minimum error. The error adopts the following function:

[0106]

[0107] The output of the trained encoder can effectively represent the input data, that is, the output information contains the deep feature information of the trajectory sequence.

[0108] Step 4 includes trajectory prediction model encoding training and trajectory prediction model decoding training. The model is divided into two parts: encoder and decoder. During training, the encoder and decoder receive different inputs respectively.

[0109] Specifically, in the trajectory prediction model encoding training, it is assumed that the representation vector of the trajectory sequence is T = (T1 + T2 + ... + T N ), where N is the number of trajectory points, T i For the semantic representation of each trajectory point, the input of the trajectory sequence needs to add a< / s> , set the length of the sequence to F. For trajectory sequences whose length is less than F, use Padding to make their length F. Similarly, add <s> , add at the end< / s> , set the output prediction sequence length of the trajectory to M; including:

[0110] First, the training is carried out in batch mode. The batch size is set to B, and the dimension of the input trajectory sequence is (B, F). The dimensions of trajectory encoding and position encoding are set to be the same, both E. After trajectory embedding and position embedding, the vector dimension is (B, F, E).

[0111] Secondly, adjust the dimension of the vector after embedding from (B, F, E) to (B*F, E), and then use the vector to construct the query, key, and value matrices so that their dimensions are all (B*F, E); then re-transpose the dimensions of the query, key, and value matrices from (B*F, E) to (B, F, E); due to the use of a multi-head attention mechanism, the number of heads is set to N, and N*H=E, then query, key, and value are divided into (B, F, N, H), and the attention scores are calculated based on query, key, and value. The formula is as follows:

[0112]

[0113] Finally, the attention vector is calculated using the scores and value matrix;

[0114] After obtaining the attention vector, the input dimension is (B, F, E), which is input into the fully connected layer. The dimension of the intermediate hidden variable is set to D. The different words in the sequence are multiplied by the weight matrix (E, D), and then multiplied by another weight matrix (D, E), so the final dimension of the matrix is ​​(B, F, E); after passing through the remaining 5 layers of encoders, the final encoder output is (B, F, E).

[0115] In the trajectory prediction model decoding training, the decoder of the training process accepts the input sequence (B, T) and performs trajectory encoding and position encoding on it. The trajectory encoding and position encoding are the same set of weight matrices as those in the encoder, including:

[0116] First, after the input sequence is trajectory encoded and position encoded, the resulting matrix dimension is (B, T, E), which is encoded using the masked multi-head attention mechanism; the masked multi-head attention has the same structure as the self-attention network, except that a mask matrix is ​​added, which is essentially a masking of future information. The input during training is a complete sequence T, and a mask matrix needs to be added to cover up future information.

[0117] Secondly, after the multi-head attention mechanism, the information encoded by the decoder needs to be calculated with the information encoded by the encoder, the key and value vectors are obtained from the information encoded by the encoder, the query vector is constructed from the internal sequence variables of the decoder, and then the attention operation is performed; the decoder repeats the same structure 5 times, and the final output sequence embedding is represented as (B, T, E);

[0118] Then, the (B, T, E) vector output by the decoder is connected to the fully connected layer to the logits space of the region table, becoming (B, T, Z); where Z is the size of the region table; the logits vector is connected to a layer of softmax to obtain the probability, and the predicted sequence is obtained based on the probability. The cross entropy loss function is used to calculate the loss function value for the predicted sequence and the target sequence, and then the parameter optimization process is started.

[0119] Furthermore, the method further includes step 5: verifying the trajectory prediction model, including inputting the trajectory sequence to be predicted into the encoder to obtain an encoder encoding vector (F, E), and then inputting the start marker into the decoder. <s>A single element sequence is used to predict the next trajectory area T1, and then <s> It is concatenated with T1 as the input sequence of the decoder, and the next trajectory area is predicted until the predicted sequence has< / s> until.

[0120] The preferred embodiments of the present invention are described in detail above in conjunction with the accompanying drawings. However, the present invention is not limited to the specific details in the above embodiments. Within the technical concept of the present invention, a variety of simple modifications can be made to the technical solution of the present invention, and these simple modifications all belong to the protection scope of the present invention.

[0121] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, the present invention will not further describe various possible combinations.

[0122] In addition, various embodiments of the present invention may be arbitrarily combined, and as long as they do not violate the concept of the present invention, they should also be regarded as the contents disclosed by the present invention.< / s>

Claims

1. A trajectory prediction method based on semantic fusion representation, characterized in that: The method comprises: Step 1: Use trajectory data preprocessing methods to remove error points and redundant points, and use sliding windows to divide the trajectory data to form a labeled trajectory sequence; Step 2: Combine the latitude and longitude information, time information, speed information and direction information of the vehicle to perform multi-dimensional trajectory fusion vector representation; Step 3: Use the autoencoder to learn the deep features of the trajectory sequence, combine the original features of the trajectory sequence, and jointly construct the semantic representation of the trajectory sequence; Step 4: The transformer-based trajectory prediction method uses a multi-head self-attention mechanism and a mask-based self-attention method to learn the correlation between trajectories and thus achieve trajectory prediction; Step 1 includes: Step 1.1: Error point removal; Let the trajectory point p i and p j The time interval between them is Δt1, and the spatial distance is Δd1; Let the trajectory point p j and p k The time interval between them is Δt2, and the spatial distance is Δd2; Assume that the upper limit of vehicle speed on urban roads is V max ; If the condition is met: Δd1>Δt1*V max And Δd2>Δt2*V max , then determine the trajectory point p j is the error point and should be removed; Step 1.2: Remove redundant points; Let the trajectory point p i 、p j 、p k 、p n The distances between them are Δd1, Δd2 and Δd n , set the circle radius threshold R = 20m, if the conditions Δd1<2*R, Δd2<2*R and Δd n <2*R, it is considered that the position of the vehicle remains basically unchanged during the time period, and the trajectory point p is used as the equivalent point of the redundant point in the area; the longitude and latitude of p are calculated according to the average value in the redundant area; Step 1.3: Trajectory sequence formation; The trajectory is divided based on a sliding window. A sliding window of fixed length is set at the beginning of the trajectory. The last trajectory point in the window is set as the position point to be predicted, and the remaining trajectory points are training features to form a training sample. The window slides forward one position in turn to form a new training sample, which is added to the training feature sequence and label sequence respectively. When the window reaches the last position of the trajectory sequence and extracts the trajectory point, the division ends. Specifically, in the time window T j , trajectory T = {p1, p2, ..., p n }, replace p1, p2, ..., p j-1 As a training feature, p j as the next position label of the trajectory; In step 2, set the multimodal semantic trajectory as Traj(o i )={p1(o i ),p2(o i ),...,p n (o i )}, where p n (o i ) represents the object o i The trajectory semantic information of the nth position, that is, p n (o i )={L n ,T n ,D n ,I n }; where L n Represents object o i At the nth position, the longitude and latitude semantic information, T n Representing temporal semantic information, D n Represents the vehicle speed information, I n Characterize the direction information of the vehicle; The multi-dimensional trajectory fusion vector representation includes: Semantic representation of trajectory latitude and longitude features, including the use of grid division to represent the area formed within a certain latitude and longitude range, in order to capture the semantic information of latitude and longitude features; Assuming the grid is divided into n segments, then L n It can be expressed as an n×1 dimensional vector; at the same time, the design dimension is D l ×n transformation vector E l , L n Convert to D l ×1 vector The formula is as follows: Semantic representation of trajectory temporal features, including characterizing time periods using a grid division method to capture the semantic information of temporal features; Set the grid division to be based on hours as the period, and divide one hour into m segments, then T n It can be expressed as an m×1 dimensional vector; at the same time, the design dimension is D t ×m transformation vector E t , T n Convert to D t ×1 vector The formula is as follows: The semantic representation of vehicle speed information corresponds to the speed information contained in the trajectory information, and the number of discrete values ​​of vehicle speed is set to V. First, the speed information D n Encoded as a V×1 vector, V represents the number of discrete speed values ​​in the current data set; then, use the dimension D d ×V conversion matrix E d D n Convert to The formula is as follows: The semantic representation of vehicle direction information corresponds to the vehicle direction information contained in the trajectory. The number of discrete values ​​of vehicle direction information is set to Q, then I n The dimension vector is Q×1, and the dimension used is D i ×Q conversion matrix E i Will I n Convert to The formula is as follows: At this point, the multi-dimensional trajectory semantics p is calculated n (o i )The formula is as follows: Step 3 includes: Step 3.1: Encoding; The encoder-decoder autoencoder is used to learn the deep features of the trajectory sequence, that is, the obtained multi-dimensional trajectory semantic sequence is input into the encoder part of the autoencoder, and the hidden layer is updated as shown in the following formula: h i =f encoder (h i-1 ,b i ) Among them, f encoder represents the encoder function of the autoencoder, b i Represents the semantic input of trajectory points; Step 3.2: Decoding; The final output of the hidden layer of the Encoder in step 3.1 is h i will represent the entire trajectory sequence and will serve as the initial hidden layer of the decoder LSTM to obtain the output sequence {c1,c2,...,c i }, the hidden layer of the decoder is updated as follows: h' i =f decoder (h' i-1 ,c i-1 ) Among them, f decoder represents the decoder function of the autoencoder, c i-1 represents the output of the decoder; Step 3.3: The learning target of the decoder is the input of the encoder. The encoder and decoder pass the minimum error. The error adopts the following function: The output of the trained encoder can effectively represent the input data, that is, the output information contains the deep feature information of the trajectory sequence.

2. The trajectory prediction method based on semantic fusion representation according to claim 1 is characterized in that: The method further comprises step 5: verifying the trajectory prediction model.

3. The trajectory prediction method based on semantic fusion representation according to claim 1 is characterized in that: Step 4 includes trajectory prediction model encoding training and trajectory prediction model decoding training.

4. The trajectory prediction method based on semantic fusion representation according to claim 3 is characterized in that: In the trajectory prediction model encoding training, it is assumed that the representation vector of the trajectory sequence is T = (T1+T2+...+T N ), where N is the number of trajectory points, T i For the semantic representation of each trajectory point, the input of the trajectory sequence needs to add one at the end of the sequence, and the length of the sequence is set to F. For trajectory sequences that are less than F, the length is made F by padding. Similarly, add <s> , add at the end< / s> , set the output prediction sequence length of the trajectory to M; including: First, the training is carried out in batch mode. The batch size is set to B, and the dimension of the input trajectory sequence is (B, F). The dimensions of trajectory encoding and position encoding are set to be the same, both E. After trajectory embedding and position embedding, the vector dimension is (B, F, E). Secondly, adjust the dimension of the vector after embedding from (B, F, E) to (B*F, E), and then use the vector to construct the query, key, and value matrices so that their dimensions are all (B*F, E); then re-transpose the dimensions of the query, key, and value matrices from (B*F, E) to (B, F, E); due to the use of a multi-head attention mechanism, the number of heads is set to N, and N*H=E, then query, key, and value are divided into (B, F, N, H), and the attention scores are calculated based on query, key, and value. The formula is as follows: Finally, the attention vector is calculated using the scores and value matrix; After obtaining the attention vector, the input dimension is (B, F, E), which is input into the fully connected layer. The dimension of the intermediate hidden variable is set to D. The different words in the sequence are multiplied by the weight matrix (E, D), and then multiplied by another weight matrix (D, E), so the final dimension of the matrix is ​​(B, F, E); after passing through the remaining 5 layers of encoders, the final encoder output is (B, F, E).

5. The trajectory prediction method based on semantic fusion representation according to claim 3 is characterized in that: In the trajectory prediction model decoding training, the decoder of the training process accepts the input sequence (B, T) and performs trajectory encoding and position encoding on it. The trajectory encoding and position encoding are the same set of weight matrices as those in the encoder, including: First, after the input sequence is trajectory encoded and position encoded, the matrix dimension is (B, T, E), which is encoded using the masked multi-head attention mechanism; the masked multi-head attention has the same structure as the self-attention network, but adds a mask matrix to add the mask matrix to the complete sequence T of the input during training to mask future information; Secondly, after the multi-head attention mechanism, the information encoded by the decoder needs to be calculated with the information encoded by the encoder, the key and value vectors are obtained from the information encoded by the encoder, the query vector is constructed from the internal sequence variables of the decoder, and then the attention operation is performed; the decoder repeats the same structure 5 times, and the final output sequence embedding is represented as (B, T, E); Then, the (B, T, E) vector output by the decoder is connected to the fully connected layer to the logits space of the region table, becoming (B, T, Z); where Z is the size of the region table; the logits vector is connected to a layer of softmax to obtain the probability, and the predicted sequence is obtained based on the probability. The cross entropy loss function is used to calculate the loss function value for the predicted sequence and the target sequence, and then the parameter optimization process is started.

6. The trajectory prediction method based on semantic fusion representation according to claim 1 is characterized in that: Step 5 includes: Input the trajectory sequence to be predicted into the encoder to obtain an encoder encoding vector (F, E), and then input the start marker into the decoder <s>A single element sequence is used to predict the next trajectory area T1, and then <s> It is concatenated with T1 as the input sequence of the decoder, and the next trajectory area is predicted until the predicted sequence has< / s> until.< / s>

Images