A vehicle trajectory prediction method and device based on destination retrieval and social attention mechanism
By using a vehicle trajectory prediction method based on destination retrieval and social attention mechanisms, the problem of insufficient accuracy and adaptability in existing trajectory prediction technologies is solved, and a more efficient trajectory prediction effect is achieved.
Patent Information
- Application Number
- CN202210774479.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-07-01
- Publication Date
- 2026-01-06
- Estimated Expiration
- 2042-07-01
AI Technical Summary
Existing vehicle trajectory prediction methods perform poorly in complex dynamic environments and cannot effectively utilize prior knowledge in driving scenarios, resulting in insufficient accuracy and adaptability in trajectory prediction.
A vehicle trajectory prediction method based on destination retrieval and social attention mechanisms is adopted. Multimodal trajectories are matched from an expert database through similarity search, and the trajectory encoder, social attention module and trajectory decoder are combined for fusion encoding and weighted calculation. The multimodal sampling algorithm is used for prediction.
It improves the accuracy and processing efficiency of vehicle trajectory prediction, reduces computational load and time consumption, and enables better utilization of surrounding environmental information for prediction.
Smart Images
Figure CN115329217B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of vehicle trajectory prediction technology, and in particular to a method and apparatus for predicting vehicle trajectories based on destination retrieval and social attention mechanisms. Background Technology
[0002] Autonomous vehicles are a crucial tool for promoting the development of intelligent transportation systems, effectively reducing traffic accidents caused by human error and possessing significant theoretical and practical value. Trajectory prediction is a vital link in autonomous vehicle development, crucial for ensuring their safety. The high-speed, continuous movement of autonomous vehicles and other vehicles in the environment creates significant uncertainty. To avoid collisions while simultaneously meeting comfort and speed requirements, it is essential to predict the behavior of other vehicles to reduce uncertainty. Therefore, accurate prediction of the future trajectories of other vehicles is a prerequisite for autonomous vehicle trajectory planning. Furthermore, fully integrating the prediction results into the trajectory planning cycle to avoid collisions is key to improving the safety of autonomous vehicles.
[0003] In the process of implementing this invention, the inventors of this application discovered that the methods of the prior art have at least the following technical problems:
[0004] Numerous studies have been conducted on trajectory prediction, with commonly used models including: physically constrained prediction models, behavioral intent-based prediction models, and interactive perception-based prediction models. Existing physically constrained prediction models focus only on the constraints of vehicle motion characteristics on the trajectory (dynamic and kinematic constraints), neglecting the influence of prior knowledge such as road structure and traffic rules, as well as posterior knowledge such as the unseen driver's subjective intent. Therefore, these methods cannot predict changes in other factors in the driving scenario that cause the vehicle to perform specific actions (such as deceleration, deceleration and turning at intersections), limiting them to low-speed, short-term predictions. Existing behavioral intent-based prediction models suffer from low scenario adaptability due to insufficient consideration of prior knowledge in the driving scenario during the driving behavior prediction stage. Furthermore, they neglect posterior knowledge in the driving scenario during the trajectory prediction stage, making it difficult to guarantee the accuracy of the obtained trajectory. While existing interactive perception-based models can avoid misclassification of driving behavior, the training data requires manual calibration, increasing the model's training difficulty, and they cannot effectively utilize prior knowledge in the driving scenario.
[0005] Therefore, researching trajectory prediction methods for autonomous vehicles is of great theoretical and practical significance in order to improve the safety and comfort of autonomous vehicles driving in complex dynamic environments. Summary of the Invention
[0006] This invention provides a vehicle trajectory prediction method and apparatus based on destination retrieval and social attention mechanisms, which solves or at least partially solves the technical problem of poor prediction performance in the prior art.
[0007] To address the aforementioned technical problems, the first aspect of this invention provides a vehicle trajectory prediction method based on destination retrieval and social attention mechanisms, comprising:
[0008] S1: Perform a similarity search between the observed historical trajectory and the pre-built expert database, and match multimodal trajectories that meet the preset conditions from the expert database. The matched multimodal trajectories constitute a set of candidate destination locations.
[0009] S2: Construct a vehicle trajectory prediction model, which includes a trajectory encoder, a social attention module, and a trajectory decoder;
[0010] S3: Use a trajectory encoder to fuse and encode the queried destination location and the vehicle's historical trajectory to obtain the encoded features, where the queried destination location comes from a set of candidate destination locations.
[0011] S4: The social attention module is used to combine the encoded features obtained by the trajectory encoder with the influence of the surrounding environment on the vehicle to perform weighted calculations and obtain the predicted features of each vehicle.
[0012] S5: Using the trajectory decoder, the predicted features of each vehicle are used as input to obtain the predicted trajectory features of each vehicle. Then, the multimodal sampling algorithm is used to obtain the final predicted trajectory based on the predicted trajectory features of each vehicle.
[0013] In one implementation, S1 uses the following formula to perform the similarity search:
[0014]
[0015] in It is the distance function between two trajectories, K above the arg min operator. e This represents the number of entries in X that produce the minimum distance, where X is the number of entries in X. i Let X represent the historical trajectory of vehicle i, which is the observed historical trajectory, and let X represent the pre-built expert database. e Let X be an instance, and let S() be a selection function used to select from the nearest K. e The destination location is extracted from each expert instance and used as a candidate destination location in the candidate destination location set, with coordinates as follows:
[0016] In one implementation, the distance function between the two trajectories It is implemented using a dynamic time warping algorithm.
[0017] In one implementation, when performing a similarity search, speed information is concatenated with historical trajectories and instances from a pre-built expert database.
[0018] In one embodiment, the trajectory encoder includes an MLP encoder and an LSTM encoder, and step S3 includes:
[0019] S3.1: Subtract the queried destination location from the historical motion trajectory to obtain the feature embedding. Then, input the feature embedding into the MLP encoder to obtain the shifted code. The specific formula is as follows:
[0020]
[0021]
[0022] Among them, X i This represents the historical trajectory of vehicle i. Let t be the coordinate value of the historical trajectory of vehicle i obtained at observation time t. obs Indicates the end time of the observation. The coordinates of the destination location being queried. For the feature embedding of vehicle i, F i W is the shifted encoding. enc For MLP encoders;
[0023] S3.2: The LSTM encoder performs recursive operations based on the shifted encoding to obtain the encoded sequence of the predicted trajectory of vehicle i, which serves as the encoding feature. The formula for the k-th recursive operation is:
[0024]
[0025] in This indicates the hidden encoder state in the k-th recursion. This indicates the hidden encoder state in the (k-1)th recursion. This represents the shifted encoding used in the k-th recursion.
[0026] In one implementation, step S4 includes: performing N rounds of social pooling based on the output of the trajectory encoder to obtain the predicted features for each vehicle, wherein the (r+1)th round of pooling recursively updates the features from the previous round according to a non-local attention mechanism.
[0027]
[0028] in, The hidden state obtained in round (r+1) is... Let {φ, θ} be the hidden state obtained in the r-th round, i.e., the feature obtained in the r-th round. The encoder is mapped to a learned latency space, in which, at each pooling iteration, an embedded Gaussian is used. Calculate the similarity between the trajectory of vehicle i and the trajectory of vehicle j, M. ij As a social mask, it encodes key information about the social positionality of different trajectories, where g represents... The transform encoder obtains features from each round, which constitute the predicted features for each vehicle.
[0029] In one embodiment, the trajectory decoder includes an LSTM decoder and an MLP decoder, and step S5 includes:
[0030] S5.1: Input the output of the social attention module and the coordinates of the observed end position of the historical trajectory into the LSTM decoder, and generate the output hidden state sequence recursively. The formula for the (k+1)th recursive process is:
[0031]
[0032] in It is the decoder state of the (k+1)th recursive output, h i This represents the predicted features of vehicle i. The coordinates of the end position of the observed historical trajectory. It is the decoder state of the kth recursive output;
[0033] S5.2: Employ an MLP decoder and set the output of the LSTM decoder to the parameters of a binary Gaussian:
[0034]
[0035] Among them W dec It is an MLP decoder, μ x ,μ y ,σ x ,σ y ,corr xy express Projection to a representation of a binary Gaussian The parameters of the 5-dimensional vector;
[0036] S5.3: The softargmax operation is used to estimate the vehicle's trajectory, resulting in a probability matrix. In the probability matrix X, the estimated probability for position (a, b) is X. ab The formula input for the softargmax operation;
[0037]
[0038] in, This means that the elements in matrix X are kept in the same row, and the calculation is performed column by column, with the value of b ranging from 1 to the largest column number. This means that the columns of elements in matrix X remain unchanged, and the calculation is performed row by row, with the value of 'a' ranging from 1 to the largest row number. This means that each element is calculated sequentially;
[0039] S5.4: Use a multimodal sampling algorithm to sample locations with probability values greater than a set probability threshold, and use these as sampling points. Then, use a K-means clustering algorithm to cluster these sampling points, and the resulting cluster centers form the motion vector.
[0040] S5.5: Based on motion vectors Compared with previously predicted coordinates get Then all of them Connecting them in sequence yields the final predicted trajectory.
[0041]
[0042]
[0043]
[0044] Among them, motion vector Represented using binary Gaussian vectors. This represents the coordinates of the trajectory of vehicle i obtained at prediction time k′. In the calculation formula, k′=t in the initial state obs+1 , The value is the coordinate of the last position in the vehicle's historical trajectory.
[0045] Based on the same inventive concept, a second aspect of the present invention provides a vehicle trajectory prediction device based on destination retrieval and social attention mechanisms, comprising:
[0046] The similarity search module is used to perform similarity searches between observed historical trajectories and a pre-built expert database, and to match multimodal trajectories that meet preset conditions from the expert database. The matched multimodal trajectories constitute a set of candidate destination locations.
[0047] The model building module is used to build a vehicle trajectory prediction model, which includes a trajectory encoder, a social attention module, and a trajectory decoder.
[0048] The fusion coding module is used to fuse and encode the queried destination location and the vehicle's historical trajectory using the trajectory encoder to obtain coded features, wherein the queried destination location comes from a set of candidate destination locations.
[0049] The environment coding module is used to perform weighted calculations on the features encoded by the social attention module combined with the trajectory encoder and the influence of the surrounding environment on the vehicle to obtain the predicted features of each vehicle.
[0050] The trajectory prediction module uses the trajectory decoder to obtain the predicted trajectory features of each vehicle by taking the predicted features of each vehicle as input, and then uses a multimodal sampling algorithm to obtain the final predicted trajectory based on the predicted trajectory features of each vehicle.
[0051] Based on the same inventive concept, a third aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed, implements the method described in the first aspect.
[0052] Based on the same inventive concept, a fourth aspect of the present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method described in the first aspect.
[0053] Compared with the prior art, the advantages and beneficial technical effects of the present invention are as follows:
[0054] This invention provides a vehicle trajectory prediction method based on destination retrieval and social attention mechanisms. It performs a similarity search among instances in an expert database on observed historical trajectories, retrieving multimodal trajectories with high similarity to obtain a small set of candidate destination locations, which are then used as potential destinations for complete trajectory prediction. Since this method is non-parametric, it does not require additional learning, reducing the computational load and time consumption of destination inference. Furthermore, a trajectory encoder is used to fuse the queried destination location and the vehicle's historical trajectory. This fusion of the retrieved destination and historical trajectory information provides effective guidance for prediction, resulting in more accurate predictions. In addition, this invention employs a social attention mechanism to describe the degree of influence of different surrounding vehicle movements on the future trajectory of the destination vehicle, filtering out more important social information for the destination vehicle, allowing the decoder to focus on this more influential social information. Moreover, when processing large amounts of input information, the neural network model, utilizing the attention mechanism, can select only key input information for processing, thereby improving the processing efficiency of the vehicle trajectory prediction model. Attached Figure Description
[0055] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0056] Figure 1 This is a technical roadmap for the vehicle trajectory prediction method based on destination retrieval and social attention mechanisms provided in the embodiments of the present invention.
[0057] Figure 2 This is an algorithmic framework diagram of the LSTM encoder-decoder architecture used in this embodiment of the invention to predict the future trajectory of traffic vehicles.
[0058] Figure 3 This is a network structure diagram of the social attention module of this invention. Detailed Implementation
[0059] Existing methods often struggle to infer the destination when predicting trajectories because they only consider the current vehicle's location and historical status, without taking into account the influence of other vehicles (the surrounding environment).
[0060] To address the technical problem of low prediction accuracy in existing technologies, the main concept of this invention is as follows:
[0061] First, a destination retrieval algorithm is designed. Observed historical trajectories are searched for similarity among instances in an expert database to retrieve multimodal trajectories with high similarity, thus obtaining a small set of candidate destination locations, which are used as potential destinations for complete trajectory prediction. Since this method is non-parametric, it does not require additional learning, reducing the computational load and time consumption of destination inference. Furthermore, the retrieved destinations are fused with historical trajectories; this fusion provides effective guidance for prediction, resulting in more accurate predictions. Further, this invention employs a social attention mechanism to describe the influence of different surrounding vehicle movements on the future trajectory of the destination vehicle, filtering out more important social information for the destination vehicle, allowing the decoder to focus on this more influential social information. Moreover, when processing large amounts of input information, the neural network model, utilizing the attention mechanism, can select only key input information for processing, thereby improving the efficiency of the neural network. In addition, a trajectory decoder combined with a multimodal sampling algorithm is used for trajectory prediction.
[0062] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0063] Example 1
[0064] This invention provides a vehicle trajectory prediction method based on destination retrieval and social attention mechanisms, comprising:
[0065] S1: Perform a similarity search between the observed historical trajectory and the pre-built expert database, and match multimodal trajectories that meet the preset conditions from the expert database. The matched multimodal trajectories constitute a set of candidate destination locations.
[0066] S2: Construct a vehicle trajectory prediction model, which includes a trajectory encoder, a social attention module, and a trajectory decoder;
[0067] S3: Use a trajectory encoder to fuse and encode the queried destination location and the vehicle's historical trajectory to obtain the encoded features, where the queried destination location comes from a set of candidate destination locations.
[0068] S4: The social attention module is used to combine the encoded features obtained by the trajectory encoder with the influence of the surrounding environment on the vehicle to perform weighted calculations and obtain the predicted features of each vehicle.
[0069] S5: Using the trajectory decoder, the predicted features of each vehicle are used as input to obtain the predicted trajectory features of each vehicle. Then, the multimodal sampling algorithm is used to obtain the final predicted trajectory based on the predicted trajectory features of each vehicle.
[0070] Please see Figure 1 This is a technical roadmap for the vehicle trajectory prediction method based on destination retrieval and social attention mechanisms provided in the embodiments of the present invention.
[0071] Specifically, meeting the preset conditions means that the similarity between the historical trajectory and the trajectory in the expert database is greater than a threshold. The trajectory encoder is mainly used to fuse the destination location with the historical trajectory, while the social attention module mainly encodes the influence of the surrounding environment.
[0072] In one implementation, S1 uses the following formula to perform the similarity search:
[0073]
[0074] in It is the distance function between two trajectories, K above the arg min operator. e This represents the number of entries in X that produce the minimum distance, where X is the number of entries in X. i Let X represent the historical trajectory of vehicle i, which is the observed historical trajectory, and let X represent the pre-built expert database. e Let X be an instance, and let S() be a selection function used to select from the nearest K. e The destination location is extracted from each expert instance and used as a candidate destination location in the candidate destination location set, with coordinates as follows:
[0075] Specifically, each instance in the constructed expert database is also a trajectory, and S() is a selection function that selects the associated end position in the trajectory, i.e., the coordinates of the predicted destination.
[0076] In one implementation, the distance function between the two trajectories It is implemented using a dynamic time warping algorithm.
[0077] Specifically, DTW is a well-established method for measuring the distance between time series. The matching function γ-Soft-DTW is defined as follows:
[0078]
[0079] Where Δ(·) is the distance matrix for measuring the adjacency of elements, A represents the alignment matrix for matching selection, and the inner product operator <> produces the minimum similarity score. γ γ≥0 is defined as:
[0080]
[0081] Where a i This represents an entry in the distance matrix, where γ is a smoothing factor whose value is set empirically.
[0082] In one implementation, when performing a similarity search, speed information is concatenated with historical trajectories and instances from a pre-built expert database.
[0083] To achieve better matching, their motion information is concatenated as velocity (V). i V e To enrich the trajectory descriptor, i.e. The parameter is (cat(X) i V i ),cat(X e V e Therefore, similarity takes into account not only geographical location, but also speed and direction.
[0084] In one embodiment, the trajectory encoder includes an MLP encoder and an LSTM encoder, and step S3 includes:
[0085] S3.1: Subtract the queried destination location from the historical motion trajectory to obtain the feature embedding. Then, input the feature embedding into the MLP encoder to obtain the shifted code. The specific formula is as follows:
[0086]
[0087]
[0088] Among them, X i This represents the historical trajectory of vehicle i. Let t be the coordinate value of the historical trajectory of vehicle i obtained at observation time t. obs Indicates the end time of the observation. The coordinates of the destination location being queried. For the feature embedding of vehicle i, F i W is the shifted encoding. enc For MLP encoders;
[0089] S3.2: The LSTM encoder performs recursive operations based on the shifted encoding to obtain the encoded sequence of the predicted trajectory of vehicle i, which serves as the encoding feature. The formula for the k-th recursive operation is:
[0090]
[0091] in This indicates the hidden encoder state in the k-th recursion. This indicates the hidden encoder state in the (k-1)th recursion. This represents the shifted encoding used in the k-th recursion.
[0092] Specifically, connecting the destination location to the historical trajectory in the high-dimensional feature space would lead to additional embedding work. Therefore, this implementation subtracts the query destination location value from the historical trajectory, thus incorporating the destination information into the feature embedding.
[0093] F i It relates to the high-dimensional features projected from the two-dimensional coordinates of each timestamp, i.e. and W enc It is implemented using MLP.
[0094] In order to obtain the predicted trajectory of vehicle i Given an input embedding sequence F iLong Short-Term Memory (LSTM) is used as the unit of the seq2seq generator. The hidden state sequence is generated by encoding and decoding the embedded sequence sequentially. Each recursive step yields an intermediate hidden state, which serves as the input for the next recursive step.
[0095] The intermediate hidden states obtained in each recursive step include the hidden encoder states in the (k-1)th recursion and the (k-2)th recursion. These intermediate hidden states serve not only as input to the next recursive step but also as the output of the LSTM encoder. Initial hidden state h o It was sampled from a normal distribution.
[0096] In one implementation, step S4 includes: performing N rounds of social pooling based on the output of the trajectory encoder to obtain the predicted features for each vehicle, wherein the (r+1)th round of pooling recursively updates the features from the previous round according to a non-local attention mechanism.
[0097]
[0098] in, The hidden state obtained in round (r+1) is... Let {φ, θ} be the hidden state obtained in the r-th round, i.e., the feature obtained in the r-th round. The encoder is mapped to a learned latency space, in which, at each pooling iteration, an embedded Gaussian is used. Calculate the similarity between the trajectory of vehicle i and the trajectory of vehicle j, M. ij As a social mask, it encodes key information about the social positionality of different trajectories, where g represents... The transform encoder obtains features from each round, which constitute the predicted features for each vehicle.
[0099] Please see Figure 3 , is the network structure diagram of the social attention module of this invention.
[0100] Specifically, the social mask is defined as follows:
[0101]
[0102] M is an α*α block diagonal matrix. This formula defines the spatiotemporal neighborhood of each vehicle. When the spatial distance between vehicles is greater than dist, they are considered non-neighbors. This represents the actual number of frames within the observation period, ensuring a consistent number of frames per time interval and guaranteeing temporal overlap. Matrix M encodes key information about the social locationality of different trajectories, which is utilized in attention-based pooling. This represents the position of vehicle i at time m. This represents the position of vehicle j at time n, where dist is a set distance threshold. This represents the number of observation frames for vehicle i at time m. This represents the number of observation frames for vehicle j at the initial moment.
[0103] After calculating the similarity between the trajectories of vehicle i and vehicle j using embedded Gaussian trajectories, a dot product M is performed, allowing pooling only on the spatiotemporal neighbors of the vehicles. g is h i The transform encoder weights the influence of all other neighbors. After repeating the entire process N times, it produces predicted features for each car, which contain information about the past locations and future destinations of all other neighbors in the scene. The hidden state obtained in round (r+1) is... Let be the hidden state obtained in the r-th round. The hidden states obtained in each round constitute the predicted features for each vehicle. For example, the predicted features for vehicle i are:
[0104] In one embodiment, the trajectory decoder includes an LSTM decoder and an MLP decoder, and step S5 includes:
[0105] S5.1: Input the output of the social attention module and the coordinates of the observed end position of the historical trajectory into the LSTM decoder, and generate the output hidden state sequence recursively. The formula for the (k+1)th recursive process is:
[0106]
[0107] in It is the decoder state of the (k+1)th recursive output, h i This represents the predicted features of vehicle i. The coordinates of the end position of the observed historical trajectory. It is the decoder state of the kth recursive output;
[0108] S5.2: Employ an MLP decoder and set the output of the LSTM decoder to the parameters of a binary Gaussian:
[0109]
[0110] Among them W dec It is an MLP decoder, μ x ,μ y ,σ x ,σ y ,corr xy express Projection to a representation of a binary Gaussian The parameters of the 5-dimensional vector;
[0111] S5.3: The softargmax operation is used to estimate the vehicle's trajectory, resulting in a probability matrix. In the probability matrix X, the estimated probability for position (a, b) is X. ab The formula input for the softargmax operation;
[0112]
[0113] in, This means that the elements in matrix X are kept in the same row, and the calculation is performed column by column, with the value of b ranging from 1 to the largest column number. This means that the columns of elements in matrix X remain unchanged, and the calculation is performed row by row, with the value of 'a' ranging from 1 to the largest row number. This means that each element is calculated sequentially;
[0114] S5.4: Use a multimodal sampling algorithm to sample locations with probability values greater than a set probability threshold, and use these as sampling points. Then, use a K-means clustering algorithm to cluster these sampling points, and the resulting cluster centers form the motion vector.
[0115] S5.5: Based on motion vectors Compared with previously predicted coordinates get Then all of them Connecting them in sequence yields the final predicted trajectory.
[0116]
[0117]
[0118]
[0119] Among them, motion vector Represented using binary Gaussian vectors. This represents the coordinates of the trajectory of vehicle i obtained at prediction time k′. In the calculation formula, k′=t in the initial state obs+1 , The value is the coordinate of the last position in the vehicle's historical trajectory.
[0120] Please see Figure 2 This is an algorithm framework diagram of the LSTM encoder-decoder architecture used in this embodiment of the invention to predict the future trajectory of traffic vehicles.
[0121] Specifically, the decoder obtained from each recursion serves as both the input and the output for the next recursion, ultimately yielding the predicted trajectory features for each vehicle.
[0122] To perform multimodal prediction, an MLP decoder is used. The output of the LSTM decoder is set to the parameters of a binary Gaussian.
[0123] Sampling is performed in the distribution. To balance sample diversity and accuracy, a softargmax operation is used to estimate the most probable location in a robust and stable manner. Specifically, the softargmax operation is used to estimate the vehicle's trajectory, resulting in a probability matrix X. In the probability matrix X, for location (a, b), the probability X is given. ab This is the estimated probability. Then, based on the relationship between the estimated probability value and the probability threshold, location sampling is performed. The probability threshold can be obtained empirically. The calculation process is an iterative process. In the initial state, That is, the last coordinate point of the vehicle's historical trajectory, k′∈(t obs+1 ,t end ), indicating that k′ belongs to this prediction interval.
[0124] Example 2
[0125] Based on the same inventive concept, this embodiment provides a vehicle trajectory prediction device based on destination retrieval and social attention mechanisms, comprising:
[0126] The similarity search module is used to perform similarity searches between observed historical trajectories and a pre-built expert database, and to match multimodal trajectories that meet preset conditions from the expert database. The matched multimodal trajectories constitute a set of candidate destination locations.
[0127] The model building module is used to build a vehicle trajectory prediction model, which includes a trajectory encoder, a social attention module, and a trajectory decoder.
[0128] The fusion coding module is used to fuse and encode the queried destination location and the vehicle's historical trajectory using the trajectory encoder to obtain coded features, wherein the queried destination location comes from a set of candidate destination locations.
[0129] The environment coding module is used to perform weighted calculations on the features encoded by the social attention module combined with the trajectory encoder and the influence of the surrounding environment on the vehicle to obtain the predicted features of each vehicle.
[0130] The trajectory prediction module uses the trajectory decoder to obtain the predicted trajectory features of each vehicle by taking the predicted features of each vehicle as input, and then uses a multimodal sampling algorithm to obtain the final predicted trajectory based on the predicted trajectory features of each vehicle.
[0131] Since the device described in Embodiment 2 of this invention is the same device used to implement the vehicle trajectory prediction method based on destination retrieval and social attention mechanisms in Embodiment 1 of this invention, those skilled in the art can understand the specific structure and variations of this device based on the method described in Embodiment 1 of this invention, and therefore will not be repeated here. All devices used in the method of Embodiment 1 of this invention fall within the scope of protection of this invention.
[0132] Example 3
[0133] Based on the same inventive concept, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed, implements the method described in Embodiment 1.
[0134] Since the computer-readable storage medium described in Embodiment 3 of this invention is the same computer-readable storage medium used in implementing the vehicle trajectory prediction method based on destination retrieval and social attention mechanisms in Embodiment 1 of this invention, those skilled in the art can understand the specific structure and variations of this computer-readable storage medium based on the method described in Embodiment 1 of this invention, and therefore will not be repeated here. All computer-readable storage media used in the method of Embodiment 1 of this invention fall within the scope of protection of this invention.
[0135] Example 4
[0136] Based on the same inventive concept, this application also provides a computer device, including storage, a processor, and a computer program stored in the storage and executable on the processor, wherein the processor executes the program to implement the method in Embodiment 1.
[0137] Since the computer device described in Embodiment 4 of this invention is the same computer device used to implement the vehicle trajectory prediction method based on destination retrieval and social attention mechanisms in Embodiment 1 of this invention, those skilled in the art can understand the specific structure and variations of this computer device based on the method described in Embodiment 1 of this invention, and therefore will not be repeated here. All computer devices used in the method of Embodiment 1 of this invention fall within the scope of protection of this invention.
[0138] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0139] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0140] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the invention.
[0141] Obviously, those skilled in the art can make various modifications and variations to the embodiments of the present invention without departing from the spirit and scope of the embodiments of the present invention. Thus, if these modifications and variations to the embodiments of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention also intends to include these modifications and variations.
Claims
1. A method for vehicle trajectory prediction based on destination retrieval and social attention mechanism, characterized in that, Comprise: S1: similarity search is carried out between the observed historical trajectory and the pre-constructed expert library, and the multimodal trajectory meeting the preset condition is matched from the expert library, and the matched multimodal trajectory constitutes a candidate destination position set; S2: a vehicle trajectory prediction model is constructed, which comprises a trajectory encoder, a social attention module and a trajectory decoder; S3: the trajectory encoder is used for fusion coding of the queried destination position and the vehicle historical trajectory, and the coded feature is obtained, wherein the queried destination position is from the candidate destination position set; S4: the social attention module is used for weighted calculation of the influence of the coded feature obtained by the trajectory encoder and the surrounding environment on the vehicle, and the prediction feature of each vehicle is obtained; S5: the trajectory decoder is used as input for the prediction feature of each vehicle, and the prediction trajectory feature of each vehicle is obtained, and then the final prediction trajectory is obtained according to the prediction trajectory feature of each vehicle through a multimodal sampling algorithm; Wherein, the trajectory encoder comprises an MLP encoder and an LSTM encoder, and step S3 comprises: S3.1: subtract the queried destination position from the historical motion trajectory to obtain a feature embedding, and then input the feature embedding into the MLP encoder to obtain a shifted coding, and the specific formula is: wherein, represents a historical trajectory of vehicle i, is a coordinate value of the historical trajectory of vehicle i at observation time t, represents an end time of the observation, is a coordinate of a destination location of the query, is a feature embedding of vehicle i, is an encoded shifted, is an MLP encoder; S3.2: the LSTM encoder performs recursive operation according to the shifted coding to obtain the coding sequence of the prediction trajectory of vehicle i as the coded feature, wherein the formula of the kth recursive operation is: wherein denotes the hidden encoder state in the kth recursion, denotes the hidden encoder state in the k-1th recursion, denotes the shifted encoding employed in the kth recursion. 2.The vehicle trajectory prediction method based on destination retrieval and social attention mechanism according to claim 1, wherein, S1 uses the following formula to realize similarity search: where is a distance function between two trajectories, above the operator denotes returning the number of entries in is the historical trajectory of vehicle i as observed, denotes a pre-constructed expert library, denotes an instance in () is a selection function that takes out the destination location from the closest expert instances as a candidate destination location in the candidate destination location set, with coordinates . 3.The vehicle trajectory prediction method based on destination retrieval and social attention mechanism according to claim 2, wherein, Distance function between two trajectories Implemented using dynamic time warping algorithm. 4.The destination-based retrieval and social attention mechanism based vehicle trajectory prediction method of claim 1, wherein, When similarity search is performed, the historical trajectory and the instances in the pre-constructed expert library are spliced with speed information. 5.The destination-based retrieval and social attention mechanism based vehicle trajectory prediction method of claim 1, wherein, Step S4 comprises: performing N rounds of social pooling according to the output of the trajectory encoder to obtain the predicted features of each vehicle, wherein the r+1th round of pooling recursively updates the features of the previous round according to a non-local attention mechanism : in, The hidden state obtained in round (r+1) is Let be the hidden state obtained in round r, i.e., the feature obtained in round r. yes The encoder is mapped to a learned latency space, in which, at each pooling iteration, an embedded Gaussian is used. Calculate the similarity between the trajectory of vehicle i and the trajectory of vehicle j. As a social mask, it encodes key information about the social positionality of different trajectories. express The transform encoder obtains features from each round, which constitute the predicted features for each vehicle. 6.The destination-based retrieval and social attention mechanism based vehicle trajectory prediction method of claim 1, wherein, The trajectory decoder comprises an LSTM decoder and an MLP decoder, and step S5 comprises: S5.1: the output of the social attention module and the coordinates of the observed historical trajectory end position are input into the LSTM decoder, and the output hidden state sequence is generated according to the recursive mode, wherein the formula of the k+1th recursive process is: wherein is the decoder state of the k+1th recursion output, denotes the predicted feature of vehicle i, is the coordinate of the end position of the observed historical trajectory, is the decoder state of the kth recursion output; S5.2: the output of the LSTM decoder is set as the parameters of the binary Gaussian by using the MLP decoder: wherein is an MLP decoder, denotes projected to the parameters of a binary Gaussian 5-dimensional vector. S5.3: Adopting The operation estimates the trajectory of the vehicle, obtaining a probability matrix, in which, for a position The estimated probability for a position is , The operation of the formula input; wherein, represents the element-wise product of the matrices is computed column-wise with the elements of the row invariant, b taking values from 1 to the maximum number of columns, represents the element-wise product of the matrices is computed row-wise with the elements of the column invariant, a taking values from 1 to the maximum number of rows, represents the element-wise product of the matrices S5.4: sample the positions with probability value greater than a set probability threshold as sampling points using a multi-modal sampling algorithm, cluster the sampling points using a K-means clustering algorithm, and obtain the cluster centers as the motion vectors ; S5.5: according to the motion vector with the previously predicted coordinate points obtained all sequentially connected to obtain the final predicted trajectory : Among them, motion vector Represented using binary Gaussian vectors. Indicates the predicted time The coordinates of the trajectory of vehicle i obtained at that time. In the calculation formula, the initial state is... = , , The value is the coordinate of the last position in the vehicle's historical trajectory.
7. A destination-based retrieval and social attention mechanism based vehicle trajectory prediction apparatus, characterized in that, Comprise: The similarity search module is used for similarity search between the observed historical trajectory and the pre-constructed expert library, and the multimodal trajectory meeting the preset condition is matched from the expert library, and the matched multimodal trajectory constitutes a candidate destination position set; The model construction module is used for constructing a vehicle trajectory prediction model, which comprises a trajectory encoder, a social attention module and a trajectory decoder; The fusion coding module is used for fusion coding of the queried destination position and the vehicle historical trajectory by the trajectory encoder, and the coded feature is obtained, wherein the queried destination position is from the candidate destination position set; The environment coding module is used for weighted calculation of the influence of the coded feature obtained by the trajectory encoder and the surrounding environment on the vehicle by the social attention module, and the prediction feature of each vehicle is obtained; The trajectory prediction module is used for taking the prediction feature of each vehicle as input by the trajectory decoder to obtain the prediction trajectory feature of each vehicle, and then the final prediction trajectory is obtained according to the prediction trajectory feature of each vehicle through a multimodal sampling algorithm. The trajectory encoder comprises an MLP encoder and an LSTM encoder, and the fusion encoding module is specifically configured to: The historical motion trajectory is subtracted from the queried destination position to obtain a feature embedding, and the feature embedding is input into the MLP encoder to obtain a shifted encoding, and a specific formula is as follows: wherein, represents a historical trajectory of vehicle i, is a coordinate value of the historical trajectory of vehicle i obtained at observation time t, represents an end time of observation, is a coordinate of a destination location of the query, is a feature embedding of vehicle i, is an encoded shifted, is an MLP encoder; The LSTM encoder performs recursive operation according to the shifted encoding to obtain an encoding sequence of the predicted trajectory of the vehicle i as an encoded feature, and a formula of kth recursive operation is as follows: wherein denotes the hidden encoder state in the kth recursion, denotes the hidden encoder state in the k-1th recursion, denotes the shifted encoding employed in the kth recursion.
8. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program is executed to implement the method according to any one of claims 1 to 6.
9. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor implements the method according to any one of claims 1 to 6 when executing the program.