A multimodal vehicle trajectory prediction method
By processing vehicle trajectories in the candidate lane centerline coordinate system and utilizing an LSTM encoder-decoder network, combined with interaction relationship modeling and one-hot encoding, the problem of insufficient long-term prediction accuracy and difficulty in data labeling in existing technologies is solved, and multimodal vehicle trajectory prediction in complex scenarios is achieved, with strong scenario adaptability and accuracy.
Patent Information
- Application Number
- CN202111528362.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-14
- Publication Date
- 2025-09-05
- Estimated Expiration
- 2041-12-14
AI Technical Summary
Existing technologies in trajectory prediction have the following problems: insufficient long-term prediction accuracy, the need for manually labeled data sets, and the inability to fully describe the multimodal characteristics of vehicle motion. In particular, it is difficult to achieve accurate multimodal trajectory prediction in complex scenarios.
A self-supervised learning method is adopted to establish a candidate lane centerline coordinate system, process the vehicle's historical motion trajectory into the trajectory under the candidate lane centerline, and use the LSTM encoder-decoder network for autonomous training. Combined with interaction relationship modeling and one-hot encoding, multimodal vehicle trajectory prediction is achieved without the need for labeled datasets and sampling processes.
It realizes long-term multimodal vehicle trajectory prediction in complex scenarios, has strong scene generalization ability, can autonomously learn the multimodal characteristics of vehicle motion, and provide comprehensive and accurate predicted trajectory distribution.
Smart Images
Figure CN114372570B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of autonomous driving technology, and in particular to a multimodal vehicle trajectory prediction method. Background Art
[0002] In recent years, the development boom of artificial intelligence has penetrated into all walks of life. As the largest application field of artificial intelligence, autonomous driving has achieved unprecedented development and progress.
[0003] However, the industry remains somewhat pessimistic about achieving fully autonomous driving in complex scenarios. One key reason is that trajectory prediction in dynamic scenarios remains a challenge. In fact, practitioners have long pointed out that trajectory prediction is one of the most challenging issues in achieving autonomous driving.
[0004] Self-driving cars on the road need to make autonomous decisions and plan their routes based on current traffic conditions. To operate safely and efficiently, they must be able to accurately understand and predict the behavior of surrounding traffic participants before making decisions and planning.
[0005] The prediction module is the key link between the perception module and the decision-making and planning module. Without this prerequisite, even the most perfect perception and decision-making planning will not be safe and efficient in actual autonomous driving applications. In recent years, research on trajectory prediction has received increasing attention, and a growing number of researchers have devoted themselves to related research. However, trajectory prediction remains a challenging task.
[0006] Trajectory prediction methods fall into three main categories. Physical model-based trajectory prediction offers relatively accurate short-term predictions, but its accuracy significantly decreases over longer periods. Machine learning-based trajectory prediction is less effective for longer-term predictions and often requires the incorporation of hand-crafted features. It can only learn explicit information, failing to capture deeper implicit information, which significantly limits its predictive capabilities. Deep learning-based trajectory prediction, while requiring extensive data, can capture deeper implicit information, thereby improving trajectory prediction accuracy.
[0007] Different traffic participants on the road may make completely different behavioral decisions under the same conditions, that is, they may manifest as different motion modes, so the prediction of these behaviors should also be uncertain. Implementing multimodal trajectory prediction is a challenging but very important task. Different multimodal trajectory prediction methods are divided into two categories based on whether or not the modalities need to be predefined. Methods that require predefined modalities require manual labeling of the different modalities in the dataset. On the one hand, this is very time-consuming and labor-intensive; on the other hand, the correctness of the labeling cannot be evaluated by the true value, and unreasonable labeling will have a negative impact on the prediction network during the training process. Methods that do not require predefined modalities do not require manual labeling of the different modalities in the dataset, but the trajectory distribution obtained by sampling is difficult to guarantee that all possible motion modes are covered. It may happen that high-probability modalities are repeatedly sampled, while low-probability modalities cannot be sampled. Summary of the Invention
[0008] The purpose of the present invention is to overcome the defects of the above-mentioned existing technologies and provide a multimodal vehicle trajectory prediction method. This method can self-supervise the multimodal characteristics of vehicle motion without the need for labeled data sets and sampling processes, comprehensively describe the predicted trajectory distribution of vehicle motion, and perform long-term predictions; and has strong scene generalization capabilities. It is not only applicable to high-speed scenes with fixed road structures, but also effective for complex urban road scenes.
[0009] The purpose of the present invention can be achieved by the following technical solutions:
[0010] The present invention provides a multimodal vehicle trajectory prediction method, comprising the following steps:
[0011] S1: Based on the road map, the historical motion trajectories of the predicted vehicle and multiple surrounding vehicles in the world coordinate system are obtained respectively, and processed into the historical motion trajectories of the predicted vehicle and each surrounding vehicle in the candidate lane centerline coordinate system;
[0012] S2: Encode the historical motion trajectories of the predicted vehicle and each surrounding vehicle in the candidate lane centerline coordinate system, obtain a historical trajectory feature vector of the predicted vehicle and a historical trajectory feature vector of each surrounding vehicle, and obtain an interaction relationship feature vector of the predicted vehicle and the surrounding vehicles at the current moment based on the historical trajectory feature vector of the predicted vehicle and the historical trajectory feature vector of each surrounding vehicle;
[0013] S3: Sampling the centerline of the candidate lane to obtain the intention modal code corresponding to the historical trajectory feature vector of the predicted vehicle, and using one-hot encoding to obtain the motion modal code of the predicted vehicle at different speeds;
[0014] S4: The intention modality encoding, motion modality encoding, historical trajectory feature vector of the predicted vehicle, and feature vector of the interaction relationship between the current predicted vehicle and surrounding vehicles are concatenated as input to the LSTM decoder. The LSTM decoder decodes and outputs multiple predicted trajectories of the predicted vehicle and corresponding confidence labels, and selects a predicted trajectory that best matches the true trajectory value through a trained neural network.
[0015] Preferably, the S1 comprises the following steps:
[0016] S1.1: Obtain the historical motion trajectories of the predicted vehicle and multiple surrounding vehicles in the world coordinate system based on the road map;
[0017] S1.2: Use a breadth-first approach based on the Manhattan distance to obtain the candidate lane centerline for the predicted vehicle. Project each trajectory point from the predicted vehicle and surrounding vehicles' historical motion trajectories in the world coordinate system onto the candidate lane centerline. Find the point on the candidate lane centerline closest to each trajectory point as a matching point.
[0018] S1.3: Establish a candidate lane centerline coordinate system based on the candidate lane centerline and the matching points, obtain the coordinates of each trajectory point in the candidate lane centerline coordinate system, and then obtain the historical motion trajectory of the predicted vehicle and each surrounding vehicle in the candidate lane centerline coordinate system.
[0019] Preferably, the candidate lane centerline coordinate system is specifically: the direction along the candidate lane centerline is the y-axis, the direction perpendicular to the candidate lane centerline is the x-axis, and the matching point corresponding to the trajectory point of the vehicle predicted at the current moment on the candidate lane centerline is the coordinate origin.
[0020] Preferably, the coordinates of each trajectory point in the candidate lane centerline coordinate system are: the value of x is the distance between the trajectory point and the matching point, and the positive direction of the x-axis is to the right of the forward direction of the candidate lane centerline; the value of y is the distance between the matching point corresponding to the trajectory point and the coordinate origin along the candidate lane centerline, and the positive direction of the y-axis is the forward direction along the candidate lane centerline.
[0021] Preferably, said S2 comprises the following steps:
[0022] S2.1: Encode the historical motion trajectories of the predicted vehicle and each surrounding vehicle in the candidate lane centerline coordinate system, respectively, to obtain a historical trajectory feature vector of the predicted vehicle and a historical trajectory feature vector of each surrounding vehicle;
[0023] S2.2: Integrate the historical trajectory feature vectors of the surrounding vehicles to obtain the historical trajectory feature vectors of the surrounding vehicles, and obtain the interaction relationship feature vector between the predicted vehicle and the surrounding vehicles at the current moment based on the historical trajectory feature vector of the predicted vehicle and the historical trajectory feature vectors of the surrounding vehicles through the interaction model.
[0024] Preferably, the formula for encoding the historical trajectory at each moment to obtain the historical trajectory feature vector at the current moment is:
[0025]
[0026] Where, φ e (· is a fully connected layer used to preliminarily extract historical trajectory features, W1 is its corresponding parameter, S t is the location coordinate information of the historical trajectory at time t; LSTM e (·) is the LSTM encoder, and W2 is its corresponding parameter; The input of the LSTM encoder in the LSTMencoder-decoder benchmark vehicle trajectory prediction network is the historical motion trajectory feature vector at the current moment; and are the hidden state and memory cell state in the LSTM encoder at time t, and are the hidden state and memory cell state in the LSTM encoder at time t-1, respectively.
[0027] Preferably, the formula for calculating the interaction model is specifically:
[0028]
[0029] Where, φ r is the interaction model, W r For its corresponding parameters, is the prediction of the interaction model input, r obs Predict the feature vector of the interaction relationship between the vehicle and its surrounding vehicles at the current moment.
[0030] Preferably, the process of obtaining the intention modal code is specifically as follows: taking the matching point of the vehicle predicted on the center line of the candidate lane at the current moment as the starting point, sampling at fixed-length intervals, forming a vector with the coordinates of all sampling points relative to the starting point, and passing the vector through a multi-layer perceptron to obtain the intention modal code.
[0031] Preferably, the input of the LSTM decoder is:
[0032]
[0033] Where, ψ line(M) is the set of candidate lane centerlines; v line is a vector consisting of n points sampled from the center line of a specific candidate lane; MLP is a multi-layer perceptron, W MLP Its corresponding parameter; τ line To predict the intention modal encoding corresponding to the historical motion trajectory feature vector of the vehicle, Predict the historical motion trajectory feature vector of the vehicle at all times, r obs is the feature vector of the interaction relationship between the current predicted vehicle and surrounding vehicles, ψ m (N) is a set of N one-hot encodings, τ m To predict the vehicle's motion mode encoding, is the input feature vector of the LSTM decoder.
[0034] Preferably, an arbitration method is used to select a predicted trajectory that best matches the true trajectory value from all the predicted trajectories of the outputted predicted vehicles. The arbitration method is specifically as follows:
[0035] When determining the intention mode, all candidate lane centerlines are sorted from most to least based on the number of points whose distances between the predicted future true trajectory of the vehicle and the candidate lane centerlines are within the Manhattan threshold range. The candidate lane centerline with the largest number of true trajectory points within the threshold range is selected as the current intention mode. When determining the motion mode, under the current intention mode, the predicted trajectory that best matches the true trajectory is determined based on the principle of minimizing the average position deviation.
[0036] Compared with the prior art, the present invention has the following advantages:
[0037] 1. The present invention customizes the candidate lane centerline coordinate system and processes the vehicle's historical motion trajectory in the world coordinate system into the vehicle's historical motion trajectory in the candidate lane centerline coordinate system. Only the coordinates in the candidate lane centerline coordinate system need to be processed, avoiding the tedious convolution processing on the world map using different convolutional networks in traditional methods, saving time and effort.
[0038] 2. By processing the historical motion trajectory of the vehicle in the world coordinate system into the historical motion trajectory of the vehicle in the candidate lane centerline coordinate system, the present invention has a strong scene generalization ability. It is not only applicable to high-speed scenes with fixed road structures, but also effective for complex urban road scenes.
[0039] 3. The present invention utilizes an LSTM encoder-decoder network, inputs the codes of each modality, and allows the network to train autonomously. This does not require pre-definition of modalities, labeling of data sets, or sampling processes. It can self-supervise and learn the multimodal characteristics of vehicle motion, comprehensively describe the predicted trajectory distribution of vehicle motion, and perform long-term predictions. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] Figure 1 Schematic diagram of a flow chart of a multimodal vehicle trajectory prediction method of this embodiment;
[0041] Figure 2 for Figure 1 Schematic diagram of the LSTM encoder-decoder network framework of the illustrated embodiment;
[0042] Figure 3 for Figure 1 Schematic diagram of the internal structure of the LSTM recurrent unit of the illustrated embodiment;
[0043] Figure 4 for Figure 1 Schematic diagram of the conversion between the world coordinate system and the candidate lane centerline coordinate system in the illustrated embodiment;
[0044] Figure 5 for Figure 1 A schematic diagram of an interaction model for modeling interaction relationships through a spatial grid according to the embodiment shown;
[0045] Figure 6 for Figure 1 A schematic diagram of an interaction model for modeling interaction relationships based on an attention mechanism according to an embodiment of the present invention;
[0046] Figure 7 for Figure 1 The illustrated embodiment is a schematic diagram of embedding an interaction model at the input position at the current moment. DETAILED DESCRIPTION
[0047] The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.
[0048] LSTM encoder-decoder network:
[0049] refer to Figure 2 As shown, the LSTM encoder-decoder network is primarily used to solve the mapping problem from one sequence to another. In this invention, it is used to achieve sequence mapping from historical observation trajectories to future predicted trajectories. The basic building block of the LSTM encoder-decoder network is the LSTM, which is an improvement on the basic recurrent neural network (RNN) and has significant advantages and performance for processing sequence data.
[0050] refer to Figure 3As shown in the figure, the LSTM encoder-decoder network is primarily divided into two processes: encoding and decoding. These processes are implemented using different LSTM networks: the LSTM encoder and the LSTM decoder. The encoding process involves cyclically storing the input sequence in the memory cells of the LSTM encoder. Each LSTM recurrent unit receives a known input sequence. The decoding process uses the final output of the LSTM encoder's memory cell state, which contains all the information about the input sequence, as the initial value for the memory cells in the LSTM decoder. The output sequence is then generated gradually using the hidden state output by each recurrent unit of the LSTM decoder. Furthermore, the prediction output by each LSTM recurrent unit is treated as the true value and fed into the next LSTM recurrent unit.
[0051] refer to Figure 1 As shown, this embodiment provides a multimodal vehicle trajectory prediction method, comprising the following steps:
[0052] S1: Based on the road map, the historical motion trajectories of the predicted vehicle and multiple surrounding vehicles in the world coordinate system are obtained respectively, and processed into the historical motion trajectories of the predicted vehicle and multiple surrounding vehicles in the candidate lane centerline coordinate system;
[0053] S1.1: Obtain the historical motion trajectories of the predicted vehicle and multiple surrounding vehicles in the world coordinate system based on the road map;
[0054] S1.2: Use a breadth-first approach based on the Manhattan distance to obtain the candidate lane centerline for the predicted vehicle. Project each trajectory point from the predicted vehicle and surrounding vehicles' historical motion trajectories in the world coordinate system onto the candidate lane centerline. Find the point on the candidate lane centerline closest to each trajectory point as a matching point.
[0055] The formula for Manhattan distance is:
[0056] S M =|x A -x B |+|y A -y B |
[0057] In the formula, (x A ,y A ) and (x B ,y B ) are the coordinates of points A and B on the road map in the world coordinate system, S M It is the Manhattan distance between points A and B on the road map in the world coordinate system.
[0058] The process for obtaining candidate lane centerlines for the predicted vehicle is as follows: A minimum Manhattan distance is defined, and a rectangular region is defined with the predicted vehicle as the center. A Manhattan threshold is used to determine whether a candidate lane centerline intersects the rectangular region. If no candidate lane centerline intersects the rectangular region, the threshold is continuously increased until the lane centerline is found to be unknown and one or more candidate lane centerlines actually exist.
[0059] S1.3: Establish a candidate lane centerline coordinate system based on the candidate lane centerline and matching points, and obtain the coordinates of each trajectory point in the candidate lane centerline coordinate system. Then, obtain the historical motion trajectory of the predicted vehicle and surrounding vehicles in the candidate lane centerline coordinate system.
[0060] refer to Figure 4 As shown in the figure, the candidate lane centerline coordinate system is specifically as follows: the y-axis direction is along the candidate lane centerline, the x-axis direction is perpendicular to the candidate lane centerline, and the matching point corresponding to the vehicle's trajectory point at the current moment on the candidate lane centerline is the coordinate origin.
[0061] The coordinates of each trajectory point in the candidate lane centerline coordinate system are: the x value is the distance between the trajectory point and the matching point, and the positive direction of the x-axis is to the right of the candidate lane centerline's forward direction; the y value is the distance between the matching point corresponding to the trajectory point and the coordinate origin along the candidate lane centerline, and the positive direction of the y-axis is the forward direction along the candidate lane centerline.
[0062] S2: Encode the historical motion trajectories of the predicted vehicle and each surrounding vehicle in the candidate lane centerline coordinate system, obtain a historical trajectory feature vector of the predicted vehicle and a historical trajectory feature vector of each surrounding vehicle, and obtain an interaction relationship feature vector of the predicted vehicle and the surrounding vehicles at the current moment based on the historical trajectory feature vector of the predicted vehicle and the historical trajectory feature vector of each surrounding vehicle;
[0063] S2.1: Encode the historical motion trajectories of the predicted vehicle and each surrounding vehicle in the candidate lane centerline coordinate system, respectively, to obtain a historical trajectory feature vector of the predicted vehicle and a historical trajectory feature vector of each surrounding vehicle;
[0064] The formula for encoding the historical trajectory at each moment to obtain the historical trajectory feature vector at the current moment is:
[0065]
[0066] Where, φ e (· is a fully connected layer used to preliminarily extract historical trajectory features, W1 is its corresponding parameter, S t is the location coordinate information of the historical trajectory at time t; LSTM e(·) is the LSTM encoder, and W2 is its corresponding parameter; The input of the LSTM encoder in the LSTMencoder-decoder benchmark vehicle trajectory prediction network is the historical motion trajectory feature vector at the current moment; and are the hidden state and memory cell state in the LSTM encoder at time t, and are the hidden state and memory cell state in the LSTM encoder at time t-1, respectively.
[0067] The initial value is set as follows:
[0068] S2.2: Integrate the historical trajectory feature vectors of the surrounding vehicles to obtain the historical trajectory feature vectors of the surrounding vehicles, and obtain the interaction relationship feature vector between the predicted vehicle and the surrounding vehicles at the current moment based on the historical trajectory feature vector of the predicted vehicle and the historical trajectory feature vectors of the surrounding vehicles through the interaction model.
[0069] There are two main types of interaction relationship modeling methods: Figure 5 As shown in the figure, the first category is to model the interaction relationship through spatial grid, such as Social Pooling and Convolutional Social Pooling; Figure 6 As shown in Figure 2, the second category is to model interaction relationships based on the attention mechanism, such as Pooling Module. These three different combined interaction relationship models can all be used in the vehicle trajectory prediction network.
[0070] Since the interaction relationship is information with time continuity, it does not need to be considered at every moment, so the interaction relationship is only calculated at the current moment. Figure 7 As shown in the figure, in the LSTM encoder-decoder network, the interaction model is calculated at the input position at the current moment (that is, the last frame of historical observation). The formula for calculating the interaction model is as follows:
[0071]
[0072] Where, φ r is the interaction model, W r For its corresponding parameters, is the prediction of the interaction model input, r obs Predict the feature vector of the interaction relationship between the vehicle and its surrounding vehicles at the current moment.
[0073] S3: Sample the center lines of candidate lanes and use a multi-layer perceptron to obtain the intention modal encoding corresponding to the historical trajectory feature vector of the predicted vehicle. Use one-hot encoding to obtain the motion modal encoding of the predicted vehicle at different speeds.
[0074] The process of using one-hot encoding to obtain the motion mode encoding of the predicted vehicle at different speeds is specifically as follows: the speed is divided into multiple categories according to size, and the speed size categories are encoded based on one-hot encoding.
[0075] The specific steps for obtaining the intention modal coding are as follows: starting from the matching point of the vehicle on the center line of the candidate lane at the current moment, 30 points are sampled at fixed-length intervals, and then the coordinates of these 30 points relative to the starting point are used to form a vector of length 60. Finally, this vector is passed through a multi-layer perceptron to obtain the intention modal coding corresponding to the historical trajectory feature vector of the predicted vehicle.
[0076] S4: The intention modality code, motion modality code, predicted vehicle historical trajectory feature vector, and the feature vector of the interaction relationship between the current predicted vehicle and surrounding vehicles are concatenated as input to the LSTM decoder. The LSTM decoder decodes and outputs multiple predicted vehicle trajectories and corresponding confidence labels. The predicted trajectory with the highest confidence is selected as the optimal trajectory through a trained neural network.
[0077] The input to the LSTM decoder is:
[0078]
[0079] Where, ψ line (M) is the set of candidate lane centerlines; v line is a vector consisting of n points sampled from the center line of a specific candidate lane; MLP is a multi-layer perceptron, W MLP Its corresponding parameter; τ line To predict the intention modal encoding corresponding to the historical motion trajectory feature vector of the vehicle, Predict the historical motion trajectory feature vector of the vehicle at all times, r obs is the feature vector of the interaction relationship between the current predicted vehicle and surrounding vehicles, ψ m (N) is a set of N one-hot encodings, τ m To predict the vehicle's motion mode encoding, is the input feature vector of the LSTM decoder.
[0080] During the training process of the neural network, an arbitration method is used to select a trajectory that best matches the true trajectory from all the output predicted vehicle trajectories.
[0081] The arbitration method is as follows: when determining the intention mode in the first stage, all candidate lane centerlines are sorted from most to least according to the number of points whose distances from the predicted future true value of the vehicle's trajectory to the candidate lane centerline are within the Manhattan threshold range, and then the candidate lane centerline with the largest number of trajectory true value points within the threshold range is selected as the current intention mode; when determining the motion mode in the second stage, still under the intention mode determined in the first stage, that is, among the predicted trajectories traveling along the corresponding candidate lane centerlines, the trajectory that best matches the trajectory true value is determined according to the principle of minimizing the average position deviation, i.e., the final "winner".
[0082] The confidence true value label of the trajectory that best matches the true value of the trajectory is set to 1, and the corresponding labels of other trajectories are set to 0. At the same time, the distance error loss between the predicted trajectory and the true value of the trajectory and the cross entropy loss between the predicted label and the true value label are calculated. These are used to backpropagate and update the network parameters, continuously optimize the predicted trajectory, set the network iteration cycle, and output the optimal network parameters and the corresponding optimal predicted trajectory.
[0083] In the network training process, in order to take into account the effective learning of simple samples and improve the robustness to abnormal samples, the loss function used in this embodiment is the Smooth loss function between the corresponding points of the predicted trajectory and the true value trajectory. L1 Loss, the specific expression is:
[0084]
[0085] In summary, the present invention provides a multimodal vehicle trajectory prediction algorithm that combines interactive relationship modeling and road map information to achieve accurate and rapid prediction of vehicle motion trajectories. This algorithm, without the need for labeled datasets or sampling, can self-supervise the multimodal characteristics of vehicle motion, obtaining a comprehensive description of the predicted trajectory distribution and enabling long-term prediction. The present invention considers road map information and combines interactive relationship modeling to perform multimodal vehicle trajectory prediction based on two-level random multi-choice learning. The present invention has strong scenario generalization capabilities and is applicable not only to high-speed scenarios with fixed road structures, but also to complex urban road scenarios, thus possessing significant practical application value.
[0086] The above description of the embodiments is intended to facilitate understanding and use of the invention by those skilled in the art. It will be apparent that those skilled in the art can readily make various modifications to these embodiments and apply the general principles described herein to other embodiments without requiring inventive effort. Therefore, the present invention is not limited to the above-described embodiments. Improvements and modifications made by those skilled in the art based on the disclosure of the present invention, without departing from the scope of the present invention, should be within the scope of protection of the present invention.
Claims
1. A multimodal vehicle trajectory prediction method, characterized in that: The following steps are involved: S1: Based on the road map, the historical motion trajectories of the predicted vehicle and multiple surrounding vehicles in the world coordinate system are obtained respectively, and processed into the historical motion trajectories of the predicted vehicle and each surrounding vehicle in the candidate lane centerline coordinate system; S2: Encode the historical motion trajectories of the predicted vehicle and each surrounding vehicle in the candidate lane centerline coordinate system, obtain a historical trajectory feature vector of the predicted vehicle and a historical trajectory feature vector of each surrounding vehicle, and obtain an interaction relationship feature vector of the predicted vehicle and the surrounding vehicles at the current moment based on the historical trajectory feature vector of the predicted vehicle and the historical trajectory feature vector of each surrounding vehicle; S3: Sampling the centerline of the candidate lane to obtain the intention modal code corresponding to the historical trajectory feature vector of the predicted vehicle, and using one-hot encoding to obtain the motion modal code of the predicted vehicle at different speeds; S4: The intention modality code, motion modality code, historical trajectory feature vector of the predicted vehicle, and the feature vector of the interaction relationship between the current predicted vehicle and surrounding vehicles are concatenated as input to the LSTM decoder. The LSTM decoder decodes and outputs multiple predicted trajectories of the predicted vehicle and their corresponding confidence labels. The neural network is trained to select the predicted trajectory that best matches the true trajectory value. Said S1 comprises the following steps: S1.1: Obtain the historical motion trajectories of the predicted vehicle and multiple surrounding vehicles in the world coordinate system based on the road map; S1.2: Use a breadth-first approach based on the Manhattan distance to obtain the candidate lane centerline for the predicted vehicle. Project each trajectory point from the predicted vehicle and surrounding vehicles' historical motion trajectories in the world coordinate system onto the candidate lane centerline. Find the point on the candidate lane centerline closest to each trajectory point as a matching point. S1.3: Establishing a candidate lane centerline coordinate system based on the candidate lane centerline and the matching points, obtaining the coordinates of each trajectory point in the candidate lane centerline coordinate system, and further obtaining the historical motion trajectories of the predicted vehicle and surrounding vehicles in the candidate lane centerline coordinate system; The process of obtaining the intent modal code is as follows: starting from the matching point of the vehicle on the candidate lane centerline at the current moment, sampling at fixed intervals, forming a vector with the coordinates of all sampling points relative to the starting point, and passing this vector through a multi-layer perceptron to obtain the intent modal code; The input of the LSTM decoder is: Where, ψ line (M) is the set of candidate lane centerlines; v line is a vector consisting of n points sampled from the center line of a specific candidate lane; MLP is a multi-layer perceptron, W MLP Its corresponding parameter; τ line To predict the intention modal encoding corresponding to the historical motion trajectory feature vector of the vehicle, Predict the historical motion trajectory feature vector of the vehicle at all times, r obs is the feature vector of the interaction relationship between the current predicted vehicle and surrounding vehicles, ψ m (N) is a set of N one-hot encodings, τ m To predict the vehicle's motion mode encoding, is the input feature vector of the LSTM decoder; An arbitration method is used to select a predicted trajectory that best matches the true trajectory value from all predicted vehicle trajectories output. The arbitration method is specifically as follows: When determining the intention mode, all candidate lane centerlines are sorted from most to least based on the number of points whose distances between the predicted future true trajectory of the vehicle and the candidate lane centerlines are within the Manhattan threshold range. The candidate lane centerline with the largest number of true trajectory points within the threshold range is selected as the current intention mode. When determining the motion mode, under the current intention mode, the predicted trajectory that best matches the true trajectory is determined based on the principle of minimizing the average position deviation.
2. A multimodal vehicle trajectory prediction method according to claim 1, characterized in that: The candidate lane centerline coordinate system is specifically: the y-axis is along the candidate lane centerline, the x-axis is perpendicular to the candidate lane centerline, and the matching point corresponding to the vehicle's trajectory point predicted at the current moment on the candidate lane centerline is the coordinate origin.
3. A multimodal vehicle trajectory prediction method according to claim 2, characterized in that: The coordinates of each trajectory point in the candidate lane centerline coordinate system are: the x value is the distance between the trajectory point and the matching point, and the positive direction of the x-axis is to the right of the candidate lane centerline's forward direction; the y value is the distance between the matching point corresponding to the trajectory point and the coordinate origin along the candidate lane centerline, and the positive direction of the y-axis is the forward direction along the candidate lane centerline.
4. A multimodal vehicle trajectory prediction method according to claim 1, characterized in that: The S2 comprises the following steps: S2.1: Encode the historical motion trajectories of the predicted vehicle and each surrounding vehicle in the candidate lane centerline coordinate system, respectively, to obtain a historical trajectory feature vector of the predicted vehicle and a historical trajectory feature vector of each surrounding vehicle; S2.2: Integrate the historical trajectory feature vectors of the surrounding vehicles to obtain the historical trajectory feature vectors of the surrounding vehicles, and obtain the interaction relationship feature vector between the predicted vehicle and the surrounding vehicles at the current moment based on the historical trajectory feature vector of the predicted vehicle and the historical trajectory feature vectors of the surrounding vehicles through the interaction model.
5. A multimodal vehicle trajectory prediction method according to claim 4, characterized in that: The formula for encoding the historical trajectory at each moment to obtain the historical trajectory feature vector at the current moment is: Where, φ e (·) is a fully connected layer used to preliminarily extract historical trajectory features, W1 is its corresponding parameter, S t is the location coordinate information of the historical trajectory at time t; LSTM e (·) is the LSTM encoder, and W2 is its corresponding parameter; The input of the LSTM encoder in the LSTMencoder-decoder benchmark vehicle trajectory prediction network is the historical motion trajectory feature vector at the current moment; and are the hidden state and memory cell state in the LSTM encoder at time t, and are the hidden state and memory cell state in the LSTM encoder at time t-1, respectively.
6. A multimodal vehicle trajectory prediction method according to claim 4, characterized in that: The formula for calculating the interaction model is: Where, φ r is the interaction model, W r For its corresponding parameters, is the prediction of the interaction model input, r obs Predict the feature vector of the interaction relationship between the vehicle and its surrounding vehicles at the current moment.