Staged vehicle trajectory prediction model fused with high-precision map
By integrating high-precision map information in the vehicle trajectory prediction model, extracting lane sections and vehicle trajectory characteristics, and performing phased prediction, the problem of unsatisfactory prediction results caused by the failure to fully utilize map information in the prior art is solved, and higher prediction accuracy and the possibility of multimodal prediction results are achieved.
Patent Information
- Application Number
- CN202411804365.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-10
- Publication Date
- 2025-04-01
AI Technical Summary
The existing vehicle trajectory prediction algorithm fails to make full use of high-precision map information, resulting in unsatisfactory prediction results.
A staged vehicle trajectory prediction model integrating high-precision maps is proposed. The lane section features and vehicle trajectory characteristics are extracted through the encoding module, and combined with waypoint extraction and precision prediction modules, rough prediction and precision prediction are performed to improve the accuracy of prediction.
By integrating high-precision map information, the information dimension considered in model prediction is expanded, the prediction accuracy of the vehicle's future trajectory is improved, and the possibility of multimodal prediction results is significantly improved.
Smart Images
Figure CN119740094A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of autonomous driving vehicle technology, and in particular to a staged vehicle trajectory prediction model integrated with a high-precision map. Background Art
[0002] The existing autonomous driving framework can be divided into three core processes: environmental perception and positioning, decision-making and planning, and execution control. The decision-making and planning module is divided into two parts: path planning and behavior decision-making. Behavior decision-making mainly includes two aspects: one is the vehicle's own behavior decision, and the other is the prediction of the behavior of other vehicles.
[0003] The prediction module is an important part of autonomous driving technology and an important reference for subsequent decision-making, planning and control processes. It reflects the evaluation of the driving intention of other vehicles. The trajectory prediction model is based on the historical trajectories of several vehicles and other auxiliary information, such as map data. The vehicle historical trajectory algorithm estimates its future trajectory. To predict the trajectory of a vehicle, not only its own historical trajectory must be considered, but also the status of moving or fixed obstacles in the environment.
[0004] Most of the known deep learning-based vehicle trajectory prediction algorithms currently use encoding and decoding structures. The encoding part usually embeds the input data (such as vehicle historical trajectory, lane coordinates, etc.) into a fixed dimension to generate corresponding features, and then uses a neural network to let the features containing different information learn from each other, so that the features can learn the complex information contained in the data (such as the interaction between vehicles and the vehicle driving strategy). The decoding part uses the encoded features and generates results for the target dimension through MLP output. Use loss to supervise model parameters. The generated results include outputting the entire trajectory, or completing the trajectory after generating the final point coordinates, etc.
[0005] Most of the existing vehicle trajectory prediction algorithms only consider all dynamic participants in the environment, such as vehicles, pedestrians, etc., but do not consider map information which has great reference value for the future trajectory of the vehicle, so the prediction effect is not ideal. Summary of the invention
[0006] The purpose of the present invention is to provide a staged vehicle trajectory prediction model integrated with high-precision maps to achieve accurate prediction of the vehicle's future trajectory.
[0007] In order to solve the above technical problems, the present invention provides a staged vehicle trajectory prediction model integrated with high-precision maps, comprising: An encoding module is used to encode lane point information and vehicle displacement information at each step to obtain encoded lane segment features and vehicle trajectory features; A coarse prediction and waypoint extraction module is used to extract waypoint features from vehicle trajectory features, and obtain a coarse prediction trajectory based on waypoint features, lane segment features and vehicle trajectory features; The waypoint recoding and fine prediction module is used to embed the rough predicted trajectory into the target dimension and then encode it to obtain the segment waypoint features. The segment predicted trajectory is obtained based on the segment waypoint features, lane segment features, and vehicle trajectory features, and the segment predicted trajectory is spliced to obtain the fine predicted trajectory. According to the above scheme, the method for obtaining the encoded lane segment features includes: Embed lane point information into a specific dimension to obtain lane point features; Encode the coordinate information and the auxiliary information into lane segment features; the lane point information includes the coordinate information and the auxiliary information; Based on the spatial relationship between lane points and lane segments, the lane point features within the lane segment are integrated into the lane segment features; Based on the spatial relationship between lane segments, the lane segment features are allowed to learn the lane segment features of adjacent lanes to obtain encoded lane segment features.
[0008] According to the above scheme, the method for obtaining the encoded vehicle trajectory features includes: The displacement information of each step of the vehicle is converted from the rectangular coordinate system to the polar coordinate system and embedded into a specific dimension to obtain the displacement characteristics of each step of the vehicle; The displacement features of each vehicle step are made to learn the features of the adjacent time and space range to obtain the encoded vehicle trajectory features.
[0009] According to the above scheme, the method for extracting waypoint features includes: extracting the last known position of the vehicle from the vehicle trajectory features as the waypoint features of the vehicle and expanding it according to the multi-modal quantity.
[0010] According to the above scheme, the method for obtaining the rough prediction trajectory includes: Use waypoint features as queries for graph attention; The historical features, map information, and other vehicle features are used as the key and value of the cross attention mechanism; the historical features are obtained by extracting the vehicle trajectory features of the vehicle itself, the map information is obtained by extracting the lane segment features, and the other vehicle features are obtained by extracting the vehicle trajectory features of other vehicles; Use graph attention to let queries learn key & value information; Use the self-attention mechanism to let multimodal features learn from each other, and obtain queries that have learned spatiotemporal features; multimodal features include historical features, map information, and other car features; Based on the query with learned spatiotemporal features, a coarse prediction trajectory is obtained using a multi-layer perceptron.
[0011] According to the above scheme, the method for obtaining the segment waypoint features includes: The coarse prediction trajectory is segmented and embedded into the target dimension, and the segment embedding is obtained according to the set duration of each segment; The segment embeddings are encoded using a gated recurrent unit to obtain segment waypoint features.
[0012] According to the above scheme, the method for obtaining the segment prediction trajectory includes: Use segment waypoint features as queries for graph attention; The historical features, map information, and other vehicle features are used as the key and value of the cross attention mechanism; the historical features are obtained by extracting the vehicle trajectory features of the vehicle itself, the map information is obtained by extracting the lane segment features, and the other vehicle features are obtained by extracting the vehicle trajectory features of other vehicles; Use the self-attention mechanism to let multimodal features learn from each other and obtain the learned query; multimodal features include historical features, map information, and other car features; The learned query is input into the segment decoder and decoded to obtain the segment prediction trajectory.
[0013] According to the above scheme, the loss function of the staged vehicle trajectory prediction model is composed of the sum of coarse prediction loss, waypoint position difference loss, fine prediction loss, and final position difference loss; the coarse prediction loss is obtained according to the coarse prediction trajectory, the waypoint position difference is obtained according to the waypoint characteristics, the fine prediction loss is obtained according to the fine prediction trajectory, and the final position difference loss is obtained according to the final coordinates of the vehicle.
[0014] The present invention also provides a staged vehicle trajectory prediction method integrating a high-precision map, comprising: According to the lane point information and the displacement information of each step of the vehicle, the encoded lane segment features and vehicle trajectory features are obtained; Extract waypoint features from vehicle trajectory features, and obtain a rough predicted trajectory based on waypoint features, lane segment features, and vehicle trajectory features; The rough predicted trajectory is segmented and embedded into the target dimension and then encoded to obtain the segment waypoint features. The segment predicted trajectory is obtained based on the segment waypoint features, lane segment features, and vehicle trajectory features. The segment predicted trajectories are spliced to obtain the refined predicted trajectory.
[0015] The present invention also provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the above-mentioned method for predicting vehicle trajectories in stages by integrating high-precision maps are implemented.
[0016] Beneficial Effects The present invention uses lane point information containing map-related information as the input of the model, which expands the dimension of information considered by the model prediction and improves the accuracy of the model prediction. By performing segmented predictions on the vehicle's historical trajectory, the predicted future trajectory is multimodal and contains multiple possible results. Compared with the single prediction result of the prior art, the possibility of accurately predicting the future trajectory is significantly improved.
[0017] Furthermore, the data accuracy of the model input is converted from the traditional per-vehicle to per-vehicle per-step, so that the model can learn more complex spatiotemporal interaction relationships, thereby improving recognition accuracy; by combining coarse prediction and fine prediction, the prediction accuracy of this model is higher than that of direct prediction, and this model alleviates the problem of rapid decrease in prediction accuracy as the prediction time increases by adopting a segmented prediction method. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] Figure 1 This is a prediction flow chart of a staged vehicle trajectory prediction model integrated with a high-precision map according to the first embodiment of the present invention; Figure 2 It is a flow chart of a method for predicting vehicle trajectories in stages by integrating high-precision maps according to the second embodiment of the present invention. DETAILED DESCRIPTION
[0019] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution of the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings of the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the described embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.
[0020] Embodiment 1: See also Figure 1 , a staged vehicle trajectory prediction model integrated with high-precision maps, including: The encoding module is used to encode the lane point information and the displacement information of each step of the vehicle to obtain the encoded lane segment features and vehicle trajectory features; in this embodiment, the encoded lane segment features and vehicle trajectory features are respectively expressed as (P, E_dim) and (A, K×T_history, E_dim). A coarse prediction and waypoint extraction module is used to extract waypoint features from vehicle trajectory features, and obtain a coarse prediction trajectory based on waypoint features, lane segment features and vehicle trajectory features; The waypoint recoding and fine prediction module is used to embed the rough predicted trajectory into the target dimension and then encode it to obtain the segment waypoint features. The segment predicted trajectory is obtained based on the segment waypoint features, lane segment features, and vehicle trajectory features, and the segment predicted trajectory is spliced to obtain the fine predicted trajectory. Furthermore, the method for obtaining the encoded lane segment features includes: Embed lane point information into a specific dimension to obtain lane point features; Encode the coordinate information and auxiliary information into lane segment features; the lane point information includes coordinate information (including the coordinates of the starting lane point) and auxiliary information (including whether there are left and right adjacent lanes); Based on the spatial relationship between lane points and lane segments, the lane point features within the lane segment are integrated into the lane segment features; Based on the spatial relationship between lane segments, the lane segment features are allowed to learn the lane segment features of adjacent lanes to obtain encoded lane segment features.
[0021] Furthermore, the method for obtaining the encoded vehicle trajectory features includes: The displacement information of each step of the vehicle is converted from the rectangular coordinate system to the polar coordinate system and embedded into a specific dimension to obtain the displacement characteristics of each step of the vehicle; The displacement feature of each vehicle step is made to learn the features of the adjacent time and space range to obtain the encoded vehicle trajectory features. In this embodiment, firstly, the displacement feature of each vehicle step is made to learn the features of the same vehicle in the past period of time, and then the displacement feature of each vehicle step is made to learn the lane segment features of the adjacent lane, as well as the features of other adjacent vehicles (obtained according to the vehicle trajectory features of other vehicles).
[0022] Furthermore, the method for extracting waypoint features includes: extracting the last known position of the vehicle from the vehicle trajectory features as the waypoint features of the vehicle and expanding it according to the number of multi-modalities; in this embodiment, the waypoint features are represented by (A, M, E_dim).
[0023] Furthermore, the method for obtaining the rough prediction trajectory includes: Use waypoint features as the query for graph attention; The historical features, map information, and other vehicle features are used as the key & value of the cross attention mechanism; the historical features are obtained by extracting the vehicle trajectory features of the vehicle itself, the map information is obtained by extracting the lane segment features, and the other vehicle features are obtained by extracting the vehicle trajectory features of other vehicles; it can be understood that the edge relationship in the map information is obtained based on the last known position of the vehicle and the spatial position of the road segment, and the edge relationship in the other vehicle features is obtained based on the last known position of each vehicle; Use graph attention to let queries learn key & value information; Use the self-attention mechanism to allow multimodal features to learn from each other and obtain queries that have learned spatiotemporal features; multimodal features include historical features, map information, and other vehicle features; Based on the query with learned spatiotemporal features, a rough prediction trajectory is obtained using a multi-layer perceptron (MLP) prediction; in this embodiment, the rough prediction trajectory is represented as (A, M, K×T_future, 2).
[0024] Furthermore, the method for obtaining the segment waypoint features includes: The rough prediction trajectory is segmented and embedded into the target dimension, and the segment embedding is obtained according to the set duration of each segment (in this embodiment, the duration of each segment is set to 1 second); in this embodiment, the segment embedding is expressed as (A, M, K×T_future, E_dim) The segment embedding is encoded using a gated recurrent unit (GRU) to obtain segment waypoint features. In this embodiment, the segment features are represented as (1, A, M, E_dim). All segment features are concatenated to obtain the input of the precise prediction (K, A, M, E_dim).
[0025] Furthermore, the method for obtaining the segment prediction trajectory includes: Use segment waypoint features as queries for graph attention; The historical features, map information, and other vehicle features are used as the key and value of the cross attention mechanism; the historical features are obtained by extracting the vehicle trajectory features of the vehicle itself, the map information is obtained by extracting the lane segment features, and the other vehicle features are obtained by extracting the vehicle trajectory features of other vehicles; Use the self-attention mechanism to let multimodal features learn from each other and obtain the learned query; multimodal features include historical features, map information, and other car features; The learned query is input into the segment decoder for decoding to obtain the segment prediction trajectory. In this embodiment, the segment prediction trajectory is represented as (A, M, K, 2), and the refined prediction trajectory obtained by concatenating the segment prediction trajectories is represented as (A, M, K×T_future, 2).
[0026] Furthermore, the loss function of the staged vehicle trajectory prediction model is composed of the sum of the coarse prediction loss, the waypoint difference loss, the fine prediction loss, and the final difference loss; the coarse prediction loss is obtained according to the coarse prediction trajectory, the waypoint difference is obtained according to the waypoint characteristics, the fine prediction loss is obtained according to the fine prediction trajectory, and the final difference loss is obtained according to the final coordinates of the vehicle; In this embodiment, the evaluation indicators of the vehicle trajectory prediction result are ADE (average displacement error) and FDE (final displacement error). Specifically, the rough prediction loss calculation includes: calculating the Euclidean distance between each rough prediction coordinate and the ground truth coordinate (real future coordinate) and then finding the average value; the waypoint difference loss calculation includes calculating the Euclidean distance between each waypoint coordinate and the ground truth coordinate and then finding the average value; the fine prediction loss includes: calculating the Euclidean distance between each fine prediction coordinate and the ground truth coordinate and then finding the average value; the final difference loss includes: calculating the Euclidean distance between the final coordinate of each vehicle and the ground truth coordinate and then finding the average value.
[0027] Embodiment 2: See also Figure 2 Based on the first embodiment, this embodiment provides a method for predicting vehicle trajectories in stages by integrating a high-precision map, including: S1. Encode lane point information and vehicle displacement information at each step to obtain encoded lane segment features and vehicle trajectory features; S2, extracting waypoint features from vehicle trajectory features, and obtaining a rough predicted trajectory based on the waypoint features, lane segment features, and vehicle trajectory features; S3. The rough predicted trajectory is segmented and embedded into the target dimension and then encoded to obtain segment waypoint features. The segment predicted trajectory is obtained according to the segment waypoint features, lane segment features, and vehicle trajectory features. The segment predicted trajectories are spliced to obtain the refined predicted trajectory.
[0028] Embodiment three: Based on the second embodiment, this embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the method for predicting vehicle trajectories in stages by integrating a high-precision map described in the second embodiment are implemented.
[0029] It should be pointed out that, according to the needs of implementation, the various steps / components described in this application can be split into more steps / components, and two or more steps / components or partial operations of steps / components can be combined into new steps / components to achieve the purpose of the present invention.
[0030] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.
Claims
1. A staged vehicle trajectory prediction model integrated with high-precision maps, characterized in that: include: An encoding module is used to encode lane point information and vehicle displacement information at each step to obtain encoded lane segment features and vehicle trajectory features; A coarse prediction and waypoint extraction module is used to extract waypoint features from vehicle trajectory features, and obtain a coarse prediction trajectory based on waypoint features, lane segment features and vehicle trajectory features; The waypoint recoding and fine prediction module is used to embed the rough predicted trajectory into the target dimension and then encode it to obtain the segment waypoint features. The segment predicted trajectory is obtained based on the segment waypoint features, lane segment features, and vehicle trajectory features, and the segment predicted trajectory is spliced to obtain the fine predicted trajectory.
2. The staged vehicle trajectory prediction model integrated with high-precision maps according to claim 1 is characterized in that: Methods for obtaining the encoded lane segment features include: Embed lane point information into a specific dimension to obtain lane point features; Encode the coordinate information and the auxiliary information into lane segment features; the lane point information includes the coordinate information and the auxiliary information; Based on the spatial relationship between lane points and lane segments, the lane point features within the lane segment are integrated into the lane segment features; Based on the spatial relationship between lane segments, the lane segment features are allowed to learn the lane segment features of adjacent lanes to obtain encoded lane segment features.
3. The staged vehicle trajectory prediction model integrated with high-precision maps according to claim 1 is characterized in that: The method for obtaining the encoded vehicle trajectory features includes: The displacement information of each step of the vehicle is converted from the rectangular coordinate system to the polar coordinate system and embedded into a specific dimension to obtain the displacement characteristics of each step of the vehicle; The displacement features of each vehicle step are made to learn the features of the adjacent time and space range to obtain the encoded vehicle trajectory features.
4. The staged vehicle trajectory prediction model integrated with high-precision maps according to claim 1 is characterized in that: The method for extracting waypoint features includes: extracting the last known position of the vehicle from the vehicle trajectory features as the waypoint features of the vehicle and expanding it according to the multi-modal quantity.
5. The staged vehicle trajectory prediction model integrated with high-precision maps according to claim 1 is characterized in that: The method for obtaining the rough prediction trajectory includes: Use waypoint features as queries for graph attention; The historical features, map information, and other vehicle features are used as the key and value of the cross attention mechanism; the historical features are obtained by extracting the vehicle trajectory features of the vehicle itself, the map information is obtained by extracting the lane segment features, and the other vehicle features are obtained by extracting the vehicle trajectory features of other vehicles; Use graph attention to let queries learn key & value information; Use the self-attention mechanism to let multimodal features learn from each other, and obtain queries that have learned spatiotemporal features; multimodal features include historical features, map information, and other car features; Based on the query with learned spatiotemporal features, a coarse prediction trajectory is obtained using a multi-layer perceptron.
6. The staged vehicle trajectory prediction model integrated with high-precision maps according to claim 1 is characterized in that: The method for obtaining the segment waypoint features includes: The coarse prediction trajectory is segmented and embedded into the target dimension, and the segment embedding is obtained according to the set duration of each segment; The segment embeddings are encoded using a gated recurrent unit to obtain segment waypoint features.
7. The staged vehicle trajectory prediction model integrated with high-precision maps according to claim 1 is characterized in that: The method for obtaining the segment prediction trajectory includes: Use segment waypoint features as queries for graph attention; The historical features, map information, and other vehicle features are used as the key and value of the cross attention mechanism; the historical features are obtained by extracting the vehicle trajectory features of the vehicle itself, the map information is obtained by extracting the lane segment features, and the other vehicle features are obtained by extracting the vehicle trajectory features of other vehicles; Use the self-attention mechanism to let multimodal features learn from each other and obtain the learned query; multimodal features include historical features, map information, and other car features; The learned query is input into the segment decoder and decoded to obtain the segment prediction trajectory.
8. The staged vehicle trajectory prediction model integrated with high-precision maps according to claim 1 is characterized in that: The loss function of the staged vehicle trajectory prediction model is composed of the sum of coarse prediction loss, waypoint difference loss, fine prediction loss, and final difference loss; the coarse prediction loss is obtained according to the coarse prediction trajectory, the waypoint difference is obtained according to the waypoint characteristics, the fine prediction loss is obtained according to the fine prediction trajectory, and the final difference loss is obtained according to the final coordinates of the vehicle.
9. A phased vehicle trajectory prediction method integrating high-precision maps, characterized in that: include: According to the lane point information and the displacement information of each step of the vehicle, the encoded lane segment features and vehicle trajectory features are obtained; Extract waypoint features from vehicle trajectory features, and obtain a rough predicted trajectory based on waypoint features, lane segment features, and vehicle trajectory features; The rough predicted trajectory is segmented and embedded into the target dimension and then encoded to obtain the segment waypoint features. The segment predicted trajectory is obtained based on the segment waypoint features, lane segment features, and vehicle trajectory features. The segment predicted trajectories are spliced to obtain the refined predicted trajectory.
10. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the staged vehicle trajectory prediction method integrating high-precision maps as described in claim 9 are implemented.