Hierarchical trajectory prior based interactive joint prediction planning method without high-definition map

Abstract

The present application relates to a kind of high-precision map interaction joint prediction planning method based on layered track prior, it is related to intelligent driving planning technical field, the present application multi-modal initial track prediction module, preliminary planning module, multi-modal track re-prediction module and the fine planning module based on layered track prior construct the closed loop joint architecture of prediction, preliminary planning, re-prediction, fine planning, wherein, fine planning module compared with preliminary planning module introduce layered track prior mechanism, the human driving behavior prior in large-scale real driving track is injected into track planning process with the way of two-stage matching of scene-level coarse screening-behavior-level fine screening, provide high-quality prior, solve the problem that prediction and planning module are strongly dependent on map in the scene without high-precision map, the lack of interactive modeling of serial architecture, the rationality of planning track is poor without prior constraint, realize the deep coupling and joint optimization of prediction and planning module.

Description

Technical Field

[0001] This invention relates to the field of intelligent driving planning and decision-making technology, and in particular to a joint predictive planning method based on hierarchical trajectory priors without high-precision maps. Background Technology

[0002] In autonomous driving systems, the prediction and planning modules jointly determine the vehicle's behavioral decisions and driving safety in dynamic traffic environments. Current mainstream prediction and planning methods heavily rely on prior information such as lane-level topology, road boundaries, and traffic rules provided by high-precision maps. This results in severely insufficient robustness and generalization ability of the models in lightweight scenarios without high-precision map coverage, such as urban and rural roads, temporary construction zones, and maps that are outdated or not updated in a timely manner. These models are prone to problems such as unreasonable behavioral decisions, trajectories deviating from the feasible region, and overly conservative planning, making it difficult to balance real-time planning, safety, and traffic efficiency. Furthermore, existing prediction and planning methods generally adopt a sequential architecture of prediction followed by planning. The lack of bidirectional information exchange and joint optimization mechanisms between the prediction and planning modules prevents the incorporation of the vehicle's future planning intentions into the prediction stage. This makes it difficult to model the bidirectional and multidirectional dynamic interaction relationships between the vehicle and surrounding participants in traffic scenarios, leading to significant deviations between the prediction results and actual traffic interactions, thus affecting the rationality of the planned trajectory. In addition, without strong prior information from high-precision maps, existing methods struggle to provide stable and reliable trajectory rationality constraints for the planning module, causing the generated planned trajectory to deviate from human driving habits. Therefore, how to achieve deep coupling between prediction and planning modules in scenarios without high-precision maps, and how to compensate for the decline in planning quality caused by the lack of strong prior information, remain key challenges for the large-scale deployment of autonomous driving technology. Summary of the Invention

[0003] To solve the above-mentioned technical problems, or at least partially solve them, this invention provides a method for interactive joint prediction planning without high-precision maps based on hierarchical trajectory priors.

[0004] In a first aspect, the present invention provides a method for interactive joint prediction and planning without high-precision maps based on hierarchical trajectory priors, comprising: Acquire multimodal environmental perception data from a bird's-eye view, including the historical trajectories and states of traffic participants, as well as a lightweight map of the road scene they are in; The multimodal initial trajectory prediction module based on the lightweight map multi-layer spatiotemporal interaction prediction network predicts the multimodal initial trajectory features of surrounding traffic participants based on the collected multimodal environmental perception data. Using the multimodal initial trajectory features of surrounding traffic participants as input, the initial trajectory features of the vehicle are extracted through the Transformer-based initial planning module to obtain the vehicle's future motion intention and reference trajectory features. The vehicle's initial planning motion intent and reference trajectory features are input into the multimodal trajectory re-prediction module based on the lightweight map multi-layer spatiotemporal interaction prediction network to perform two-way interaction modeling between the vehicle and surrounding participants, and closed-loop interaction re-prediction to obtain refined multimodal re-predicted trajectory features of traffic participants that conform to the real interaction logic. Based on the multimodal re-predicted trajectory features of traffic participants, and through a fine planning module based on hierarchical trajectory priors, under a two-level prior matching mechanism consisting of scenario-level coarse screening and behavior-level fine screening, a unified motion decoder outputs the final safe and executable control trajectory of the vehicle.

[0005] Furthermore, the lightweight map multi-layer spatiotemporal interaction prediction network includes: The vector map encoder branch extracts map features from the light map for each traffic participant. The vector map encoder branch first encodes the vector light map through a multilayer perceptron to obtain low-order map features, and then uses a stacked VectorNet to upscale the low-order map features to obtain map features. The trajectory / lane context interaction branch organizes the trajectory segments and lane relationships of the time series; the historical agent feature branch organizes the state of the surrounding traffic participants of the time series; the trajectory / lane context interaction branch and the historical agent feature branch jointly provide the organized time series to the spatiotemporal historical trajectory encoding module, which encodes the heterogeneous time series into a unified historical spatiotemporal motion feature vector. The historical spatiotemporal motion feature vector encoded by the spatiotemporal historical trajectory encoding module is sent to the spatiotemporal multi-agent interaction module to obtain the spatiotemporal motion interaction features that affect the planning. The spatiotemporal motion interaction features and map features that influence the planning are processed through multi-head attention to obtain the trajectory features of traffic participants; among them, the trajectory features of traffic participants can be decoded by the motion decoder to obtain the trajectory.

[0006] Furthermore, the vectorized map encoder branch includes vectorizing lane, intersection, and topology information in the light map; vectorization includes semantic encoding of lane segments and recording the start and end coordinates of lane segments, and the semantic encoding, start and end coordinates of lane segments form a vectorized representation; The lane multilayer perceptron in the vector map encoder extracts corresponding low-order lane features for traffic participants based on the rotation matrix of the local region and the vectorized representation of the lane. , in, It is a multi-layer sensor encoder used for lanes. Traffic participants Rotation matrix of local region , These are the coordinates of the start and end positions of the lane segment. It is a lane section semantic attributes; The origin of the local coordinate system; By stacking The VectorNet layer upscales low-order lane features, where the VectorNet layer 1... l The characteristics of the layer output are: ; in, For average pooling layer, It is a multilayer perceptron. This is a concatenation operation along the feature dimensions. For VectorNet Layer output.

[0007] Furthermore, the spatiotemporal multi-agent interaction module uses the MGNN multi-layer spatial interaction network to perform spatial interaction on historical spatiotemporal motion features, capturing the spatial influence relationship between motion features of vehicles. It also uses the Mamba module to perform temporal interaction on historical spatiotemporal motion features, capturing the temporal influence relationship. The module uses two cross-attention methods to perform cross-attention on the spatial interaction results and the temporal interaction results respectively, extracting the spatiotemporal motion interaction features that influence the planning. Finally, it uses a fusion gate to fuse the spatial motion interaction features and temporal motion interaction features that influence the planning, thus obtaining the spatiotemporal motion interaction features that influence the planning.

[0008] Furthermore, multi-head attention uses the vehicle's spatiotemporal motion interaction features as the query and map features as the key and value to achieve the interaction between the vehicle's spatiotemporal motion interaction features and map features, thereby obtaining trajectory features: , ; in, , , For multi-head attention queries, the linear transformation weight matrix of keys and values ​​is used. For single-head attention, the feature dimension satisfies , For the number of attention heads, For the first Spatiotemporal motion interaction characteristics of individual traffic participants For the first Trajectory characteristics of individual traffic participants.

[0009] Furthermore, the Transformer-based initial planning module performs inter-vehicle interaction modeling on the trajectory features, including historical and predicted trajectories, in the temporal dimension. Based on the inter-vehicle interaction modeling, a temporal Transformer encoder is constructed to complete long-term temporal dependency modeling, resulting in new trajectory features. The new trajectory features after temporal interaction are used as queries, and map features are used as keys and values ​​to perform spatial-dimensional Transformer calculations to obtain the vehicle's final trajectory features. The vehicle's final trajectory features are used by a motion decoder to generate multimodal trajectories and corresponding probabilities. The trajectory with the highest probability is output as the initial planned trajectory, and its corresponding trajectory features are passed to downstream modules.

[0010] Furthermore, in the initial planning module's modeling of vehicle trajectory features, the target vehicle's trajectory features are used as the query, and the trajectory features of traffic participants within the scene are used as keys and values ​​for interaction modeling, resulting in the vehicle-to-vehicle interaction trajectory features. : ,

[0011] in, The query, key, and value weight matrix is ​​used for modeling the interaction between vehicles in the initial planning module. These represent the trajectory features of the target vehicle at time t and the trajectory features of traffic participants within the scene, respectively. Subsequently, a new trajectory feature is obtained by performing temporal dimension modeling on the trajectory features after modeling the interaction between vehicles using a temporal Transformer: , ; in, Query, key, and value weight matrices for time series modeling; Finally, using the new trajectory features as the query and the map features as the key and value, a Transformer computation is performed in the spatial dimension to obtain the vehicle's final trajectory features: , ; in, This is a matrix of query, key, and value weights for interacting with a map. For map features; The final trajectory features are used to generate multimodal trajectories and corresponding probabilities through a motion decoder. The trajectory with the highest probability is output as the initial planned trajectory, and its corresponding trajectory features are passed to the downstream module.

[0012] Furthermore, the fine planning module based on hierarchical trajectory priors introduces prior knowledge of human driving behavior contained in large-scale real-world driving data into trajectory planning through a two-level hierarchical filtering mechanism, compensating for the lack of strong prior information in scenarios without high-precision maps; wherein, the two-level hierarchical filtering mechanism specifically includes: (1) Scene-level coarse screening: Using the scene topology information provided by the lightweight map as input, a multilayer perceptron is used to upgrade a set number of prior trajectories to obtain prior trajectory features and construct a prior library; a lightweight attention is used to interact the prior trajectory features with the scene topology information of the lightweight map, and through cosine similarity calculation, the top-256 highly similar prior trajectories that are consistent with the current map scene behavior type are matched from the prior library to form a first-level candidate set, and prior trajectories that exceed the feasible domain or conflict with the scene topology are filtered out; (2) Behavioral-level fine screening: The 256 prior trajectory features in the primary candidate set are matched with the trajectory features of the interaction between passing vehicles and between the vehicle map through multi-head attention, and the Top-9 prior trajectories that conform to the spatiotemporal relationship of the current traffic participants and the behavioral intention of the main vehicle are selected as the final prior trajectory set. The nine prior trajectory features in the final prior trajectory set are fused with the interacted trajectory features using multi-head attention: , in, Representing multi-head attention, the three elements are query, key, and value. The trajectory features are used as the final input to the motion decoder.

[0013] Furthermore, except for the final fine planning module, none of the modules use a motion decoder to decode the trajectory features into two-dimensional coordinates representing the trajectory. Instead, the features are directly passed to the next module. Finally, a unified motion decoder completes the trajectory decoding output of the trajectory features predicted by the multimodal trajectory re-prediction module and the trajectory features planned by the fine planning module. The output is then optimized through end-to-end multi-task joint training. The multi-task joint loss function includes three types: trajectory imitation learning loss, interactive obstacle avoidance loss, and prior trajectory selection loss, which are weighted and summed to achieve end-to-end joint optimization. Among them, the planned trajectory imitation learning loss is based on negative log-likelihood loss, which guides the planned trajectory to conform to the motion pattern of real driving data; the interaction obstacle avoidance loss calculates the L2 distance between the vehicle's planning pattern and other traffic participants, and is activated only when the distance is less than the safety threshold, imposing a repulsive force penalty on close-range interaction pairs; the prior trajectory screening loss is designed for the two-level screening process, and soft and hard cross-entropy loss is designed to enhance the model's ability to learn and match hierarchical prior trajectories. The soft and hard cross-entropy loss includes: cross-entropy loss combining hard labels and soft labels dynamically generated based on distance, negative log-likelihood classification loss, and regression loss.

[0014] Secondly, the present invention provides an interactive joint prediction and planning device based on hierarchical trajectory priors without high-precision maps, comprising: at least one processing unit, the processing unit being connected to a storage unit via a bus unit, the storage unit storing a computer program, and the processing unit implementing the interactive joint prediction and planning method based on hierarchical trajectory priors without high-precision maps by running the computer program stored in the storage unit.

[0015] The technical solutions provided in the embodiments of the present invention have the following advantages compared with the prior art: This invention constructs a closed-loop joint architecture of prediction, initial planning, re-prediction, and fine planning through a multimodal initial trajectory prediction module, an initial planning module, a multimodal trajectory re-prediction module, and a fine planning module based on hierarchical trajectory priors. This breaks the unidirectional limitations of the traditional sequential structure of prediction followed by planning, achieving deep coupling and joint optimization of prediction and planning. This enables the model to model the bidirectional dynamic interaction between the vehicle and traffic participants, improving the rationality and safety of behavioral decisions in complex dynamic multi-vehicle interaction scenarios. Simultaneously, through a hierarchical trajectory prior mechanism, this invention injects prior knowledge of human driving behavior contained in large-scale real-world driving trajectories into the trajectory planning process using a two-level matching method: scenario-level coarse screening and behavior-level fine screening. This provides the model with high-quality prior information with low computational burden, effectively compensating for the lack of strong prior information in scenarios without high-precision maps. While ensuring real-time planning, it significantly improves the anthropomorphism, rationality, and robustness of the planned trajectory, making it more suitable for the real-time deployment requirements of autonomous driving vehicles. Attached Figure Description

[0016] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A flowchart of a joint prediction and planning method for interaction without high-precision maps based on hierarchical trajectory priors provided in an embodiment of the present invention; Figure 2 This is an architecture diagram of a joint prediction planning system without high-precision map interaction based on hierarchical trajectory priors, provided in an embodiment of the present invention. Figure 3 A schematic diagram of a lightweight map multi-layer spatiotemporal interaction prediction network provided in an embodiment of the present invention; Figure 4 A schematic diagram of the initial planning module provided in an embodiment of the present invention; Figure 5 A schematic diagram of the fine planning module provided in an embodiment of the present invention; Figure 6 This is a schematic diagram of a high-precision map-free interactive joint prediction and planning device based on hierarchical trajectory priors provided in an embodiment of the present invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0021] Example 1 like Figure 1 and Figure 2 As shown, this invention proposes a hierarchical trajectory prior-based interactive joint prediction and planning method without high-precision maps. It aims to address the problems of strong map dependence between prediction and planning modules, insufficient interactive modeling in a serial architecture, and poor trajectory rationality without prior constraints in scenarios without high-precision maps. The method constructs a closed-loop joint architecture of prediction, initial planning, re-prediction, and fine planning to achieve deep coupling and joint optimization of prediction and planning modules. Specifically, the technical solution of this invention includes the following steps: Step 1: Obtain multimodal environmental perception data from a bird's-eye view through the vehicle's sensing terminal. The environmental perception data includes the historical trajectory and location status of traffic participants, as well as a lightweight map of the road scene. The environmental perception data is used to guide planning.

[0022] In the specific implementation process, the vehicle's perception end includes sensors such as forward-facing cameras, surround-view cameras, and LiDAR. During driving, it collects relevant data on the road scene and environment in real time. For example, LiDAR collects and records the positions of traffic participants from a bird's-eye view to form the historical trajectory of traffic participants, which includes position and heading; surround-view cameras and forward-facing cameras collect road images and convert them to a bird's-eye view to obtain a lightweight map of the corresponding road scene; surround-view cameras and forward-facing cameras collect images of traffic participants, and LiDAR collects the speed, acceleration, and position of traffic participants, converting them to a bird's-eye view to obtain the position, speed, and acceleration status from the bird's-eye view.

[0023] Step 2: The multimodal initial trajectory prediction module based on the lightweight map multi-layer spatiotemporal interaction prediction network performs preliminary trajectory planning based on the collected multimodal environmental perception data, and predicts the multimodal initial trajectory features of surrounding traffic participants.

[0024] The multimodal initial trajectory prediction module employs a lightweight map-based multi-layer spatiotemporal interactive prediction network; such as Figure 3 As shown, the lightweight map multi-layer spatiotemporal interaction prediction network includes: The vectorized map encoder branch extracts map features from the light map for each traffic participant.

[0025] In the specific implementation process, the lane, intersection, and topology information in the light map are vectorized. Vectorization includes semantic encoding of lane segments and recording the coordinates of the start and end positions of lane segments. The semantic encoding, start and end position coordinates of lane segments form the vectorized representation of lanes.

[0026] The vector map encoder branch first encodes the vectorized lightweight map through a multilayer perceptron to obtain low-order map features. Then, the low-order map features are up-dimensionalized through stacked VectorNet to obtain map features. VectorNet is a variant of graph neural networks in the field of autonomous driving.

[0027] Specifically, taking feature extraction from the vectorized representation of lanes as an example: the lane multilayer perceptron in the vectorized map encoder extracts corresponding low-order lane features for traffic participants based on the rotation matrix of the local region of the traffic participant and the vectorized representation of the lane. , in, It is used in lane multi-layer sensor encoders. Traffic participants Rotation matrix of local region , These are the coordinates of the start and end positions of the lane segment. It is a lane section The semantic attributes, such as sidewalk, motor vehicle lane, non-motor vehicle lane, straight ahead, left turn, stop line attributes; The origin of the local coordinate system. Traffic participants. The rotation matrix of the local region transforms the lane segment coordinates to the vehicle's local coordinate system, making the lane features translation and rotation invariant and enhancing the model's generalization ability under different intersection orientations. The multilayer perceptron encoder of the lane simultaneously inputs the start and end point coordinates and semantic attributes, ensuring the simultaneous injection of static topology and traffic rules.

[0028] The encoded low-level features require further processing to capture the map's topological structure, such as the connectivity between lanes. This application utilizes stacking... The VectorNet layer upscales low-order lane features, where the VectorNet layer 1... l The map features output by the layer are: ; in, For average pooling layer, It is a multilayer perceptron. This is a concatenation operation along the feature dimensions. For VectorNet Layer output. Neighborhood information is aggregated using average pooling and then mapped to a higher dimension through a multilayer perceptron. The symmetric aggregation method ensures insensitivity to the order of input map points.

[0029] The trajectory / lane context interaction branch organizes the trajectory segments and lane relationships of the time series; the historical agent feature branch organizes the state of the surrounding traffic participants of the time series; the trajectory / lane context interaction branch and the historical agent feature branch jointly provide the organized time series to the spatiotemporal historical trajectory encoding module, which transforms the original, multi-source historical data into high-dimensional features that the model can understand.

[0030] The spatiotemporal historical trajectory encoding module receives heterogeneous trajectory / lane context time-series data and agent feature time-series data as input. The trajectory / lane context time-series data contains the relationship between the trajectory and the lane, while the historical agent features contain the absolute or relative motion states of surrounding vehicles. The spatiotemporal historical trajectory encoding module maps these two heterogeneous data streams of different dimensions into the same high-dimensional feature space, enabling the heterogeneous data to be uniformly processed by subsequent neural networks. The features extracted by the spatiotemporal historical trajectory encoding module contain historical spatiotemporal motion patterns, capturing the past motion patterns of traffic participants (the vehicle itself and surrounding vehicles).

[0031] The historical spatiotemporal motion feature vector encoded by the spatiotemporal historical trajectory encoding module is fed into the spatiotemporal multi-agent interaction module. The spatiotemporal multi-agent interaction module uses a multi-layered MGNN spatial interaction network to perform spatial interaction on the historical spatiotemporal motion features, capturing the spatial influence relationships between vehicle motion features. It also uses a Mamba module to perform temporal interaction on the historical spatiotemporal motion features, capturing temporal influence relationships. Two cross-attention mechanisms are used to cross-attention the spatial and temporal interaction results respectively, extracting the spatiotemporal motion interaction features that influence planning. A fusion gate is then used to fuse the spatial and temporal motion interaction features that influence planning, resulting in the spatiotemporal motion interaction features that influence planning. Spatial and temporal features are not simply added together; instead, bidirectional information permeation is first achieved through cross-attention, and then the weights of both are adaptively adjusted through a learnable fusion gate, achieving true spatiotemporal feature entanglement.

[0032] The spatiotemporal motion interaction features and map features that influence the planning are processed through multi-head attention. Multi-head attention uses the vehicle's spatiotemporal motion interaction features as queries and map features as keys and values ​​to achieve the interaction between the vehicle's spatiotemporal motion interaction features and map features. , ; in, , , For multi-head attention queries, the linear transformation weight matrix of keys and values ​​is used. For single-head attention, the feature dimension satisfies , For the number of attention heads, For the first Spatiotemporal motion interaction characteristics of individual traffic participants For the first Trajectory characteristics of individual traffic participants.

[0033] The final trajectory features of traffic participants can be decoded by a motion decoder to obtain the trajectory.

[0034] Step 3: Using the multimodal initial trajectory features of surrounding traffic participants as input, the initial trajectory features of the vehicle are extracted through the Transformer-based initial planning module to obtain the vehicle's future motion intention and reference trajectory features.

[0035] The Transformer-based initial planning module performs inter-vehicle interaction modeling on the trajectory features, including historical and predicted trajectories, in the temporal dimension. Based on the inter-vehicle interaction modeling, a temporal Transformer encoder is constructed to complete long-term temporal dependency modeling, resulting in new trajectory features. The new trajectory features after temporal interaction are used as queries, and map features are used as keys and values ​​to perform Transformer calculations in the spatial dimension to obtain the final trajectory features of the vehicle. The final trajectory features of the vehicle are used by a motion decoder to generate multimodal trajectories and corresponding probabilities. The trajectory with the highest probability is output as the initial planned trajectory, and its corresponding trajectory features are passed to the downstream module.

[0036] Specifically, such as Figure 4 As shown, in the initial planning module's modeling of vehicle trajectory features, the target vehicle's trajectory features are used as the query, and the trajectory features of traffic participants within the scene are used as keys and values ​​for interaction modeling, resulting in the vehicle-to-vehicle interaction trajectory features. : ,

[0037] in, The query, key, and value weight matrix is ​​used for modeling the interactions between vehicles in the initial planning module. These represent the trajectory features of the target vehicle at time t and the trajectory features of traffic participants within the scene, respectively. Subsequently, a new trajectory feature is obtained by performing temporal dimension modeling on the trajectory features after modeling the interaction between vehicles using a temporal Transformer: , ; in, Query, key, and value weight matrices for time series modeling; Finally, using the new trajectory features as the query and the map features as the key and value, a Transformer computation is performed in the spatial dimension to obtain the vehicle's final trajectory features: , ; in, This is a matrix of query, key, and value weights for interacting with a map. For map features; The final trajectory features of the vehicle support the generation of multimodal trajectories and corresponding probabilities through a motion decoder, and the trajectory with the highest probability is output as the initial planned trajectory, and its corresponding trajectory features are passed to downstream modules.

[0038] Step 4: Input the motion intent and reference trajectory features obtained from the initial planning of the vehicle into the multimodal trajectory re-prediction module based on the multi-layer spatiotemporal interaction prediction network of the lightweight map to perform two-way interaction modeling between the vehicle and surrounding participants, perform closed-loop interaction re-prediction, and obtain refined multimodal re-predicted trajectory features of traffic participants that conform to the real interaction logic.

[0039] The multimodal trajectory re-prediction module is also based on the lightweight map multi-layer spatiotemporal interactive prediction network. The re-prediction stage receives the future trajectory features of the scene vehicles output by the prediction and initial planning modules. It only updates the trajectory features of the predicted vehicles to achieve closed-loop interactive re-prediction that takes into account the planned trajectory of the vehicle.

[0040] Step 5: Based on the multimodal re-predicted trajectory features of traffic participants, the vehicle's final safe and executable control trajectory is output by a unified motion decoder through a fine planning module based on hierarchical trajectory priors and a two-level prior matching mechanism consisting of scenario-level coarse screening and behavior-level fine screening.

[0041] The network architecture of the fine planning module is similar to that of the initial planning module, and a hierarchical trajectory prior is introduced.

[0042] The fine planning module based on hierarchical trajectory priors, such as... Figure 5 As shown, the final output of the entire joint architecture decodes trajectory feature information into a two-dimensional executable trajectory. The fine planning module introduces prior knowledge of human driving behavior contained in large-scale real-world driving data into the trajectory planning process through a two-level hierarchical filtering mechanism, compensating for the lack of strong prior information in scenarios without high-precision maps. The two-level hierarchical filtering mechanism specifically includes: (1) Scene-level coarse screening: Using the scene topology information provided by the lightweight map as input, the 4096 prior trajectories are upgraded using a multilayer perceptron to obtain prior trajectory features and construct a prior library; a lightweight attention module is used to interact the prior trajectory features with the scene topology information of the lightweight map, and the top-256 highly similar prior trajectories that are consistent with the current map scene behavior type are matched from the prior library through cosine similarity calculation to form a first-level candidate set, filtering out prior trajectories that exceed the feasible domain or conflict with the scene topology, thereby reducing the subsequent computational complexity; (2) Behavioral-level fine screening: The 256 prior trajectory features in the primary candidate set are matched with the trajectory features that have undergone vehicle-to-vehicle and vehicle-to-map interactions through multi-head attention to screen out the Top-9 prior trajectories that conform to the spatiotemporal relationship of the current traffic participants and the behavioral intention of the main vehicle, which are used as the final prior trajectory set; The nine prior trajectory features in the final prior trajectory set are fused with the interactive vehicle trajectory features using multi-head attention: ; in, Representing multi-head attention, the three elements are query, key, and value. The trajectory features are used as the final input to the motion decoder. This mechanism injects hierarchical trajectory priors into the trajectory planning process, ensuring that the planned trajectory not only conforms to human driving habits but also adapts to real-time traffic interactions.

[0043] In this invention, except for the final fine planning module, none of the modules use a motion decoder to decode feature information into two-dimensional coordinate information. Instead, the feature information is directly passed to the next module. Finally, a unified motion decoder decodes and outputs the trajectory features predicted by the re-prediction module and planned by the fine planning module, thus eliminating the information loss caused by continuous encoding and decoding between modules in the traditional serial structure. The prediction and planning tasks are completed simultaneously through end-to-end joint optimization. This invention constructs a multi-task joint loss function, including three types: trajectory imitation learning loss, obstacle avoidance loss, and prior trajectory selection loss. End-to-end joint optimization is achieved through weighted summation. Specifically, the trajectory imitation learning loss is based on negative log-likelihood loss, guiding the planned trajectory to conform to the motion pattern of real driving data; the obstacle avoidance loss calculates the relationship between the vehicle's planning pattern and other traffic participants. Distance, only when the distance is less than the safety threshold The model is activated at specific times and a repulsive penalty is applied to close-range interactions. The prior trajectory screening loss is designed with soft and hard cross-entropy loss for the two-level screening process. The soft and hard cross-entropy loss includes: a cross-entropy loss combining hard labels and soft labels dynamically generated based on distance for the first-level coarse screening, a negative log-likelihood classification loss for the second-level fine screening, and a regression loss for the positional accuracy of the final prior trajectory, so as to enhance the model's ability to learn and match hierarchical prior trajectories.

[0044] Example 2 like Figure 6As shown, this embodiment of the invention provides an interactive joint prediction and planning device based on hierarchical trajectory priors without high-precision maps, comprising: at least one processing unit, the processing unit being connected to a storage unit via a bus unit, the storage unit serving as a computer-readable storage medium for storing software programs, computer-executable programs, and modules, such as the software program, computer-executable program, and module corresponding to the interactive joint prediction and planning method based on hierarchical trajectory priors without high-precision maps in this embodiment of the invention. The processing unit implements the aforementioned interactive joint prediction and planning method based on hierarchical trajectory priors without high-precision maps by running the software program, computer-executable program, and module stored in the storage unit, including: Acquire multimodal environmental perception data from a bird's-eye view, including the historical trajectories and states of traffic participants, as well as a lightweight map of the road scene they are in; The multimodal initial trajectory prediction module based on the lightweight map multi-layer spatiotemporal interaction prediction network predicts the multimodal initial trajectory features of traffic participants based on the collected multimodal environmental perception data. Using the multimodal initial trajectory features of traffic participants as input, the initial trajectory features of the vehicle are extracted through the Transformer-based initial planning module to obtain the vehicle's future motion intention and reference trajectory features; The vehicle's initial planning motion intent and reference trajectory features are input into the multimodal trajectory re-prediction module based on the lightweight map multi-layer spatiotemporal interaction prediction network to perform two-way interaction modeling between the vehicle and surrounding participants, and closed-loop interaction re-prediction to obtain refined multimodal re-predicted trajectory features of traffic participants that conform to the real interaction logic. Based on the multimodal re-predicted trajectory features of traffic participants, and through a fine planning module based on hierarchical trajectory priors, under a two-level prior matching mechanism consisting of scenario-level coarse screening and behavior-level fine screening, a unified motion decoder outputs the final safe and executable control trajectory of the vehicle.

[0045] Of course, the computer program stored in the storage unit of the hierarchical trajectory prior-based interactive joint prediction planning system without high-precision map provided in the embodiments of the present invention is not limited to the method operation described above, and can also execute related operations in the hierarchical trajectory prior-based interactive joint prediction planning method without high-precision map provided in any embodiment of the present invention.

[0046] Example 3 This invention provides a computer-readable storage medium storing a computer program. When executed, the computer program implements the hierarchical trajectory prior-based interactive joint prediction and planning method without high-precision maps, comprising: Acquire multimodal environmental perception data from a bird's-eye view, including the historical trajectories and states of traffic participants, as well as a lightweight map of the road scene they are in; The multimodal initial trajectory prediction module based on the lightweight map multi-layer spatiotemporal interaction prediction network predicts the multimodal initial trajectory features of traffic participants based on the collected multimodal environmental perception data. Using the multimodal initial trajectory features of traffic participants as input, the initial trajectory features of the vehicle are extracted through the Transformer-based initial planning module to obtain the vehicle's future motion intention and reference trajectory features; The vehicle's initial planning motion intent and reference trajectory features are input into the multimodal trajectory re-prediction module based on the lightweight map multi-layer spatiotemporal interaction prediction network to perform two-way interaction modeling between the vehicle and surrounding participants, and closed-loop interaction re-prediction to obtain refined multimodal re-predicted trajectory features of traffic participants that conform to the real interaction logic. Based on the multimodal re-predicted trajectory features of traffic participants, and through a fine planning module based on hierarchical trajectory priors, under a two-level prior matching mechanism consisting of scenario-level coarse screening and behavior-level fine screening, a unified motion decoder outputs the final safe and executable control trajectory of the vehicle.

[0047] In the embodiments provided by this invention, it should be understood that the disclosed structures and methods can be implemented in other ways. For example, the structural embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interfaces, structures, or units, and may be electrical, mechanical, or other forms.

[0048] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0049] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0050] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A hierarchical trajectory prior-based interactive joint prediction planning method without high-precision maps, characterized in that, include: Acquire multimodal environmental perception data from a bird's-eye view, including the historical trajectories and states of traffic participants, as well as a lightweight map of the road scene they are in; The multimodal initial trajectory prediction module based on the lightweight map multi-layer spatiotemporal interaction prediction network predicts the multimodal initial trajectory features of surrounding traffic participants based on the collected multimodal environmental perception data. Using the multimodal initial trajectory features of surrounding traffic participants as input, the initial trajectory features of the vehicle are extracted through the Transformer-based initial planning module to obtain the vehicle's future motion intention and reference trajectory features. The vehicle's initial planning motion intent and reference trajectory features are input into the multimodal trajectory re-prediction module based on the lightweight map multi-layer spatiotemporal interaction prediction network to perform two-way interaction modeling and closed-loop interaction re-prediction between the vehicle and surrounding traffic participants, resulting in refined multimodal re-predicted trajectory features of traffic participants that conform to real interaction logic. Based on the refined multimodal re-predicted trajectory features of traffic participants, a hierarchical trajectory prior-based fine planning module, under a two-level prior matching mechanism consisting of scenario-level coarse screening and behavior-level fine screening, outputs the final safe and executable control trajectory of the vehicle by a unified motion decoder. The hierarchical trajectory prior-based fine planning module introduces the prior knowledge of human driving behavior contained in large-scale real-world driving data into trajectory planning through a two-level hierarchical screening mechanism, compensating for the lack of strong prior information in scenarios without high-precision maps. Specifically, the two-level hierarchical screening mechanism includes: (1) Scene-level coarse screening: Using the scene topology information provided by the lightweight map as input, a multilayer perceptron is used to upgrade a set number of prior trajectories to obtain prior trajectory features and construct a prior library; a lightweight attention is used to interact the prior trajectory features with the scene topology information of the lightweight map, and through cosine similarity calculation, the top-256 highly similar prior trajectories that are consistent with the current map scene behavior type are matched from the prior library to form a first-level candidate set, and prior trajectories that exceed the feasible domain or conflict with the scene topology are filtered out; (2) Behavioral-level fine screening: The 256 prior trajectory features in the primary candidate set are matched with the trajectory features of the interaction between passing vehicles and between the vehicle map through multi-head attention, and the Top-9 prior trajectories that conform to the spatiotemporal relationship of the current traffic participants and the behavioral intention of the main vehicle are selected as the final prior trajectory set. The nine prior trajectory features in the final prior trajectory set are combined with the trajectory features after interaction. Fusion through multi-head attention: ， in, Representing multi-head attention, the three elements are query, key, and value. The trajectory features are used as the final input to the motion decoder.

2. The method for joint prediction and planning without high-precision maps based on hierarchical trajectory priors according to claim 1, characterized in that, The lightweight map multi-layer spatiotemporal interaction prediction network includes: The vector map encoder branch extracts map features from the light map for each traffic participant. The vector map encoder branch first encodes the vector light map through a multilayer perceptron to obtain low-order map features, and then uses a stacked VectorNet to upscale the low-order map features to obtain map features. The trajectory / lane context interaction branch organizes the trajectory segments and lane relationships of the time series; the historical agent feature branch organizes the state of the surrounding traffic participants of the time series; the trajectory / lane context interaction branch and the historical agent feature branch jointly provide the organized time series to the spatiotemporal historical trajectory encoding module, which encodes the heterogeneous time series into a unified historical spatiotemporal motion feature vector. The historical spatiotemporal motion feature vector encoded by the spatiotemporal historical trajectory encoding module is sent to the spatiotemporal multi-agent interaction module to obtain the spatiotemporal motion interaction features that affect the planning. The spatiotemporal motion interaction features and map features that influence the planning are processed through multi-head attention to obtain the trajectory features of traffic participants; among them, the trajectory features of traffic participants can be decoded by the motion decoder to obtain the trajectory.

3. The method for joint prediction and planning without high-precision maps based on hierarchical trajectory priors according to claim 2, characterized in that, The vectorized map encoder branch includes vectorizing lane, intersection, and topology information in the light map; vectorization includes semantic encoding of lane segments and recording the start and end coordinates of lane segments. The semantic encoding, start and end coordinates of lane segments form a vectorized representation. The lane multilayer perceptron in the vector map encoder extracts corresponding low-order lane features for traffic participants based on the rotation matrix of the local region and the vectorized representation of the lane. ， in, It is a multi-layer sensor encoder used for lanes. Traffic participants Rotation matrix of local region , These are the coordinates of the start and end positions of the lane segment. It is a lane section semantic attributes; The origin of the local coordinate system; By stacking The VectorNet layer upscales low-order lane features, where the VectorNet layer 1... l The map features output by the layer are: ； in, For average pooling layer, It is a multilayer perceptron. This is a concatenation operation along the feature dimensions. For VectorNet Layer output.

4. The method for joint prediction and planning without high-precision maps based on hierarchical trajectory priors according to claim 2, characterized in that, The spatiotemporal multi-agent interaction module uses the MGNN multi-layer spatial interaction network to perform spatial interaction on historical spatiotemporal motion features, capturing the spatial influence relationship between motion features of vehicles. It also uses the Mamba module to perform temporal interaction on historical spatiotemporal motion features, capturing the temporal influence relationship. The module uses two cross-attention methods to perform cross-attention on the spatial interaction results and the temporal interaction results respectively, extracting the spatiotemporal motion interaction features that influence the planning. Finally, the module uses a fusion gate to fuse the spatial motion interaction features and temporal motion interaction features that influence the planning, thus obtaining the spatiotemporal motion interaction features that influence the planning.

5. The method for joint prediction and planning without high-precision maps based on hierarchical trajectory priors according to claim 2, characterized in that, Multi-head attention uses the vehicle's spatiotemporal motion interaction features as the query and map features as the key and value to perform attention, thereby achieving the interaction between the vehicle's spatiotemporal motion interaction features and map features to obtain trajectory features: ， ； in, , , For multi-head attention queries, the linear transformation weight matrix of keys and values ​​is used. For single-head attention, the feature dimension satisfies , For the number of attention heads, For the first Spatiotemporal motion interaction characteristics of individual traffic participants For the first Trajectory characteristics of individual traffic participants.

6. The method for joint prediction and planning without high-precision maps based on hierarchical trajectory priors according to claim 1, characterized in that, The Transformer-based initial planning module performs vehicle-to-vehicle interaction modeling on the time-series dimension using trajectory features including historical and predicted trajectories. Based on the vehicle-to-vehicle interaction modeling, a time-series Transformer encoder is constructed to complete long-time-series dependency modeling, resulting in new trajectory features. The new trajectory features after time-series interaction are used as queries, and map features are used as keys and values ​​to perform spatial-dimensional Transformer calculations to obtain the vehicle's final trajectory features. The final trajectory features of the vehicle are used to generate multimodal trajectories and corresponding probabilities through a motion decoder. The trajectory with the highest probability is then used as the initial planned trajectory output, and its corresponding trajectory features are passed to downstream modules.

7. The method for joint prediction and planning without high-precision maps based on hierarchical trajectory priors according to claim 6, characterized in that, In the initial planning module's modeling of vehicle trajectory features, the target vehicle's trajectory features are used as the query, and the trajectory features of traffic participants within the scene are used as keys and values ​​for interaction modeling, resulting in the vehicle-to-vehicle interaction trajectory features. : ， in, The query, key, and value weight matrix is ​​used for modeling the interactions between vehicles in the initial planning module. These represent the trajectory features of the target vehicle at time t and the trajectory features of traffic participants within the scene, respectively. Subsequently, a new trajectory feature is obtained by performing temporal dimension modeling on the trajectory features after modeling the interaction between vehicles using a temporal Transformer: ， ； in, Query, key, and value weight matrices for time series modeling; Finally, using the new trajectory features as the query and the map features as the key and value, a Transformer computation is performed in the spatial dimension to obtain the vehicle's final trajectory features: ， ； in, This is a matrix of query, key, and value weights for interacting with a map. For map features; The final trajectory features are used to generate multimodal trajectories and corresponding probabilities through a motion decoder. The trajectory with the highest probability is output as the initial planned trajectory, and its corresponding trajectory features are passed to the downstream module.

8. The method for joint prediction and planning without high-precision map interaction based on hierarchical trajectory priors according to claim 1, characterized in that, Except for the final fine planning module, none of the modules use a motion decoder to decode trajectory features into two-dimensional coordinates representing the trajectory. Instead, the features are directly passed to the next module. Finally, a unified motion decoder completes the trajectory decoding output of the trajectory features predicted by the multimodal trajectory re-prediction module and the trajectory features planned by the fine planning module. The output is then optimized through end-to-end multi-task joint training. The multi-task joint loss function includes three types: trajectory imitation learning loss, interactive obstacle avoidance loss, and prior trajectory selection loss, which are weighted and summed to achieve end-to-end joint optimization. Among them, the planned trajectory imitation learning loss is based on negative log-likelihood loss, which guides the planned trajectory to conform to the motion pattern of real driving data; the interaction obstacle avoidance loss calculates the L2 distance between the vehicle's planning pattern and other traffic participants, and is activated only when the distance is less than the safety threshold, imposing a repulsive force penalty on close-range interaction pairs; the prior trajectory screening loss is designed for the two-level screening process, and soft and hard cross-entropy loss is designed to enhance the model's ability to learn and match hierarchical prior trajectories. The soft and hard cross-entropy loss includes: cross-entropy loss combining hard labels and soft labels dynamically generated based on distance, negative log-likelihood classification loss, and regression loss.

9. A joint prediction and planning device based on hierarchical trajectory priors without high-precision maps, comprising: At least one processing unit, the processing unit being connected to a storage unit via a bus unit, characterized in that the storage unit stores a computer program, and the processing unit implements the interactive joint prediction planning method based on hierarchical trajectory priors without high-precision maps as described in any one of claims 1-8 by running the computer program stored in the storage unit.

Images