Multimodal trajectory prediction model, training method and device thereof, and autonomous vehicle

Abstract

The present disclosure provides a multimodal trajectory prediction model and a training method and device thereof and an autonomous vehicle, relates to the fields of artificial intelligence, large models and autonomous driving, and also relates to multimodal reweighting technology. The specific implementation scheme is: training the multimodal trajectory prediction model through imitation learning and reinforcement learning; wherein, the multimodal probability distribution and the weight predicted by the model are obtained in the imitation learning stage, in the reinforcement learning stage, the value of the reward function of each modality is determined based on the differentiable Euclidean signed distance field (ESDF) distance from the predicted trajectory point on each modality to the boundary of the impassable area, and the multimodal probability distribution in the imitation learning stage is reweighted based on the value of the reward function of each modality to update the weight of the predicted trajectory of each modality.

Description

Technical Field

[0001] This disclosure relates to the fields of artificial intelligence, large models and autonomous driving technology, and also to multimodal reweighting technology, specifically to a multimodal trajectory prediction model and its training method, device, electronic device, autonomous vehicle, storage medium and program product. Background Technology

[0002] In autonomous driving systems, trajectory prediction probabilistically predicts the future movement trajectories of surrounding traffic participants, and the results influence downstream path planning and decision-making. Current trajectory prediction solutions still face the problem of predicted trajectories potentially crossing physical barriers (such as guardrails or median strips), resulting in physically infeasible paths. Summary of the Invention

[0003] This disclosure provides a multimodal trajectory prediction model and its training method, apparatus, equipment, autonomous vehicle, storage medium, and program product.

[0004] According to one aspect of this disclosure, a training method for a multimodal trajectory prediction model is provided, comprising: training the multimodal trajectory prediction model through imitation learning and reinforcement learning; wherein, in the imitation learning stage, the multimodal probability distribution and weights predicted by the model are obtained; in the reinforcement learning stage, the value of the reward function of each mode is determined based on the differentiable Euclidean signed distance field (ESDF) from the predicted trajectory points on each mode to the boundary of the impassable region; and the multimodal probability distribution in the imitation learning stage is reweighted based on the value of the reward function of each mode to update the weights of the predicted trajectories of each mode.

[0005] According to another aspect of this disclosure, a multimodal trajectory prediction device is provided, comprising: a multimodal Gaussian mixture prediction module, used to output an initial multimodal trajectory probability distribution and weights based on input data; an ESDF distance query module, used to construct an ESDF grid based on map information and determine the differentiable ESDF distance values ​​from trajectory points on each modal trajectory to the boundary of an impassable area using a bilinear interpolation method; a penalty value calculation module, used to calculate the penalty value and reward value for each modality based on the ESDF distance values; and a probability reweighting module, used to calculate new multimodal weights based on the reward value and the reweighting formula. The initial multimodal trajectory probability distribution is reweighted; the result output module is used to output one or more predicted trajectories based on the reweighted multimodal predicted trajectory.

[0006] According to another aspect of this disclosure, a training apparatus for a multimodal trajectory prediction model is provided, comprising: an imitation learning training module and a reinforcement learning training module, for training the multimodal trajectory prediction model; wherein, the imitation learning training module is used to obtain the multimodal probability distribution and weights predicted by the model, and the reinforcement learning module is used to determine the value of the reward function of each mode based on the differentiable Euclidean signed distance field (ESDF) from the predicted trajectory points on each mode to the boundary of the impassable region, and to reweight the multimodal probability distribution during the imitation learning stage based on the value of the reward function of each mode, so as to update the weights of the predicted trajectories of each mode.

[0007] According to another aspect of this disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform a training method for a multimodal trajectory prediction model as described above.

[0008] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause the computer to perform the training method of the multimodal trajectory prediction model as described above.

[0009] According to another aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the training method for the multimodal trajectory prediction model as described above.

[0010] According to another aspect of this disclosure, an autonomous vehicle is provided, which includes the electronic devices as described above.

[0011] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0012] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein: Figure 1 This is a schematic diagram of the multimodal trajectory prediction model training method according to an embodiment of the present disclosure; Figure 2 This is a schematic block diagram of a multimodal trajectory prediction device according to an embodiment of the present disclosure; Figure 3 This is a schematic diagram of the multimodal trajectory prediction model architecture and two-stage training process according to an embodiment of this disclosure; Figure 4 and Figure 5These are schematic diagrams illustrating the imitation learning and reinforcement learning processes according to embodiments of this disclosure; Figure 6 This is a schematic diagram of the architecture of the multimodal trajectory prediction model according to an embodiment of the present disclosure; Figure 7 and Figure 8 Schematic diagrams of the ESDF distance field and penalty function according to embodiments of this disclosure are shown respectively; Figure 9 and Figure 10 The diagrams show a comparison of the multimodal probabilities before and after reweighting in the embodiments of this disclosure. Figure 11 This is a schematic diagram of a multimodal trajectory prediction device according to an embodiment of the present disclosure; Figure 12 This is a block diagram of an electronic device used to implement the training method of the multimodal trajectory prediction model according to the embodiments of this disclosure. Detailed Implementation

[0013] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description. Terms such as first, second, third, etc., used herein are used only to distinguish one entity (or operation) from another, and are not intended to require or imply a specific order or association between these entities (or operations).

[0014] Before detailing the embodiments of this disclosure, a brief overview of the typical trajectory prediction process will be provided.

[0015] In autonomous or assisted driving systems, trajectory prediction of traffic participants is typically achieved using an end-to-end modeling approach based on deep learning. The system collects motion state data (position coordinates, velocity, acceleration, and heading angle, etc.) of the target vehicle within a historical time window, and obtains the status of surrounding traffic participants, as well as high-precision map information (such as lane centerlines, road boundaries, and intersection topology). After coordinate transformation and time alignment, the data is processed into a historical sequence, which serves as the input to the initial model. The model often employs recurrent neural networks, convolutional neural networks, or Transformer structures to extract temporal features from the input data, learning the target's motion characteristics and trends. Attention mechanisms or graph neural networks can be used to encode and learn the motion influence of neighboring traffic participants. For high-precision map information, it can be vectorized or rasterized and then fused with motion features to form a high-dimensional feature representation for prediction.

[0016] At the model output layer, two main approaches are currently employed: a single-modal regression approach, which directly outputs a sequence of two-dimensional position coordinates for multiple future time steps through a fully connected layer, using mean squared error or average displacement error as the training objective; and a multimodal prediction approach, which outputs multiple candidate trajectories to represent the uncertainty of the target's future trajectory. This approach utilizes a Gaussian Mixture Model (GMM) to output the distribution modes of multiple trajectories. The model can simultaneously output the weight parameters, mean, and covariance parameters of each mode (corresponding to each candidate trajectory). During the training phase, maximum likelihood estimation is used to measure the degree of matching between the predicted and true distributions, thereby optimizing the model.

[0017] During the inference phase, multimodal prediction schemes typically select the mode with the highest weight as the final prediction result, or output several candidate trajectories with high probability for use by downstream planning modules. Threshold strategies can also be set to filter out low-probability modes to reduce computational overhead.

[0018] When using the above solutions for trajectory prediction, in scenarios with hard barriers such as guardrails, central dividers, and road boundaries, the predicted trajectory may still traverse these barriers, resulting in a physically infeasible trajectory. Research indicates that the underlying causes of this problem include, but are not limited to: ① Neural network-based trajectory prediction models typically learn the future movement distribution of traffic participants based on the historical movement trajectory of the target object and the characteristics of the road environment. However, due to the lack of specific algorithm rules or constraints to prevent the model from generating trajectories that enter infeasible areas, the predicted trajectories may cross the hard isolation structure of the road surface to form physically infeasible trajectories. Although obvious illegal trajectories may eventually be removed through filtering, it still has a negative impact on the overall planning and wastes computing resources.

[0019] ② In engineering practice, to prevent illegal trajectories from affecting downstream planning, rule-based filtering of obstacle-crossing trajectories is typically used during the model inference phase. However, filtering rules only take effect during the inference phase and cannot impose relevant constraints on model parameters during the training phase. Furthermore, filtering uses hard threshold judgments (e.g., triggering an alarm when the distance is less than 1 meter). Such filtering rules lack differentiability, meaning the system cannot determine "how the system performance will change or be optimized if this threshold is adjusted." Consequently, parameter optimization cannot be performed during model training, meaning constraints cannot be imposed on model parameters during the training phase.

[0020] ③ In commonly used models, map information is mostly used as ordinary input data for model feature processing. During training, the model learns how to use this map data in a conventional way, which is a kind of implicit learning method. It cannot constrain the trajectory prediction process and results based on the physical meaning of the map data. This leads to the possibility that when the model encounters off-site scenes or complex road structures, it may still produce trajectories that violate common sense. For example, when the model encounters a scene that is not present in the training data or a complex road structure, it is very likely to predict trajectories that do not conform to physical laws, such as predicted trajectories in which vehicles pass through walls, drive over medians, or obstacles.

[0021] To address at least one of the above problems, embodiments of this disclosure start from the underlying logic of probability distribution and explicitly constrain impassable regions in model trajectory prediction.

[0022] refer to Figure 1 This disclosure provides a training method for a multimodal trajectory prediction model, including: S101, The multimodal trajectory prediction model is trained through imitation learning and reinforcement learning; wherein, the multimodal probability distribution and weights predicted by the model are obtained in the imitation learning stage; S102, In the reinforcement learning phase, the reward function of each modality is determined based on the differentiable Euclidean symbolic distance field (ESDF) from the predicted trajectory points on each modality to the boundary of the impassable region. Based on the reward function values ​​of each modality, the multimodal probability distribution in the imitation learning phase is reweighted to update the weights of the predicted trajectories of each modality.

[0023] The embodiments of this disclosure, based on the existing trajectory prediction network structure, set up a two-stage collaborative training of imitation learning and reinforcement learning. In the reinforcement learning stage, a continuously differentiable constraint reward mechanism is established for impassable regions using the Euclidean symbolic distance field (ESDF), achieving reweighting of multimodal predicted trajectories. This achieves the purpose of penalizing and suppressing the probability of predicted trajectories with hard barriers, improving the physical feasibility of trajectory prediction results from the probability distribution level. Reweighting refers to adjusting and updating the probability distribution of different trajectory prediction results to increase the probability of legal trajectories and decrease the probability of illegal trajectories. For example, the probability weight of legal predicted trajectories that comply with traffic rules and avoid obstacles is increased, while the probability weight of illegal predicted trajectories that collide with walls or cross guardrails is decreased.

[0024] Compared to common solutions that rely solely on data-driven imitation learning for prediction or rule filtering during the inference phase, this disclosure sets up a constraint reward mechanism during the training phase to suppress illegal modes through probability adjustment. This enables the model parameters to form a road structure that constrains trajectory prediction during the training phase, thereby achieving stable physical feasibility of the trajectory prediction results.

[0025] According to embodiments of this disclosure, optionally, after the imitation learning phase is completed, the model encoder parameters are frozen so that the model decoder participates in parameter updates during the reinforcement learning phase, while the model encoder does not participate in parameter updates. For example, in some embodiments, the scene encoder parameters can be frozen after imitation learning is completed, and only relevant parameters in the prediction decoder (such as motion head and modality classification head parameters) are updated during the reinforcement learning phase. The multimodal probability distribution strategy at the end of the imitation learning phase is used as a reference strategy as the basis for subsequent reweighting. The progressive enhancement of the constraint optimization of this disclosure is achieved through a two-stage training strategy of imitation learning and reinforcement learning.

[0026] According to embodiments of this disclosure, optionally, a joint loss function is employed in the imitation learning stage. The joint loss function includes: a binary Gaussian negative log-likelihood loss function, a Laplace distribution loss function, a focal loss function, and a penalty loss function. The resulting joint loss function is continuously differentiable. In the imitation learning stage, a continuously differentiable constraint loss can provide direct gradient guidance for the model. Combined with subsequent reinforcement learning, this can significantly improve the constraint effect.

[0027] According to embodiments of this disclosure, optionally, the differentiable ESDF distance from the predicted trajectory point to the boundary of the impassable area is calculated using a bilinear interpolation method; the value of the penalty function for each mode is determined based on the differentiable ESDF distance, and the value of the penalty function is used to determine the reward function value for each mode.

[0028] For example, based on the ESDF distance field, impassable areas in a map can be encoded as two-dimensional grids. The value corresponding to each ESDF grid cell represents the signed distance from that location point to the boundary of the nearest impassable obstacle. This distance value can be queried through bilinear interpolation, which is computationally efficient. Furthermore, the differentiability provided by bilinear interpolation makes it easy to construct the constraint signal as a differentiable penalty function, realize the reweighting of the trajectory probability distribution, and participate in the end-to-end training of the model.

[0029] According to embodiments of this disclosure, optionally, a continuous constraint reward function can be constructed using a penalty function, and the continuous constraint reward function and a specified driving reward can be aggregated to form a multi-reward aggregation function; and the group relative advantage estimation (GRPO) method can be used to calculate the within-group standardized advantage value of the multi-reward aggregation function for each modality, wherein the advantage value is restricted within a predetermined range, and gradient updates are performed using KL divergence regularization to obtain the reward-based loss function.

[0030] In other words, during the reinforcement learning phase, the multi-reward aggregation based on GRPO, combined with the KL divergence regularization strategy, can further refine the modality probability allocation during reinforcement learning training. Compared with the previous simple imitation learning, the two-stage training improves the constraint effect and output stability of the model.

[0031] refer to Figure 2 This disclosure also provides a multimodal trajectory prediction device 100, which can be trained and generated using at least one of the above embodiments of this disclosure, and includes: The multimodal Gaussian mixture prediction module 110 is used to output the initial multimodal trajectory probability distribution and weights based on the input data; The ESDF distance query module 120 is used to construct an ESDF grid based on map information and use bilinear interpolation to determine the differentiable ESDF distance values ​​from trajectory points on each modal trajectory to the boundary of the impassable area. The penalty value calculation module 130 is used to calculate the penalty value and reward value of each mode based on the ESDF distance value; The probability reweighting module 140 is used to calculate new multimodal weights based on the reward value and the reweighting formula. This is to reweight the initial multimodal trajectory probability distribution; The result output module 150 is used to output one or more predicted trajectories based on the reweighted multimodal predicted trajectory.

[0032] Based on at least one of the above embodiments, the multimodal trajectory prediction device 100 of this disclosure optimizes the probability distribution of the multimodal predicted trajectory output by the Gaussian mixture model. For the hard isolation structure of the impassable area, the predicted trajectory of each mode is reweighted by a constraint reward mechanism. Without changing the main structure of the trajectory prediction network, continuous differentiable constraints are established for the impassable area at the probability distribution level, so as to suppress the predicted trajectory from crossing the hard isolation structure and improve the physical feasibility of the prediction results.

[0033] The implementation methods and advantages of the embodiments of this application have been described above through multiple examples. The specific processing procedures and result analysis of the embodiments of this application are described in detail below based on at least one implementation method and in conjunction with illustrative examples.

[0034] (a) Constructing a trajectory prediction model refer to Figure 3 First, let the sequence of motion states of the target traffic participant within the historical time window be:

[0035] in, Includes target traffic participants at time Position, speed, and heading angle, etc.

[0036] The feature encoding module can fuse relevant historical motion information, surrounding traffic participant information, and map element information to obtain a high-dimensional feature representation for predicting future trajectories.

[0037] For example, a multi-layer attention structure based on Transformer can be used as the encoder to perform the following feature interaction processing: (a) Modal self-attention: Self-attention is calculated on the query vector of each prediction modality to model the correlation between modalities; (b) Cross-agent attention: Combining relative position coding (RPE), the interaction between the predicted target and other traffic participants is encoded; (c) Agent-Map Cross-Attention: Using map features as keys, the spatial relationship between the movement trend of the predicted target and the road structure is encoded.

[0038] And, suppose the trajectory that needs to be predicted in the future is:

[0039] By modeling the future trajectory using a multimodal Gaussian Mixture Model (MM-GMM), we can obtain... The modality at time... The probability distribution is as follows:

[0040] Among them, the mean vector The predicted position coordinates represent the predicted center location in this mode; And, covariance matrix Let be the term used to describe the degree of dispersion of the predicted trajectory under this mode:

[0041] in, , They are respectively and Standard deviation of direction This is the correlation coefficient. The model can directly output the log-standard deviation. , and correlation coefficient In addition, effective parameter values ​​can be obtained through exponential transformation and interval truncation.

[0042] Output via classification header dimensional vector, and through Normalization yields the weights for each mode. It satisfies:

[0043] No. k Weights of each modality That is the first k The probability of the predicted trajectory (straight or left turn) occurring in each modality. .

[0044] The multimodal probability distribution of the trajectory predicted by the K-mode Gaussian mixture model is expressed as:

[0045] The above model needs to be trained with a large amount of data. The training objective is to maximize the negative log-likelihood (NLL) of the true trajectory, that is, to make the true trajectory fall in the high probability region.

[0046] This disclosure proposes adding reweighted optimization to the aforementioned multimodal Gaussian mixture model, and constructing a joint loss function for the optimized model during the imitation learning phase. Furthermore, after the imitation learning is completed, a reinforcement learning phase is added. During the reinforcement learning phase, a constructed reward constraint mechanism is used to reweight the probabilities of each modality, achieving the simultaneous optimization of trajectory fitting accuracy and constraint satisfaction as described in this disclosure. The constraint mechanism of this disclosure will be explained in detail below.

[0047] (ii) Introducing a reward and constraint mechanism 2.1 Differentiable distance calculation based on ESDF We introduce the Euclidean Signed Distance Field (ESDF) as an efficient differentiable representation of the distance of hard-isolated structures for the multimodal Gaussian mixture model.

[0048] Based on the ESDF distance field, impassable areas in a map can be encoded into a two-dimensional raster. The value corresponding to each ESDF raster cell represents that location (denoted as a point). The signed distance to the boundary of the nearest impassable obstacle can be expressed as: :

[0049] in, This represents a set of impassable areas. For its boundary, For point The minimum Euclidean distance to the boundary.

[0050] For any predicted trajectory point The distance (minimum Euclidean distance) to the hard isolation structure can be obtained by performing a bilinear interpolation query on the ESDF raster:

[0051] in, , The radius of the ESDF grid coverage area. , This refers to the raster resolution.

[0052] Bilinear interpolation refers to linear interpolation of discrete data points in two-dimensional space, estimating a new value by weighted averaging of the nearest neighbor pixels around the target point. Bilinear interpolation is continuously differentiable with respect to the input coordinates, and its gradient can be derived from the ESDF distance value. The position parameters are backpropagated to the predicted trajectory, allowing the entire constraint mechanism to be incorporated into end-to-end optimization. The use of bilinear interpolation in this embodiment also eliminates quantization errors introduced by discretization.

[0053] 2.2 Continuously Differentiable Penalty Function Based on the distance value obtained from ESDF, this embodiment of the disclosure constructs a piecewise continuous penalty function. :

[0054] in, This is the distance attenuation coefficient. Used to control access within a passable area, with penalties varying with distance. The decay rate; when the predicted trajectory point is far from the obstacle, the penalty value approaches zero; when it approaches the obstacle boundary, the penalty value approaches 1; in, This is the internal penalty growth coefficient. This is used to control the rate at which the penalty increases after a trajectory point enters an impassable area. It is understandable that this piecewise function... It is continuous at all points (both left and right limits are 1) and differentiable within each segment; in, The exponential decay of the segment ensures that the penalty decreases rapidly when moving away from the obstacle; in, The linear growth of the segment ensures that the gradient does not vanish after entering the interior of the obstacle, providing a continuous and stable optimization signal for training.

[0055] Based on this, for the first The mean sequence of predicted trajectories for each modality Define the penalty value for each modality. as follows:

[0056] And based on the penalty values ​​for each modality, a modal-constrained reward function is constructed. :

[0057] in, As a penalty weighting coefficient, The penalty value is applied when the trajectory crosses or approaches a hard isolation area. Increase, reward value reduce.

[0058] 2.3 Reward-based modal probability reweighting This embodiment of the disclosure performs exponential reweighting on the original modal weights obtained from the multimodal Gaussian mixture model. Specifically, it reweights the weights of each mode. Adjusted to :

[0059] in, For temperature parameters, This is used to control the strength of the effect of rewards on probability adjustment; understandably, a smaller effect is better. This can make the reweighting effect more significant, and larger This can make the adjustment smoother.

[0060] Here, the reweighting operation is equivalent to adding a constraint bias in the logarithmic probability space:

[0061] in, The constraint bias term, equivalent to tilting the distribution towards higher reward levels while maintaining the original modal distribution, is used. logZ is the normalization term, ensuring the probability sum is 1. Since... It is based on a differentiable penalty function. The reweighting process is calculated using ESDF bilinear interpolation, therefore the reweighting process is differentiable with respect to the model parameters.

[0062] Therefore, the multimodal probability distribution of the predicted trajectory in this embodiment of the present disclosure can be updated as follows:

[0063] The above describes the multimodal probability distribution of the predicted trajectory obtained through reweighted updates based on a reward constraint mechanism. The following details the joint loss function for the imitation learning stage and the optimization objective for the reinforcement learning stage, used to train the multimodal prediction model in stages.

[0064] (III) Imitation learning stage: Constructing joint loss training: During the imitation learning phase, this disclosure constructs a joint loss function to simultaneously optimize trajectory fitting accuracy and constraint satisfaction. Specifically, the joint loss function in this embodiment includes: a binary Gaussian negative log-likelihood loss function, a Laplace distribution loss function, a focal loss function, and a penalty loss function, which are described below.

[0065] (a) Binary Gaussian negative log-likelihood loss function: During training, for the most recent mode (This can be determined by matching the distance to the actual trajectory), and the regression loss is:

[0066] in, This is the prediction error vector.

[0067] (b) Laplace loss of heading angle: To model the uncertainty of the heading angle based on the Laplace distribution, the Laplace distribution loss function is used:

[0068] (c) Modality classification loss: Focal Loss is used for training on modality classification to alleviate the class imbalance problem.

[0069] (d) Constraint on penalty loss:

[0070] During model parameter updates, a constraint penalty loss term penalizes predicted modes that approach or enter hard isolation regions. This is because... and Both are differentiable with respect to the model parameters, and the gradient can act on both modal probabilities (to encourage the reduction of weights for high-penalty modes) and trajectory positions (to encourage trajectories to move away from obstacles).

[0071] Based on the above, construct the joint loss function. for:

[0072] in, To constrain the loss weights.

[0073] (iv) Reinforcement learning phase: Constructing reward constraints and optimization objectives: After the imitation learning is completed, this disclosure introduces a reinforcement learning phase to further optimize the modality probabilities.

[0074] (a) Training strategy: Save the distribution strategy at the end of the imitation learning phase as a reference strategy. In other words, the scene encoder parameters of the model are frozen, and only the motion head and modality classification head parameters in the predictive decoder are updated.

[0075] (b) Continuous constraint reward: Based on differentiable penalty function Constructing a continuous constraint reward signal:

[0076] The continuous reward disclosed herein can provide a reward signal when the trajectory is far away from the obstacle (traditional binary collision rewards only focus on collision or non-collision), which can guide the distribution strategy to make fine adjustments near the safety boundary.

[0077] (c) Multi-reward aggregation: The constraint reward is weighted and aggregated with other reward components (such as collision safety, driving comfort, driving efficiency, etc.):

[0078] in, For the first j The weight of each reward component.

[0079] (d) Group Relative Policy Optimization (GRPO): exist k Advantages of intra-group standardization across modalities:

[0080] in, and These are the mean and standard deviation of the group rewards, respectively. Here, is the numerical stability constant, where the dominant value can be set to truncate to . The interval, which is the calculated advantage value Limited to the range [-2, 2], if If >2, then set it to 2; if If the value is less than -2, then set it to -2; the other values ​​remain unchanged to prevent individual abnormal data from affecting the overall model training.

[0081] (e) Gradient updates are performed using the KL (Kullback-Leibler divergence) regularization strategy to obtain the overall optimization objective and loss function for the reinforcement learning stage. as follows:

[0082] in, For KL divergence penalty term, As weights, the KL divergence penalty term can be added based on the currently generated distribution strategy. Distribution strategy of production in the imitation learning stage The differences between the two policies limit the scope of policy updates. When the new policy differs too much from the basic policy, it will be penalized. This allows the model to learn new capabilities while preventing the policy from deviating excessively from the policy in the imitation learning stage, thereby improving the overall stability of model training and obtaining reliable output.

[0083] After the model is trained, a multimodal Gaussian mixture model for predicting trajectories is obtained. The model trained in two stages can form a constraint awareness on the road structure at the parameter level, effectively suppressing illegal trajectories. The model can infer and output one or more physically legal and feasible predicted trajectories based on the user's historical motion state sequence (such as position coordinates, speed, acceleration, heading angle, etc.), the state information of surrounding traffic participants, and high-precision map information (such as lane lines, road boundaries, intersection topology, etc.).

[0084] In addition, for out-of-distribution scenarios (such as map scenarios that were never involved in training or scenarios where the real map has changed), the reweighting of the model during the inference phase can further play a role in safety redundancy. Therefore, the model prediction can still effectively suppress illegal trajectories in out-of-distribution scenarios.

[0085] As an example, during the model inference phase, the model first outputs the parameters of the multimodal Gaussian mixture distribution (the mean, covariance, and probability weights of the trajectories for each mode), and then performs the following processing: 1. ESDF Distance Query: Construct an ESDF raster based on map information, and query the distance value for the trajectory mean sequence of each modality using bilinear interpolation. d ; 2. Penalty value calculation: Calculate the overall penalty value for each modality based on the distance value. ; 3. Probability Reweighting: Calculate the new modal weights based on the reweighting formula. ; 4. Output Results: Select the mode with the highest weight as the prediction output, or output several high-weight candidate trajectories for the planning module to use.

[0086] As an example, Figure 6 The architecture of a multimodal trajectory prediction model according to a specific embodiment of this disclosure is shown, mainly including: Input layer (Scene Input): This layer contains historical trajectory sequences, the status of surrounding traffic participants, high-precision map features, and ESDF rasters. Feature Encoder: A multi-layer attention network based on Transformer, including modal self-attention module, inter-agent cross-attention module, and agent-map cross-attention module; Multimodal prediction decoder: used to obtain K modes, each mode (Raw GMM) contains the trajectory mean, time-step covariance parameter and mode probability; The Constraint Reweighting module calculates the penalty function based on ESDF distance and reweights the probabilities of the K modalities. Output layer: Outputs the reweighted multimodal Gaussian mixture distribution.

[0087] Figure 7 and Figure 8 A schematic diagram comparing the ESDF range field and the penalty function is shown. Among them, Figure 7 This displays an ESDF raster visualization of a road scene, where impassable areas (Barrier) are represented by negative values ​​and warm colors, while passable areas (Road) are represented by positive values ​​and cool colors; the shade of color indicates the distance. Figure 8 Show the penalty function The curve, in The region exhibits exponential decay, in The region grows linearly, in Continuous.

[0088] Figure 9 and Figure 10 A schematic diagram showing the comparison before and after modal probability reweighting is presented. Figure 9 The multimodal prediction results before reweighting are shown. The mode that crosses the barrier (red) has a higher probability weight and is still considered as one of the planned paths. Figure 10 The results after reweighting show that the modal probability of crossing the guardrail is suppressed, while the probability of the legal modal probability is increased accordingly.

[0089] Based on at least one of the above embodiments of this disclosure, an optimized design for output trajectory distribution based on a Gaussian mixture model is provided. A constraint reward mechanism is established in conjunction with a hard isolation structure to probabilistically reweight the predicted modes. Without changing the main structure of the trajectory prediction network, continuous differentiable constraints are established for impassable regions at the probability distribution level. Simultaneous training using both imitation learning and reinforcement learning stages is employed to suppress predicted trajectories from traversing the hard isolation structure, thereby improving the physical feasibility of the prediction results.

[0090] Accordingly, this disclosure also provides a training device 200 for a multimodal trajectory prediction model, with reference to Figure 11 ,include: The imitation learning training module 210 and the reinforcement learning training module 220 are used to train the multimodal trajectory prediction model. The imitation learning training module 210 is used to obtain the multimodal probability distribution and weights predicted by the model, and the reinforcement learning module 220 is used to determine the value of the reward function of each mode based on the differentiable Euclidean signed distance field ESDF distance from the predicted trajectory points on each mode to the boundary of the impassable area. Based on the value of the reward function of each mode, the multimodal probability distribution in the imitation learning stage is reweighted to update the weights of the predicted trajectories of each mode.

[0091] The technical solutions disclosed herein involve the collection, storage, use, processing, transmission, provision, and disclosure of various information, such as user personal information, in accordance with relevant laws and regulations and without violating public order and good morals.

[0092] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, a computer program product, and an autonomous vehicle.

[0093] Figure 12 A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, autonomous vehicles, and other similar computing devices. The electronic device may also be mounted on or connected to an autonomous vehicle. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0094] like Figure 12As shown, device 800 includes a computing unit 801, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 802 or a computer program loaded from storage unit 808 into random access memory (RAM) 803. RAM 803 may also store various programs and data required for the operation of device 800. The computing unit 801, ROM 802, and RAM 803 are interconnected via bus 804. Input / output (I / O) interface 805 is also connected to bus 804.

[0095] Multiple components in device 800 are connected to I / O interface 805, including: input unit 806, such as keyboard, mouse, etc.; output unit 807, such as various types of monitors, speakers, etc.; storage unit 808, such as disk, optical disk, etc.; and communication unit 809, such as network card, modem, wireless transceiver, etc. Communication unit 809 allows device 800 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0096] The computing unit 801 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as the training method for a multimodal trajectory prediction model. For example, in some embodiments, the training method for a multimodal trajectory prediction model can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed on device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the training method for the multimodal trajectory prediction model described above can be performed. Alternatively, in other embodiments, computing unit 801 may be configured by any other suitable means (e.g., by means of firmware) to perform a training method for a multimodal trajectory prediction model.

[0097] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0098] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0099] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0100] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0101] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0102] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0103] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0104] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A training method for a multimodal trajectory prediction model, comprising: The multimodal trajectory prediction model is trained using imitation learning and reinforcement learning. In the imitation learning stage, the multimodal probability distribution and weights predicted by the model are obtained. In the reinforcement learning stage, the reward function of each mode is determined based on the differentiable Euclidean signed distance field (ESDF) from the predicted trajectory points on each mode to the boundary of the impassable area. The multimodal probability distribution of the imitation learning stage is reweighted based on the reward function values ​​of each mode to update the weights of the predicted trajectories of each mode.

2. The method according to claim 1, further comprising: After the imitation learning phase is completed, the model encoder parameters are frozen so that the model decoder participates in parameter updates during the reinforcement learning phase, while the model encoder does not participate in parameter updates.

3. The method according to claim 1, wherein, The multimodal trajectory prediction model includes a multimodal Gaussian mixture model, and a joint loss function is used in the imitation learning stage. The joint loss function includes: a binary Gaussian negative log-likelihood loss function, a Laplace distribution loss function, a focal loss function, and a penalty loss function.

4. The method according to claim 1, wherein, Before determining the value of the reward function for each modality, the method further includes: The differentiable ESDF distance from the predicted trajectory point to the boundary of the impassable area is calculated using the bilinear interpolation method. The value of the penalty function for each mode is determined based on the differentiable ESDF distance, and the value of the penalty function is used to determine the value of the reward function for each mode.

5. The method according to claim 4, wherein, The reweighting process includes: The weights of the multimodal probability distribution are adjusted based on the reward function value, and the training optimization objective of the reinforcement learning stage is taken as the reward-based loss function. After training is completed, the weights of the multimodal probability distribution are updated.

6. The method according to claim 5, wherein, Constructing the reward-based loss function includes: A continuous constraint reward function is constructed using a penalty function, and the continuous constraint reward function and the specified driving reward are aggregated to form a multi-reward aggregation function; The group relative advantage estimation (GRPO) method is used to calculate the within-group standardized advantage value of the multi-reward aggregation function for each modality, wherein the advantage value is restricted to a predetermined range, and gradient updates are performed through KL divergence regularization to obtain the reward-based loss function.

7. The method according to claim 4, wherein, The penalty function for each mode in, For ESDF distance values Constructed piecewise continuous penalty function: Among them, the piecewise continuous penalty function exist Continuous at, and in Duan Ji All segments are infinitesimal; This is the distance attenuation coefficient. Used to control penalties based on distance within a passable area. The rate of decay; This is the internal penalty growth coefficient. This is used to control the rate at which the penalty increases after a trajectory point enters an impassable area. When the predicted trajectory point is far from the obstacle, the penalty value approaches zero; when the predicted trajectory point is close to the obstacle boundary, the penalty value approaches 1.

8. The method according to claim 7, wherein, The reward function for each modality Represented as: in, As a penalty weighting coefficient, Penalty is applied when the predicted trajectory point crosses or approaches an impassable area. The value increases, the reward The value decreases.

9. The method according to claim 8, wherein, Adjusting the weights of the multimodal probability distribution based on the reward function value includes: The weights of each mode in the multimodal probability distribution Adjusted to , represented as: in, For temperature parameters, This is used to control the strength of the impact of rewards on probability adjustments.

10. The method of claim 7, further comprising: A continuous constraint reward function is constructed using the penalty function, expressed as: in, As a penalty weighting coefficient, The continuous constraint reward function is used to construct the loss function for the reinforcement learning stage.

11. A multimodal trajectory prediction device, trained and generated based on the method of any one of claims 1-10, comprising: The multimodal Gaussian mixture prediction module is used to output the initial multimodal trajectory probability distribution and weights based on the input data; The ESDF distance query module is used to construct an ESDF grid based on map information and use bilinear interpolation to determine the differentiable ESDF distance values ​​from trajectory points on each modal trajectory to the boundary of the impassable area. The penalty value calculation module is used to calculate the penalty value and reward value for each mode based on the ESDF distance value; The probability reweighting module is used to calculate new multimodal weights based on the reward value and the reweighting formula. This is to reweight the initial multimodal trajectory probability distribution; The results output module is used to output one or more predicted trajectories based on the reweighted multimodal predicted trajectory.

12. A training device for a multimodal trajectory prediction model, comprising: The imitation learning training module and the reinforcement learning training module are used to train the multimodal trajectory prediction model. The imitation learning training module is used to obtain the multimodal probability distribution and weights predicted by the model, while the reinforcement learning module is used to determine the reward function value of each mode based on the differentiable Euclidean signed distance field (ESDF) from the predicted trajectory points on each mode to the boundary of the impassable area. Based on the reward function values ​​of each mode, the multimodal probability distribution in the imitation learning stage is reweighted to update the weights of the predicted trajectories of each mode.

13. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-10 or to implement the multimodal trajectory prediction apparatus of claim 11.

14. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to execute the method according to any one of claims 1-10 or to implement the multimodal trajectory prediction device according to claim 11.

15. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-10 or the multimodal trajectory prediction apparatus according to claim 11.

16. An autonomous vehicle, including the electronic equipment as claimed in claim 13.

Images