A method and device for predicting motion trajectory

Abstract

An embodiment of the present invention discloses a motion trajectory prediction method and device, which includes: obtaining historical trajectory and motion attribute information of each traffic participant corresponding to a target object and corresponding current map information; using a feature extraction layer of a target trajectory prediction model and initial features corresponding to each traffic participant to determine trajectory prediction features corresponding to each traffic participant; using the feature extraction layer of the target trajectory prediction model and trajectory prediction features corresponding to each traffic participant to determine a multimodal probability distribution of latent random variables corresponding to each traffic participant; using a feature regression layer of the target trajectory prediction model, trajectory prediction features corresponding to each traffic participant, and multimodal probability distribution of latent random variables corresponding to each traffic participant to determine a multimodal predicted trajectory corresponding to each traffic participant, so as to reduce the limitations of motion trajectory prediction and better adapt to relatively complex autonomous driving scenarios.

Description

Technical Field

[0001] The present invention relates to the field of trajectory prediction technology, and in particular to a motion trajectory prediction method and device. Background Art

[0002] In the field of autonomous driving, autonomous vehicles need to consider the future motion trajectories of surrounding traffic participants to plan their own driving trajectory, ensuring the safety of both the vehicle and the traffic participants. Consequently, it is crucial for autonomous vehicles to accurately and timely predict the future motion trajectories of traffic participants.

[0003] Considering the significant uncertainty in the future behavior of traffic participants, namely, the significant uncertainty in the future motion trajectories of traffic participants, related technologies generally use a mixed Gaussian distribution to model the probability distribution of the future motion trajectories of traffic participants. Modal categories and classification rules for future motion trajectories are manually designed, and a neural network model is trained using classification plus regression. The neural network model, along with the historical motion trajectories of each traffic participant and other corresponding dynamic and static information, is used to predict the future trajectory and probability of each modal category of each traffic participant. The other dynamic information corresponding to a traffic participant may include the historical trajectories of other traffic participants and autonomous vehicles other than the traffic participant, and the static information may include the current map information corresponding to the autonomous vehicle.

[0004] In the above process, it is necessary to manually design the number of categories and classification rules of the modes of the future motion trajectory predicted by the neural network model. This, to a certain extent, makes the predicted future motion trajectory limited and difficult to adapt to more complex autonomous driving scenarios. Summary of the Invention

[0005] The present invention provides a motion trajectory prediction method and device to reduce the limitations of motion trajectory prediction and better adapt to more complex autonomous driving scenarios. The specific technical solution is as follows:

[0006] In a first aspect, an embodiment of the present invention provides a method for predicting a motion trajectory, the method comprising:

[0007] Obtain the historical trajectory and motion attribute information of each traffic participant corresponding to the target object and the corresponding current map information;

[0008] Determining trajectory prediction features corresponding to each traffic participant using a feature extraction layer of a target trajectory prediction model and initial features corresponding to each traffic participant, wherein the initial features corresponding to the traffic participant include: historical trajectory and motion attribute information of the traffic participant, historical trajectory and motion attribute information of other corresponding traffic participants and the target object, and the current map information;

[0009] Using the feature extraction layer of the target trajectory prediction model and the trajectory prediction features corresponding to each traffic participant, the multimodal probability distribution of the latent random variable corresponding to each traffic participant is determined, where the latent random variable represents the randomness of the behavior of each traffic participant.

[0010] The multimodal predicted trajectory corresponding to each traffic participant is determined by utilizing the feature regression layer of the target trajectory prediction model, the trajectory prediction features corresponding to each traffic participant, and the multimodal probability distribution of the latent random variables corresponding to each traffic participant.

[0011] Optionally, the initial features corresponding to the traffic participant objects are features arranged in chronological order, including features of multiple historical moments corresponding to the traffic participant objects;

[0012] The step of determining the trajectory prediction features corresponding to each traffic participant using the feature extraction layer of the target trajectory prediction model and the initial features corresponding to each traffic participant includes:

[0013] For each traffic participant, use the feature extraction layer of the target trajectory prediction model to repeatedly execute the following steps AC on the initial features corresponding to the traffic participant to determine the intermediate prediction features corresponding to the traffic participant;

[0014] For each traffic participant, based on a graph neural network, intermediate prediction features corresponding to each static object in the intermediate prediction features corresponding to the traffic participant are fused to determine a trajectory prediction feature corresponding to the traffic participant, wherein the static objects include each static object in the current map information;

[0015] Step A: Perform nonlinear mapping on the features to be processed corresponding to the traffic participant object from the feature dimension to obtain the mapped features corresponding to the traffic participant object, wherein the features to be processed are the initial features corresponding to the traffic participant object or the intermediate predicted features corresponding to the traffic participant object generated in the previous iteration;

[0016] Step B: performing a feature aggregation operation on the mapping features from a time dimension to obtain the aggregated features corresponding to the traffic participant object;

[0017] Step C: Fusing the aggregated features with the features of each historical moment in the features to be processed.

[0018] Optionally, the step of determining the multimodal probability distribution of latent random variables corresponding to each traffic participant using the feature extraction layer of the target trajectory prediction model and the trajectory prediction features corresponding to each traffic participant includes:

[0019] For each traffic participant, the unimodal probability distribution of the latent random variable corresponding to the traffic participant is determined using the feature extraction layer of the target trajectory prediction model and the trajectory prediction features corresponding to the traffic participant;

[0020] For each traffic participant, the normalized flow mapping algorithm and the single-modal probability distribution of the latent random variables corresponding to the traffic participant are used to obtain the multi-modal probability distribution of the latent random variables corresponding to the traffic participant.

[0021] Optionally, before the step of determining the trajectory prediction features corresponding to each traffic participant using the feature extraction layer of the target trajectory prediction model and the initial features corresponding to each traffic participant, the method further includes:

[0022] The process of training a target trajectory prediction model includes:

[0023] Obtain an initial trajectory prediction model;

[0024] Obtaining sample training information corresponding to each sample traffic object and a sample future trajectory corresponding to each sample traffic object, wherein the sample training information corresponding to the sample traffic object includes: a sample historical trajectory and sample motion attribute information of the sample traffic object, and a sample historical trajectory, sample motion attribute information, and sample static object information of the corresponding sample dynamic object;

[0025] For each sample traffic object, using the feature extraction layer of the initial trajectory prediction model and the initial sample features corresponding to the sample traffic object, determine the sample prediction features corresponding to the sample traffic object, wherein the initial sample features corresponding to the sample traffic object include: the sample historical trajectory and sample motion attribute information of the sample traffic object, and the sample historical trajectory, sample motion attribute information, and sample static object information of the corresponding sample dynamic object;

[0026] For each sample traffic object, determine the multimodal probability distribution of the latent random variable corresponding to the sample traffic object using the feature extraction layer of the initial trajectory prediction model and the sample prediction features corresponding to the sample traffic object;

[0027] For each sample traffic object, determine the multimodal predicted trajectory corresponding to the sample traffic object using the feature regression layer of the initial trajectory prediction model, the sample prediction features corresponding to the sample traffic object, and the multimodal probability distribution of the latent random variables corresponding to the sample traffic object;

[0028] For each sample traffic object, the preset variational algorithm is used to process the sample future trajectory corresponding to the sample traffic object to obtain the variational probability distribution of the latent random variable corresponding to the sample traffic object;

[0029] For each sample traffic object, the KL divergence value of the latent random variable corresponding to the sample traffic object is determined using the multimodal probability distribution of the latent random variable corresponding to the sample traffic object and the variational probability distribution of the latent random variable corresponding to the sample traffic object;

[0030] For each sample traffic object, the trajectory reconstruction loss value corresponding to the sample traffic object is determined using the multimodal predicted trajectory corresponding to the sample traffic object, the latent random variable variational probability distribution, and the sample future trajectory corresponding to the sample traffic object;

[0031] For each sample traffic object, the KL divergence value of the latent random variable corresponding to the sample traffic object and the trajectory reconstruction loss value corresponding to the sample traffic object are used to construct the variational lower bound of the maximum likelihood function; and whether the variational lower bound of the constructed maximum likelihood function is maximized is determined;

[0032] If the variational lower bound of the constructed maximum likelihood function is not maximized, adjusting model parameters of the feature extraction layer and the feature regression layer of the initial trajectory prediction model, and returning to the step of determining, for each sample traffic object, a sample prediction feature corresponding to the sample traffic object using the feature extraction layer of the initial trajectory prediction model and the initial sample features corresponding to the sample traffic object;

[0033] If the variational lower bound of the constructed maximum likelihood function is maximized, it is determined that the initial trajectory prediction model has converged, and the target trajectory prediction model including a feature extraction layer and a feature regression layer is obtained.

[0034] Optionally, the step of determining the multimodal predicted trajectory corresponding to each traffic participant using the feature regression layer of the target trajectory prediction model, the trajectory prediction features corresponding to each traffic participant, and the multimodal probability distribution of latent random variables corresponding to each traffic participant includes:

[0035] For each traffic participant, sampling is performed on the multimodal probability distribution of the latent random variables corresponding to the traffic participant to obtain multiple latent random variable samples corresponding to the traffic participant;

[0036] The multimodal prediction trajectory corresponding to each traffic participant is determined by utilizing the feature regression layer of the target trajectory prediction model, the trajectory prediction features corresponding to each traffic participant, and a plurality of latent random variable samples corresponding to each traffic participant.

[0037] In a second aspect, an embodiment of the present invention provides a motion trajectory prediction device, the device comprising:

[0038] An acquisition module configured to obtain historical trajectory and motion attribute information of each traffic participant corresponding to the target object and corresponding current map information;

[0039] The first determination module is configured to determine trajectory prediction features corresponding to each traffic participant using a feature extraction layer of the target trajectory prediction model and initial features corresponding to each traffic participant, wherein the initial features corresponding to the traffic participant include: historical trajectory and motion attribute information of the traffic participant, historical trajectory and motion attribute information of other corresponding traffic participants and the target object, and the current map information;

[0040] a second determination module configured to determine a multimodal probability distribution of a latent random variable corresponding to each traffic participant using a feature extraction layer of the target trajectory prediction model and trajectory prediction features corresponding to each traffic participant, wherein the latent random variable represents the randomness of the behavior of each traffic participant;

[0041] The third determination module is configured to determine the multimodal prediction trajectory corresponding to each traffic participant object by using the feature regression layer of the target trajectory prediction model, the trajectory prediction features corresponding to each traffic participant object, and the multimodal probability distribution of the latent random variables corresponding to each traffic participant object.

[0042] Optionally, the initial features corresponding to the traffic participant objects are features arranged in chronological order, including features of multiple historical moments corresponding to the traffic participant objects;

[0043] The first determination module is specifically configured to, for each traffic participant, use the feature extraction layer of the target trajectory prediction model to repeatedly perform the following steps AC based on the initial features corresponding to the traffic participant to determine the intermediate prediction features corresponding to the traffic participant;

[0044] For each traffic participant, based on a graph neural network, intermediate prediction features corresponding to each static object in the intermediate prediction features corresponding to the traffic participant are fused to determine a trajectory prediction feature corresponding to the traffic participant, wherein the static objects include each static object in the current map information;

[0045] Step A: Perform nonlinear mapping on the features to be processed corresponding to the traffic participant object from the feature dimension to obtain the mapped features corresponding to the traffic participant object, wherein the features to be processed are the initial features corresponding to the traffic participant object or the intermediate predicted features corresponding to the traffic participant object generated in the previous iteration;

[0046] Step B: performing a feature aggregation operation on the mapping features from a time dimension to obtain the aggregated features corresponding to the traffic participant object;

[0047] Step C: Fusing the aggregated features with the features of each historical moment in the features to be processed.

[0048] Optionally, the second determination module is specifically configured to determine, for each traffic participant, a unimodal probability distribution of a latent random variable corresponding to the traffic participant using a feature extraction layer of the target trajectory prediction model and trajectory prediction features corresponding to the traffic participant;

[0049] For each traffic participant, the normalized flow mapping algorithm and the single-modal probability distribution of the latent random variables corresponding to the traffic participant are used to obtain the multi-modal probability distribution of the latent random variables corresponding to the traffic participant.

[0050] Optionally, the device further includes:

[0051] The training module is configured to train the target trajectory prediction model before determining the trajectory prediction features corresponding to each traffic participant using the feature extraction layer of the target trajectory prediction model and the initial features corresponding to each traffic participant. The training module is specifically configured as follows:

[0052] Obtain an initial trajectory prediction model;

[0053] Obtaining sample training information corresponding to each sample traffic object and a sample future trajectory corresponding to each sample traffic object, wherein the sample training information corresponding to the sample traffic object includes: a sample historical trajectory and sample motion attribute information of the sample traffic object, and a sample historical trajectory, sample motion attribute information, and sample static object information of the corresponding sample dynamic object;

[0054] For each sample traffic object, using the feature extraction layer of the initial trajectory prediction model and the initial sample features corresponding to the sample traffic object, determine the sample prediction features corresponding to the sample traffic object, wherein the initial sample features corresponding to the sample traffic object include: the sample historical trajectory and sample motion attribute information of the sample traffic object, and the sample historical trajectory, sample motion attribute information, and sample static object information of the corresponding sample dynamic object;

[0055] For each sample traffic object, determine the multimodal probability distribution of the latent random variable corresponding to the sample traffic object using the feature extraction layer of the initial trajectory prediction model and the sample prediction features corresponding to the sample traffic object;

[0056] For each sample traffic object, determine the multimodal predicted trajectory corresponding to the sample traffic object using the feature regression layer of the initial trajectory prediction model, the sample prediction features corresponding to the sample traffic object, and the multimodal probability distribution of the latent random variables corresponding to the sample traffic object;

[0057] For each sample traffic object, the preset variational algorithm is used to process the sample future trajectory corresponding to the sample traffic object to obtain the variational probability distribution of the latent random variable corresponding to the sample traffic object;

[0058] For each sample traffic object, the KL divergence value of the latent random variable corresponding to the sample traffic object is determined using the multimodal probability distribution of the latent random variable corresponding to the sample traffic object and the variational probability distribution of the latent random variable corresponding to the sample traffic object;

[0059] For each sample traffic object, the trajectory reconstruction loss value corresponding to the sample traffic object is determined using the multimodal predicted trajectory corresponding to the sample traffic object, the latent random variable variational probability distribution, and the sample future trajectory corresponding to the sample traffic object;

[0060] For each sample traffic object, the KL divergence value of the latent random variable corresponding to the sample traffic object and the trajectory reconstruction loss value corresponding to the sample traffic object are used to construct the variational lower bound of the maximum likelihood function; and whether the variational lower bound of the constructed maximum likelihood function is maximized is determined;

[0061] If the variational lower bound of the constructed maximum likelihood function is not maximized, adjusting model parameters of the feature extraction layer and the feature regression layer of the initial trajectory prediction model, and returning to the step of determining, for each sample traffic object, a sample prediction feature corresponding to the sample traffic object using the feature extraction layer of the initial trajectory prediction model and the initial sample features corresponding to the sample traffic object;

[0062] If the variational lower bound of the constructed maximum likelihood function is maximized, it is determined that the initial trajectory prediction model has converged, and the target trajectory prediction model including a feature extraction layer and a feature regression layer is obtained.

[0063] Optionally, the second determining module is specifically configured to sample, for each traffic participant, a multimodal probability distribution of latent random variables corresponding to the traffic participant to obtain a plurality of latent random variable samples corresponding to the traffic participant;

[0064] The multimodal prediction trajectory corresponding to each traffic participant is determined by utilizing the feature regression layer of the target trajectory prediction model, the trajectory prediction features corresponding to each traffic participant, and a plurality of latent random variable samples corresponding to each traffic participant.

[0065] From the above content, it can be seen that an embodiment of the present invention provides a motion trajectory prediction method and device, which obtains the historical trajectory and motion attribute information of each traffic participant corresponding to the target object and the corresponding current map information; uses the feature extraction layer of the target trajectory prediction model and the initial features corresponding to each traffic participant to determine the trajectory prediction features corresponding to each traffic participant, wherein the initial features corresponding to the traffic participant include: the historical trajectory and motion attribute information of the traffic participant, and the corresponding historical trajectory and motion attribute information of other traffic participants and the target object, as well as the current map information; uses the feature extraction layer of the target trajectory prediction model and the trajectory prediction features corresponding to each traffic participant to determine the multimodal probability distribution of the latent random variables corresponding to each traffic participant, wherein the latent random variables characterize the behavioral randomness of each traffic participant; uses the feature regression layer of the target trajectory prediction model, the trajectory prediction features corresponding to each traffic participant, and the multimodal probability distribution of the latent random variables corresponding to each traffic participant to determine the multimodal predicted trajectory corresponding to each traffic participant.

[0066] By applying the embodiments of the present invention, the target trajectory prediction model can utilize the latent random variables that have learned the behavioral randomness of each traffic participant, as well as the historical trajectory and motion attribute information of each traffic participant and its corresponding dynamic object information, i.e., the historical trajectory and motion attribute information of other traffic participants and the target object, and the static object information, i.e., the current map information, to fit the latent random variable multimodal probability distribution of each traffic participant, i.e., the latent random variable multimodal prior distribution, which represents the multiple possibilities of the future trajectory of each traffic participant and the target object, and then determine the multimodal predicted trajectory corresponding to each traffic participant, so as to achieve accurate determination of the multimodal predicted trajectory of each traffic participant. The target trajectory prediction model including the latent random variables that have learned the behavioral randomness of each traffic participant is universal for this scenario, and there is no bottleneck constraint in the algorithm design. As the scale of the training data for training the target trajectory prediction model increases, the algorithm's ability to model the future trajectory distribution can be continuously enhanced, and thus the trajectory prediction ability can also be continuously improved. Of course, it is not necessary to achieve all of the advantages described above at the same time when implementing any product or method of the present invention.

[0067] The innovative features of the embodiments of the present invention include:

[0068] 1. Combining the latent random variables that have learned the behavioral randomness of each traffic participant in the target trajectory prediction model, the historical trajectory and motion attribute information of each traffic participant, and the corresponding current map information, a multimodal probability distribution of the latent random variables corresponding to each traffic participant is constructed, and then the multimodal prediction trajectory corresponding to each traffic participant is determined, so as to achieve accurate determination of the multimodal prediction trajectory of each traffic participant. The target trajectory prediction model that includes the latent random variables that have learned the behavioral randomness of each traffic participant is universal for this scenario and does not have any bottleneck constraints in algorithm design. As the scale of training data for training the target trajectory prediction model expands, the algorithm's ability to model future trajectory distribution can be continuously enhanced, and thus the trajectory prediction ability can also be continuously improved.

[0069] 2. Feature processing is performed on the features to be processed corresponding to the traffic participant objects from the feature dimension and time dimension in turn, and the features of different feature dimensions and different time dimensions of the features to be processed are aggregated and extracted to obtain aggregated features. The aggregated features are then fused with the features of each historical moment in the initial features. The above operations are repeated many times to obtain the deep abstract intermediate prediction features corresponding to the traffic participant objects. Subsequently, the graph neural network is used to fuse the intermediate prediction features with the information of each dynamic object and the static object information to determine the trajectory prediction features corresponding to the traffic participant object, so as to provide a basis for ensuring the accuracy of subsequent future trajectory prediction results.

[0070] 3. Through the normalized flow mapping algorithm and the single-modal probability distribution of the latent random variables corresponding to the traffic participants, a multi-modal probability distribution of the latent random variables corresponding to the traffic participants is constructed, providing a basis for the subsequent prediction of the multi-modal trajectories of the traffic participants.

[0071] 4. The initial trajectory prediction model is trained using the sample historical trajectories and sample motion attribute information corresponding to each sample traffic object, the sample historical trajectories, sample motion attribute information and static object information of its corresponding sample dynamic object, and the sample future trajectories corresponding to each sample traffic object. This allows the hidden random variables in the initial trajectory prediction model to learn the randomness of the behavior of each sample traffic object, providing a basis for accurate prediction of the future trajectories of subsequent traffic participants. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely some embodiments of the present invention. Those skilled in the art can derive other drawings based on these drawings without inventive effort.

[0073] Figure 1 A schematic diagram of a flow chart of a motion trajectory prediction method provided by an embodiment of the present invention;

[0074] Figure 2 A schematic diagram of mapping a single-modal probability distribution of a latent random variable to a multi-modal probability distribution of a latent random variable provided by an embodiment of the present invention;

[0075] Figure 3 A schematic diagram of a flow chart of the training process of the target trajectory prediction model provided by an embodiment of the present invention;

[0076] Figure 4 A schematic structural diagram of a motion trajectory prediction device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0077] The following will provide a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0078] It should be noted that the terms "including" and "having" and any variations thereof in the embodiments of the present invention and the accompanying drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to the process, method, product, or apparatus.

[0079] The present invention provides a motion trajectory prediction method and apparatus to reduce the limitations of motion trajectory prediction and better adapt to complex autonomous driving scenarios. The following describes embodiments of the present invention in detail.

[0080] Figure 1 A flow chart of a motion trajectory prediction method provided by an embodiment of the present invention. The method may include the following steps:

[0081] S101: Obtain historical trajectory and motion attribute information of each traffic participant object corresponding to the target object and corresponding current map information.

[0082] The motion trajectory prediction method provided in the embodiments of the present invention can be applied to any electronic device with computing capabilities, such as a terminal or a server. In one implementation, the functional software implementing the motion trajectory prediction method can exist as standalone client software or as a plug-in for currently available client software.

[0083] The target object can be either an autonomous vehicle or an intelligent robot. The target object can obtain the historical trajectory and motion attribute information of each corresponding traffic participant through its sensors. The motion attribute information of the traffic participant may include both motion and attribute information of the traffic participant. The motion information of the traffic participant includes, but is not limited to, information such as the speed and acceleration of the traffic participant. The attribute information of the traffic participant may include, but is not limited to, the type, shape, and size of the traffic participant. The historical trajectory of the traffic participant includes the location and posture information of the traffic participant at each historical moment within a preset time period before the current moment.

[0084] The current moment may refer to the moment at which the electronic device is currently predicting its trajectory. The "history" in the above historical trajectory and historical moment refers to the trajectory generated in the time period before the current moment, relative to the moment at which the electronic device is currently predicting its trajectory.

[0085] The electronic device or a connected storage device can pre-store complete map information of the area where the target object is located. The current map information can be the complete map information, or it can be the map information within the area corresponding to the current position information of the target object at the current moment determined from the complete map information. Both are possible.

[0086] The sensors installed on the target object may include, but are not limited to: image acquisition equipment, wheel speed sensors, radar, IMU (Inertial measurement unit), GPS (Global Positioning System) and GNSS (Global Navigation Satellite System), etc.

[0087] In one scenario, if the target object is an autonomous vehicle, the corresponding traffic participants may include, but are not limited to, motor vehicles, bicycles, tricycles, pedestrians, and animals. If the traffic participant is a motor vehicle, the electronic device may also obtain the vehicle's headlight information, such as whether the turn signal is on or off.

[0088] In one case, the target object is an autonomous driving vehicle, and the current map information may include but is not limited to traffic sign information such as lane lines, parking spaces, sidewalks, traffic signs, traffic arrows, and lampposts. Among them, the traffic sign information included in the current map information can be called static objects, and may also include static objects such as buildings, plants, and other objects with fixed positions in the scene.

[0089] S102: Determine trajectory prediction features corresponding to each traffic participant using the feature extraction layer of the target trajectory prediction model and the initial features corresponding to each traffic participant.

[0090] The initial features corresponding to the traffic participant objects include: the historical trajectory and motion attribute information of the traffic participant objects, the historical trajectory and motion attribute information of other corresponding traffic participant objects and target objects, and the current map information.

[0091] The target trajectory prediction model is trained based on the historical trajectories and motion attribute information of sample traffic objects, the historical trajectories and motion attribute information of their corresponding dynamic objects, static object information, and the future trajectories of each sample traffic object. This target trajectory prediction model is a neural network latent variable model. For clarity, the training process of the target trajectory prediction model will be explained below.

[0092] The sample dynamic objects corresponding to the sample traffic object may include: other dynamic traffic objects in the scene where the sample traffic object is located. The static object information corresponding to the sample traffic object may include: various static objects in the map information corresponding to the scene where the sample traffic object is located.

[0093] The electronic device can input the initial features corresponding to each traffic participant into the feature extraction layer of the target trajectory prediction model, so as to extract the features of the initial features corresponding to the traffic participant through the feature extraction layer of the target trajectory prediction model, and determine the trajectory prediction features corresponding to the traffic participant object, so as to determine the trajectory prediction features corresponding to each traffic participant object.

[0094] S103: Determine the multimodal probability distribution of the latent random variables corresponding to each traffic participant by using the feature extraction layer of the target trajectory prediction model and the trajectory prediction features corresponding to each traffic participant.

[0095] Among them, the latent random variables represent the randomness of the behavior of each traffic participant.

[0096] In this step, the target trajectory prediction model is a model equipped with latent random variables. After the electronic device obtains the trajectory prediction features corresponding to each traffic participant, it uses the trajectory prediction features corresponding to each traffic participant and the latent random variables of the feature extraction layer of the target trajectory prediction model to determine the multimodal probability distribution of the latent random variables corresponding to each traffic participant. This latent random variable represents the randomness and uncertainty of the traffic participant's trajectory.

[0097] In one implementation of the present invention, the step S103 may include the following steps 011-012:

[0098] 011: For each traffic participant, use the feature extraction layer of the target trajectory prediction model and the trajectory prediction features corresponding to the traffic participant to determine the unimodal probability distribution of the latent random variable corresponding to the traffic participant.

[0099] 012: For each traffic participant, the normalized flow mapping algorithm and the unimodal probability distribution of the latent random variable corresponding to the traffic participant are used to obtain the multimodal trajectory distribution corresponding to the latent random variable corresponding to the traffic participant.

[0100] In this implementation, the electronic device first uses the feature extraction layer of the target trajectory prediction model and the trajectory prediction features corresponding to the traffic participant to determine the unimodal probability distribution of the latent random variable corresponding to the traffic participant; then, through the normalizing flow mapping algorithm, the unimodal probability distribution of the latent random variable corresponding to the traffic participant is mapped into a multimodal probability distribution, thereby obtaining the multimodal probability distribution of the latent random variable corresponding to the traffic participant.

[0101] In one case, assuming that the unimodal probability distribution of the latent random variable corresponding to the traffic participant is a multivariate Gaussian distribution, the feature extraction layer of the target trajectory prediction model can output its corresponding mean and variance. Through its corresponding mean and variance, the unimodal probability distribution of the latent random variable corresponding to the traffic participant can be constructed. Subsequently, through the normalized flow mapping algorithm, the unimodal probability distribution of the latent random variable corresponding to the traffic participant is mapped into a multimodal trajectory distribution, and the multimodal probability distribution of the latent random variable corresponding to the traffic participant is obtained, so as to simplify the subsequent target trajectory prediction model to map the randomness of the trajectory, that is, the randomness of the latent random variable, into the multimodal probability distribution of the trajectory space, and achieve better multimodal future trajectory modeling effect. The effect diagram is as follows. Figure 2 As shown, “unimodal distribution” represents the unimodal probability distribution of the latent random variables corresponding to the traffic participants, and “multimodal distribution” represents the multimodal trajectory distribution of the latent random variables corresponding to the traffic participants.

[0102] S104: Determine the multimodal predicted trajectory corresponding to each traffic participant using the feature regression layer of the target trajectory prediction model, the trajectory prediction features corresponding to each traffic participant, and the multimodal probability distribution of the latent random variables corresponding to each traffic participant.

[0103] The electronic device determines the multimodal probability distribution of latent random variables corresponding to each traffic participant. For each traffic participant, the feature regression layer of the target trajectory prediction model is used to fuse the trajectory prediction features corresponding to the traffic participant and the multimodal probability distribution of latent random variables corresponding to the traffic participant to determine the multimodal predicted trajectory corresponding to the traffic participant.

[0104] In one implementation of the present invention, the step S104 may include the following steps 021-022:

[0105] 021: For each traffic participant, sample the multimodal probability distribution of the latent random variables corresponding to the traffic participant to obtain multiple latent random variable samples corresponding to the traffic participant.

[0106] 022: Determine the multimodal prediction trajectory of each traffic participant using the feature regression layer of the target trajectory prediction model, the trajectory prediction features corresponding to each traffic participant, and multiple latent random variable samples corresponding to each traffic participant.

[0107] In this implementation, the electronic device samples the multimodal probability distribution of the latent random variables corresponding to each traffic participant object to obtain multiple latent random variable samples corresponding to the traffic participant object. Then, using the feature regression layer of the target trajectory prediction model, the trajectory prediction features corresponding to the traffic participant object and the multiple latent random variable samples corresponding to the traffic participant object are mapped to the trajectory space, that is, the trajectory prediction features corresponding to the traffic participant object and each latent random variable sample corresponding to the traffic participant object are fused to obtain the multimodal prediction trajectory corresponding to each traffic participant object.

[0108] By applying the embodiments of the present invention, the target trajectory prediction model can utilize the latent random variables that have learned the behavioral randomness of each traffic participant, as well as the historical trajectory and motion attribute information of each traffic participant and its corresponding dynamic object information, i.e., the historical trajectory and motion attribute information of other traffic participants and target objects, and the static object information, i.e., the current map information, to fit the conditional probability distribution of the future trajectory of the participant, i.e., the multimodal probability distribution of the latent random variables corresponding to each traffic participant, and then determine the multimodal predicted trajectory corresponding to each traffic participant, so as to achieve accurate determination of the multimodal predicted trajectory of each traffic participant. Moreover, the target trajectory prediction model that includes the latent random variables that have learned the behavioral randomness of each traffic participant is universal for this scenario, and there is no bottleneck constraint in the algorithm design. As the scale of the training data for training the target trajectory prediction model expands, the algorithm's ability to model the future trajectory distribution can be continuously enhanced, and thus the trajectory prediction ability can also be continuously improved.

[0109] In another embodiment of the present invention, the initial features corresponding to the traffic participant objects are features arranged in chronological order, which include features corresponding to multiple historical moments of the traffic participant objects;

[0110] The S102 may include the following steps 031-032:

[0111] 031: For each traffic participant, use the feature extraction layer of the target trajectory prediction model to repeatedly execute the following steps AC on the initial features corresponding to the traffic participant to determine the intermediate prediction features corresponding to the traffic participant.

[0112] 032: For each traffic participant, the intermediate prediction features corresponding to each static object in the intermediate prediction features corresponding to the traffic participant are fused based on the graph neural network to determine the trajectory prediction features corresponding to the traffic participant.

[0113] The static objects include all static objects in the current map information.

[0114] Step A: Perform nonlinear mapping on the to-be-processed features corresponding to the traffic participant object from the feature dimension to obtain the mapping features corresponding to the traffic participant object.

[0115] The feature to be processed is the initial feature corresponding to the traffic participant object or the intermediate prediction feature corresponding to the traffic participant object generated in the previous iteration.

[0116] Step B: Perform feature aggregation operation on the mapping features from the time dimension to obtain the aggregated features corresponding to the traffic participant object.

[0117] Step C: Fuse the aggregated features with the features at each historical moment in the features to be processed.

[0118] In this implementation, the initial features corresponding to a traffic participant object are features arranged in chronological order, including features corresponding to multiple historical moments of the traffic participant object. It is understood that each historical moment may correspond to multiple features corresponding to the traffic participant object, such as: the traffic participant object's position information, attitude information such as heading angle, speed, shape, type, and size; the position information and attitude information such as heading angle, speed, shape, type, and size of other traffic participants and target objects corresponding to the traffic participant object; and the relative position information and type of the traffic participant object and each static object in the corresponding current map information. The features corresponding to each historical moment are arranged in the order of the information at each historical moment.

[0119] The electronic device can, for each traffic participant, first use the feature extraction layer of the target trajectory prediction model to perform nonlinear mapping on the initial features corresponding to the traffic participant object from the feature dimension, that is, for each historical moment, perform nonlinear mapping on the features corresponding to the historical moment from the feature dimension to obtain the mapping features corresponding to the traffic participant object; then perform feature aggregation operation on the mapping features from the time dimension, that is, perform feature aggregation operation on the mapping features corresponding to each historical moment to obtain the aggregated features corresponding to the traffic participant object; fuse the aggregated features corresponding to the traffic participant object with the features of each historical moment in the initial features, use the fused features as the new features to be processed corresponding to the traffic participant object, and re-execute the steps of first using the feature extraction layer of the target trajectory prediction model to perform nonlinear mapping on the initial features corresponding to the traffic participant object from the feature dimension to obtain the mapping features corresponding to the traffic participant object, until the above steps are repeated multiple times to obtain the intermediate prediction features corresponding to the traffic participant object containing deep abstract features.

[0120] Furthermore, for each traffic participant, the electronic device fuses the intermediate prediction features corresponding to each static object in the intermediate prediction features corresponding to the traffic participant based on the graph neural network to determine the trajectory prediction features corresponding to the traffic participant.

[0121] In another embodiment of the present invention, before S102, the method may further include:

[0122] The process of training the target trajectory prediction model, where Figure 3 As shown, the process includes:

[0123] S301: Obtain an initial trajectory prediction model.

[0124] S302: Obtain sample training information corresponding to each sample traffic object and a sample future trajectory corresponding to each sample traffic object.

[0125] The sample training information corresponding to the sample traffic object includes: the sample historical trajectory and sample motion attribute information of the sample traffic object, the sample historical trajectory, sample motion attribute information and sample static object information of the corresponding sample dynamic object. The sample motion attribute information may include the motion information and attribute information of the sample traffic object, wherein the motion information of the sample traffic object includes but is not limited to: information such as the speed and acceleration of the sample traffic object. The attribute information of the sample traffic object may include but is not limited to: the type, shape and size of the sample traffic object. The sample historical trajectory of the sample traffic object includes: the position information and posture information of the sample traffic object at each historical moment within a preset time period before the sample trajectory is collected. The sample future trajectory of the sample traffic object is: the actual running trajectory of the sample traffic object at the time of sample trajectory collection and the first time period thereafter, including the actual position information and posture information of the sample traffic object at each moment within the time period thereafter and the first time period thereafter.

[0126] The sample dynamic objects corresponding to the sample traffic objects are dynamic objects around the sample traffic objects in the environment, which may include vehicles, pedestrians, animals, etc.; the sample static object information corresponding to the sample traffic objects includes the static objects in the map information of the environment in which the sample traffic objects are located.

[0127] In one case, the target object is an autonomous driving vehicle. Accordingly, the sample vehicle can collect corresponding information for each object in its environment during driving. The sample training information and sample future trajectory corresponding to a sample traffic object can be determined based on the sensor data collected by the sample vehicle through the sensors set thereon.

[0128] In one case, when the sample traffic object is a motor vehicle, the sample training information corresponding to the sample traffic object may also include headlight information of the motor vehicle, such as whether the turn signal light is on or off.

[0129] S303: For each sample traffic object, using the feature extraction layer of the initial trajectory prediction model and the initial sample features corresponding to the sample traffic object, determine the sample prediction features corresponding to the sample traffic object.

[0130] The initial sample features corresponding to the sample traffic object include: sample historical trajectory and sample motion attribute information of the sample traffic object, and the sample historical trajectory, sample motion attribute information and sample static object information of the corresponding sample dynamic object.

[0131] S304: For each sample traffic object, determine the multimodal probability distribution of the latent random variable corresponding to the sample traffic object using the feature extraction layer of the initial trajectory prediction model and the sample prediction features corresponding to the sample traffic object.

[0132] S305: For each sample traffic object, determine the multimodal predicted trajectory corresponding to the sample traffic object using the feature regression layer of the initial trajectory prediction model, the sample prediction features corresponding to the sample traffic object, and the multimodal probability distribution of the latent random variables corresponding to the sample traffic object.

[0133] S306: For each sample traffic object, use a preset variational algorithm to process the sample future trajectory corresponding to the sample traffic object to obtain a variational probability distribution of the latent random variable corresponding to the sample traffic object.

[0134] S307: For each sample traffic object, determine the KL divergence value of the latent random variable corresponding to the sample traffic object using the multimodal probability distribution of the latent random variable corresponding to the sample traffic object and the variational probability distribution of the latent random variable corresponding to the sample traffic object.

[0135] S308 : For each sample traffic object, determine a trajectory reconstruction loss value corresponding to the sample traffic object using the multimodal predicted trajectory corresponding to the sample traffic object and the sample future trajectory corresponding to the sample traffic object.

[0136] S309: For each sample traffic object, use the KL divergence value of the latent random variable corresponding to the sample traffic object and the trajectory reconstruction loss value corresponding to the sample traffic object to construct a variational lower bound of the maximum likelihood function; and determine whether the constructed variational lower bound of the maximum likelihood function is maximized.

[0137] S310: If the variational lower bound of the constructed maximum likelihood function is not maximized, the model parameters of the feature extraction layer and the feature regression layer of the initial trajectory prediction model are adjusted, and the process returns to S303.

[0138] S311: If the variational lower bound of the constructed maximum likelihood function is maximized, it is determined that the initial trajectory prediction model has converged, and a target trajectory prediction model including a feature extraction layer and a feature regression layer is obtained.

[0139] To ensure accurate prediction of the future trajectory of each traffic participant, embodiments of the present invention also include a target trajectory prediction model training process. Accordingly, the electronic device can first obtain an initial trajectory prediction model, which can be a neural network latent variable model; obtain sample training information corresponding to each sample traffic object, and obtain sample future trajectories corresponding to each sample traffic object.

[0140] Furthermore, for each sample traffic object, the electronic device inputs the initial sample features corresponding to the sample traffic object into the feature extraction layer of the initial trajectory prediction model, and uses the feature extraction layer of the initial trajectory prediction model to perform feature extraction and fusion on the initial sample features corresponding to the sample traffic object to determine the sample prediction features corresponding to the sample traffic object. The process of initially using the feature extraction layer of the initial trajectory prediction model to perform feature extraction and fusion on the initial sample features corresponding to the sample traffic object can be referred to the process of feature extraction and fusion of the initial features of the traffic participant objects by the feature extraction layer of the target trajectory prediction model, which will not be repeated here.

[0141] For each sample traffic object, the feature extraction layer of the initial trajectory prediction model and the sample prediction features corresponding to the sample traffic object are used to obtain the unimodal probability distribution of the latent random variable corresponding to the sample traffic object. Then, through the normalized flow mapping algorithm, the unimodal probability distribution of the latent random variable corresponding to the sample traffic object is mapped to the multimodal probability distribution of the latent random variable corresponding to the sample traffic object.

[0142] For each sample traffic object, the sample prediction features corresponding to the sample traffic object and the multimodal probability distribution of the latent random variables corresponding to the sample traffic object are input into the feature regression layer of the initial trajectory prediction model, so that the sample prediction features corresponding to the sample traffic object and the multimodal probability distribution of the latent random variables corresponding to the sample traffic object are fused through the feature regression layer of the initial trajectory prediction model to obtain the multimodal prediction trajectory corresponding to the sample traffic object.

[0143] Subsequently, to ensure the accuracy of the prediction results of the constructed target trajectory prediction model, the multimodal predicted trajectory corresponding to the sample traffic object and the sample future trajectory corresponding to the sample traffic object can be used to construct a variational lower bound for the maximized likelihood function. The constructed variational lower bound for the maximized likelihood function is used to adjust the model parameters of the initial trajectory prediction model, thereby obtaining the final target trajectory prediction model. To construct the variational lower bound for the maximized likelihood function, the electronic device processes the sample future trajectory corresponding to each sample traffic object using a preset variational algorithm to obtain a variational probability distribution of the latent random variable corresponding to the sample traffic object, wherein the preset variational algorithm can be a variational algorithm constructed based on the variational Bayesian principle.

[0144] For each sample traffic object, the KL divergence algorithm is used to determine the KL divergence value of the latent random variable corresponding to the sample traffic object, using the multimodal probability distribution of the latent random variable corresponding to the sample traffic object and the variational probability distribution of the latent random variable corresponding to the sample traffic object. The trajectory reconstruction loss value for the sample traffic object is also determined using the multimodal predicted trajectory corresponding to the sample traffic object and the sample future trajectory corresponding to the sample traffic object. A variational lower bound of the maximum likelihood function is constructed using the trajectory reconstruction loss value corresponding to the sample traffic object and the KL divergence value of the latent random variable corresponding to the sample traffic object. The function value corresponding to the variational lower bound of the maximum likelihood function is calculated to determine whether the variational lower bound of the maximum likelihood function is maximized, that is, whether the function value corresponding to the variational lower bound of the maximum likelihood function is maximized. If the variational lower bound of the maximum likelihood function is not maximized, the model parameters of the feature extraction layer and the feature regression layer of the initial trajectory prediction model are adjusted using a preset optimization algorithm, and the process returns to S203. If the variational lower bound of the maximum likelihood function is maximized, it is determined that the initial trajectory prediction model has converged, and a target trajectory prediction model including the feature extraction layer and the feature regression layer is obtained.

[0145] In one case, the probability distribution corresponding to the multimodal predicted trajectory of the sample traffic object is constructed. The multimodal predicted trajectory corresponding to the sample traffic object can be constructed using the probability distribution, which can be expressed by the following formula (1):

[0146] p(x f |x p ,Φ)=∫p(x f |z,x p ,Φ)p(z|x p ,Φ)dz; (1)

[0147] Among them, x p represents the sample historical trajectory corresponding to the sample traffic object, x f represents the multimodal predicted trajectory corresponding to the sample traffic object; Φ represents other information in the initial sample features corresponding to the sample traffic object except the sample historical trajectory corresponding to the sample traffic object; p(x f |x p ,Φ) represents the probability distribution of the multimodal predicted trajectory of the sample traffic object; z represents the latent random variable; p(z|x p,Φ) represents the multimodal probability distribution of the latent random variable corresponding to the sample traffic object, which is the prior distribution of the latent random variable z given the sample historical trajectory and other information in the initial sample features except the sample historical trajectory corresponding to the sample traffic object. It represents the randomness of the future trajectory of the sample traffic object based on the historical trajectory of the sample traffic object and the surrounding map, that is, the sample static object information and the sample dynamic object as a whole; p(x f |z,x p ,Φ) represents the probability distribution of the multimodal predicted trajectory corresponding to the sample traffic object. This is the probability distribution of the future trajectory given additional information such as the latent random variable, the sample's historical trajectory, and the map. This outputs the predicted future trajectory by comprehensively considering all deterministic and random information. This modeling approach can represent the behavioral randomness of the sample traffic object or traffic participant through the latent random variable z. This randomness is then mapped to the original trajectory data space using a neural network model, namely the initial trajectory prediction model or the target trajectory prediction model. Theoretically, it can fit any future trajectory distribution, demonstrating high versatility and effectiveness.

[0148] Correspondingly, the variational lower bound of the constructed maximum likelihood function can be expressed by the following formula;

[0149] logp(x f |x p ,Φ)≥E q [(x f |z,x p ,Φ)]-KL(q(z|x f ,x p ,Φ)||p(z|x p ,Φ));

[0150] Among them, logp(x f |x p ,Φ) represents the constructed maximum likelihood function, E q [(x f |z,x p ,Φ)] represents the trajectory reconstruction loss value corresponding to the sample traffic object, KL(q(z|x f ,x p ,Φ)||p(z|x p ,Φ) represents the KL divergence value of the latent random variable corresponding to the sample traffic object, E q [(x f |z,x p ,Φ)]-KL(q(z|x f ,x p ,Φ)||p(z|x p ,Φ)) represents the variational lower bound of the maximized likelihood function.

[0151] In this implementation, in the process of constructing the target trajectory prediction model, the historical trajectory, operation attribute information and surrounding static object information of each traffic participant are fully considered. This information presents high-dimensional multi-source heterogeneous characteristics. Through the feature extraction layer of the initial trajectory prediction model, the features of each direction between the features, namely the feature dimension and time dimension, are extracted and fused, thereby achieving the full extraction and fusion of the features corresponding to the traffic participants to support the subsequent model's prediction of future trajectories.

[0152] Corresponding to the above method embodiment, the embodiment of the present invention provides a motion trajectory prediction device, such as Figure 4 As shown, the device may include:

[0153] An acquisition module 410 is configured to obtain historical trajectory and motion attribute information of each traffic participant corresponding to the target object and corresponding current map information;

[0154] The first determination module 420 is configured to determine trajectory prediction features corresponding to each traffic participant using a feature extraction layer of the target trajectory prediction model and initial features corresponding to each traffic participant, wherein the initial features corresponding to the traffic participant include: historical trajectory and motion attribute information of the traffic participant, historical trajectory and motion attribute information of other corresponding traffic participants and the target object, and the current map information;

[0155] The second determination module 430 is configured to determine a multimodal probability distribution of a latent random variable corresponding to each traffic participant using a feature extraction layer of the target trajectory prediction model and trajectory prediction features corresponding to each traffic participant, wherein the latent random variable represents the randomness of the behavior of each traffic participant;

[0156] The third determination module 440 is configured to determine the multimodal predicted trajectory corresponding to each traffic participant using the feature regression layer of the target trajectory prediction model, the trajectory prediction features corresponding to each traffic participant, and the multimodal probability distribution of the latent random variables corresponding to each traffic participant.

[0157] By applying the embodiments of the present invention, the target trajectory prediction model can utilize the latent random variables that have learned the behavioral randomness of each traffic participant, as well as the historical trajectory and motion attribute information of each traffic participant and its corresponding dynamic object information, i.e., the historical trajectory and motion attribute information of other traffic participants and target objects, and the static object information, i.e., the current map information, to fit the conditional probability distribution of the future trajectory of the participant, i.e., the multimodal probability distribution of the latent random variables corresponding to each traffic participant, and then determine the multimodal predicted trajectory corresponding to each traffic participant, so as to achieve accurate determination of the multimodal predicted trajectory of each traffic participant. Moreover, the target trajectory prediction model that includes the latent random variables that have learned the behavioral randomness of each traffic participant is universal for this scenario, and there is no bottleneck constraint in the algorithm design. As the scale of the training data for training the target trajectory prediction model expands, the algorithm's ability to model the future trajectory distribution can be continuously enhanced, and thus the trajectory prediction ability can also be continuously improved.

[0158] In another embodiment of the present invention, the initial features corresponding to the traffic participant objects are features arranged in chronological order, which include features corresponding to multiple historical moments of the traffic participant objects;

[0159] The first determination module 420 is specifically configured to, for each traffic participant, use the feature extraction layer of the target trajectory prediction model to repeatedly perform the following steps AC based on the initial features corresponding to the traffic participant to determine the intermediate prediction features corresponding to the traffic participant;

[0160] For each traffic participant, based on a graph neural network, intermediate prediction features corresponding to each static object in the intermediate prediction features corresponding to the traffic participant are fused to determine a trajectory prediction feature corresponding to the traffic participant, wherein the static objects include each static object in the current map information;

[0161] Step A: Perform nonlinear mapping on the features to be processed corresponding to the traffic participant object from the feature dimension to obtain the mapped features corresponding to the traffic participant object, wherein the features to be processed are the initial features corresponding to the traffic participant object or the intermediate predicted features corresponding to the traffic participant object generated in the previous iteration;

[0162] Step B: performing a feature aggregation operation on the mapping features from a time dimension to obtain the aggregated features corresponding to the traffic participant object;

[0163] Step C: Fusing the aggregated features with the features of each historical moment in the features to be processed.

[0164] In another embodiment of the present invention, the second determination module 430 is specifically configured to determine, for each traffic participant, a unimodal probability distribution of a latent random variable corresponding to the traffic participant using a feature extraction layer of the target trajectory prediction model and trajectory prediction features corresponding to the traffic participant;

[0165] For each traffic participant, the normalized flow mapping algorithm and the single-modal probability distribution of the latent random variables corresponding to the traffic participant are used to obtain the multi-modal probability distribution of the latent random variables corresponding to the traffic participant.

[0166] In another embodiment of the present invention, the apparatus further comprises:

[0167] The training module (not shown in the figure) is configured to train the target trajectory prediction model before determining the trajectory prediction features corresponding to each traffic participant using the feature extraction layer of the target trajectory prediction model and the initial features corresponding to each traffic participant. The training module is specifically configured as follows:

[0168] Obtain an initial trajectory prediction model;

[0169] Obtaining sample training information corresponding to each sample traffic object and a sample future trajectory corresponding to each sample traffic object, wherein the sample training information corresponding to the sample traffic object includes: a sample historical trajectory and sample motion attribute information of the sample traffic object, and a sample historical trajectory, sample motion attribute information, and sample static object information of the corresponding sample dynamic object;

[0170] For each sample traffic object, using the feature extraction layer of the initial trajectory prediction model and the initial sample features corresponding to the sample traffic object, determine the sample prediction features corresponding to the sample traffic object, wherein the initial sample features corresponding to the sample traffic object include: the sample historical trajectory and sample motion attribute information of the sample traffic object, and the sample historical trajectory, sample motion attribute information, and sample static object information of the corresponding sample dynamic object;

[0171] For each sample traffic object, determine the multimodal probability distribution of the latent random variable corresponding to the sample traffic object using the feature extraction layer of the initial trajectory prediction model and the sample prediction features corresponding to the sample traffic object;

[0172] For each sample traffic object, determine the multimodal predicted trajectory corresponding to the sample traffic object using the feature regression layer of the initial trajectory prediction model, the sample prediction features corresponding to the sample traffic object, and the multimodal probability distribution of the latent random variables corresponding to the sample traffic object;

[0173] For each sample traffic object, a preset variational algorithm is used to perform variational processing on the sample future trajectory corresponding to the sample traffic object to obtain the variational distribution probability corresponding to the sample traffic object;

[0174] For each sample traffic object, the KL divergence value of the latent random variable corresponding to the sample traffic object is determined using the multimodal probability distribution of the latent random variable corresponding to the sample traffic object and the variational probability distribution of the latent random variable corresponding to the sample traffic object;

[0175] For each sample traffic object, the trajectory reconstruction loss value corresponding to the sample traffic object is determined using the multimodal predicted trajectory corresponding to the sample traffic object, the latent random variable variational probability distribution, and the sample future trajectory corresponding to the sample traffic object;

[0176] For each sample traffic object, the KL divergence value of the latent random variable corresponding to the sample traffic object and the trajectory reconstruction loss value corresponding to the sample traffic object are used to construct the variational lower bound of the maximum likelihood function; and whether the variational lower bound of the constructed maximum likelihood function is maximized is determined;

[0177] If the variational lower bound of the constructed maximum likelihood function is not maximized, adjusting model parameters of the feature extraction layer and the feature regression layer of the initial trajectory prediction model, and returning to the step of determining, for each sample traffic object, a sample prediction feature corresponding to the sample traffic object using the feature extraction layer of the initial trajectory prediction model and the initial sample features corresponding to the sample traffic object;

[0178] If the variational lower bound of the constructed maximum likelihood function is maximized, it is determined that the initial trajectory prediction model has converged, and the target trajectory prediction model including a feature extraction layer and a feature regression layer is obtained.

[0179] In another embodiment of the present invention, the second determining module 430 is specifically configured to sample, for each traffic participant, the multimodal probability distribution of the latent random variables corresponding to the traffic participant to obtain a plurality of latent random variable samples corresponding to the traffic participant;

[0180] The multimodal prediction trajectory corresponding to each traffic participant is determined by utilizing the feature regression layer of the target trajectory prediction model, the trajectory prediction features corresponding to each traffic participant, and a plurality of latent random variable samples corresponding to each traffic participant.

[0181] The above-mentioned system and device embodiments correspond to the system embodiment and have the same technical effects as the method embodiment. For detailed descriptions, please refer to the method embodiment. The device embodiment is obtained based on the method embodiment. For detailed descriptions, please refer to the method embodiment section and will not be repeated here. It should be understood by those skilled in the art that the accompanying drawings are only schematic diagrams of one embodiment, and the modules or processes in the accompanying drawings are not necessarily required to implement the present invention.

[0182] Those skilled in the art will appreciate that the modules in the apparatuses of the embodiments may be distributed in the apparatuses of the embodiments as described in the embodiments, or may be located in one or more apparatuses different from the embodiments with corresponding changes. The modules in the above embodiments may be combined into one module or further divided into multiple sub-modules.

[0183] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A motion trajectory prediction method, characterized in that: The method comprises: Obtain the historical trajectory and motion attribute information of each traffic participant corresponding to the target object and the corresponding current map information; Determining trajectory prediction features corresponding to each traffic participant using a feature extraction layer of a target trajectory prediction model and initial features corresponding to each traffic participant, wherein the initial features corresponding to the traffic participant include: historical trajectory and motion attribute information of the traffic participant, historical trajectory and motion attribute information of other corresponding traffic participants and the target object, and the current map information; Determine the multimodal probability distribution of latent random variables corresponding to each traffic participant by utilizing the feature extraction layer of the target trajectory prediction model and the trajectory prediction features corresponding to each traffic participant, including: for each traffic participant, determine the unimodal probability distribution of latent random variables corresponding to the traffic participant by utilizing the feature extraction layer of the target trajectory prediction model and the trajectory prediction features corresponding to the traffic participant; for each traffic participant, obtain the multimodal probability distribution of latent random variables corresponding to the traffic participant by utilizing the normalized flow mapping algorithm and the unimodal probability distribution of latent random variables corresponding to the traffic participant, wherein the latent random variables represent the randomness of the behavior of each traffic participant; Determine the multimodal predicted trajectory corresponding to each traffic participant using the feature regression layer of the target trajectory prediction model, the trajectory prediction features corresponding to each traffic participant, and the multimodal probability distribution of the latent random variables corresponding to each traffic participant; Before the step of determining the trajectory prediction features corresponding to each traffic participant using the feature extraction layer of the target trajectory prediction model and the initial features corresponding to each traffic participant, the method further includes: The process of training a target trajectory prediction model includes: Obtain an initial trajectory prediction model; Obtaining sample training information corresponding to each sample traffic object and a sample future trajectory corresponding to each sample traffic object, wherein the sample training information corresponding to the sample traffic object includes: a sample historical trajectory and sample motion attribute information of the sample traffic object, and a sample historical trajectory, sample motion attribute information, and sample static object information of the corresponding sample dynamic object; For each sample traffic object, using the feature extraction layer of the initial trajectory prediction model and the initial sample features corresponding to the sample traffic object, determine the sample prediction features corresponding to the sample traffic object, wherein the initial sample features corresponding to the sample traffic object include: the sample historical trajectory and sample motion attribute information of the sample traffic object, and the sample historical trajectory, sample motion attribute information, and sample static object information of the corresponding sample dynamic object; For each sample traffic object, determine the multimodal probability distribution of the latent random variable corresponding to the sample traffic object using the feature extraction layer of the initial trajectory prediction model and the sample prediction features corresponding to the sample traffic object; For each sample traffic object, determine the multimodal predicted trajectory corresponding to the sample traffic object using the feature regression layer of the initial trajectory prediction model, the sample prediction features corresponding to the sample traffic object, and the multimodal probability distribution of the latent random variables corresponding to the sample traffic object; For each sample traffic object, the preset variational algorithm is used to process the sample future trajectory corresponding to the sample traffic object to obtain the variational probability distribution of the latent random variable corresponding to the sample traffic object; For each sample traffic object, the KL divergence value of the latent random variable corresponding to the sample traffic object is determined using the multimodal probability distribution of the latent random variable corresponding to the sample traffic object and the variational probability distribution of the latent random variable corresponding to the sample traffic object; For each sample traffic object, determine a trajectory reconstruction loss value corresponding to the sample traffic object using the multimodal predicted trajectory corresponding to the sample traffic object and the sample future trajectory corresponding to the sample traffic object; For each sample traffic object, the KL divergence value of the latent random variable corresponding to the sample traffic object and the trajectory reconstruction loss value corresponding to the sample traffic object are used to construct the variational lower bound of the maximum likelihood function; and whether the variational lower bound of the constructed maximum likelihood function is maximized is determined; If the variational lower bound of the constructed maximum likelihood function is not maximized, adjusting model parameters of the feature extraction layer and the feature regression layer of the initial trajectory prediction model, and returning to the step of determining, for each sample traffic object, a sample prediction feature corresponding to the sample traffic object using the feature extraction layer of the initial trajectory prediction model and the initial sample features corresponding to the sample traffic object; If the variational lower bound of the constructed maximum likelihood function is maximized, it is determined that the initial trajectory prediction model has converged, and the target trajectory prediction model including a feature extraction layer and a feature regression layer is obtained.

2. The method according to claim 1, wherein The initial features corresponding to the traffic participant objects are features arranged in chronological order, which include features of multiple historical moments corresponding to the traffic participant objects; The step of determining the trajectory prediction features corresponding to each traffic participant using the feature extraction layer of the target trajectory prediction model and the initial features corresponding to each traffic participant includes: For each traffic participant, use the feature extraction layer of the target trajectory prediction model to repeatedly execute the following steps AC on the initial features corresponding to the traffic participant to determine the intermediate prediction features corresponding to the traffic participant; For each traffic participant, based on a graph neural network, intermediate prediction features corresponding to each static object in the intermediate prediction features corresponding to the traffic participant are fused to determine a trajectory prediction feature corresponding to the traffic participant, wherein the static objects include each static object in the current map information; Step A: Perform nonlinear mapping on the features to be processed corresponding to the traffic participant object from the feature dimension to obtain the mapped features corresponding to the traffic participant object, wherein the features to be processed are the initial features corresponding to the traffic participant object or the intermediate predicted features corresponding to the traffic participant object generated in the previous iteration; Step B: performing a feature aggregation operation on the mapping features from a time dimension to obtain the aggregated features corresponding to the traffic participant object; Step C: Fusing the aggregated features with the features of each historical moment in the features to be processed.

3. The method according to any one of claims 1 to 2, characterized in that The step of determining the multimodal predicted trajectory corresponding to each traffic participant object by using the feature regression layer of the target trajectory prediction model, the trajectory prediction features corresponding to each traffic participant object, and the multimodal probability distribution of the latent random variables corresponding to each traffic participant object includes: For each traffic participant, sampling is performed on the multimodal probability distribution of the latent random variables corresponding to the traffic participant to obtain multiple latent random variable samples corresponding to the traffic participant; The multimodal prediction trajectory corresponding to each traffic participant is determined by utilizing the feature regression layer of the target trajectory prediction model, the trajectory prediction features corresponding to each traffic participant, and a plurality of latent random variable samples corresponding to each traffic participant.

4. A motion trajectory prediction device, characterized in that: The device comprises: An acquisition module configured to obtain historical trajectory and motion attribute information of each traffic participant corresponding to the target object and corresponding current map information; The first determination module is configured to determine trajectory prediction features corresponding to each traffic participant using a feature extraction layer of the target trajectory prediction model and initial features corresponding to each traffic participant, wherein the initial features corresponding to the traffic participant include: historical trajectory and motion attribute information of the traffic participant, historical trajectory and motion attribute information of other corresponding traffic participants and the target object, and the current map information; The second determination module is configured to use the feature extraction layer of the target trajectory prediction model and the trajectory prediction features corresponding to each traffic participant to determine the multimodal probability distribution of the latent random variables corresponding to each traffic participant, wherein the latent random variables represent the randomness of the behavior of each traffic participant; the second determination module is specifically configured to use the feature extraction layer of the target trajectory prediction model and the trajectory prediction features corresponding to the traffic participant to determine the unimodal probability distribution of the latent random variables corresponding to each traffic participant; for each traffic participant, use the normalized flow mapping algorithm and the unimodal probability distribution of the latent random variables corresponding to the traffic participant to obtain the multimodal probability distribution of the latent random variables corresponding to the traffic participant; A third determination module is configured to determine a multimodal predicted trajectory corresponding to each traffic participant using a feature regression layer of the target trajectory prediction model, trajectory prediction features corresponding to each traffic participant, and a multimodal probability distribution of latent random variables corresponding to each traffic participant; The device further comprises: a training module configured to train a target trajectory prediction model before determining trajectory prediction features corresponding to each traffic participant using the feature extraction layer of the target trajectory prediction model and the initial features corresponding to each traffic participant, wherein the training module is specifically configured to obtain the initial trajectory prediction model; Obtaining sample training information corresponding to each sample traffic object and a sample future trajectory corresponding to each sample traffic object, wherein the sample training information corresponding to the sample traffic object includes: a sample historical trajectory and sample motion attribute information of the sample traffic object, and a sample historical trajectory, sample motion attribute information, and sample static object information of the corresponding sample dynamic object; For each sample traffic object, using the feature extraction layer of the initial trajectory prediction model and the initial sample features corresponding to the sample traffic object, determine the sample prediction features corresponding to the sample traffic object, wherein the initial sample features corresponding to the sample traffic object include: the sample historical trajectory and sample motion attribute information of the sample traffic object, and the sample historical trajectory, sample motion attribute information, and sample static object information of the corresponding sample dynamic object; For each sample traffic object, determine the multimodal probability distribution of the latent random variable corresponding to the sample traffic object using the feature extraction layer of the initial trajectory prediction model and the sample prediction features corresponding to the sample traffic object; For each sample traffic object, determine the multimodal predicted trajectory corresponding to the sample traffic object using the feature regression layer of the initial trajectory prediction model, the sample prediction features corresponding to the sample traffic object, and the multimodal probability distribution of the latent random variables corresponding to the sample traffic object; For each sample traffic object, the preset variational algorithm is used to process the sample future trajectory corresponding to the sample traffic object to obtain the variational probability distribution of the latent random variable corresponding to the sample traffic object; For each sample traffic object, the KL divergence value of the latent random variable corresponding to the sample traffic object is determined using the multimodal probability distribution of the latent random variable corresponding to the sample traffic object and the variational probability distribution of the latent random variable corresponding to the sample traffic object; For each sample traffic object, the trajectory reconstruction loss value corresponding to the sample traffic object is determined using the multimodal predicted trajectory corresponding to the sample traffic object, the latent random variable variational probability distribution, and the sample future trajectory corresponding to the sample traffic object; For each sample traffic object, the KL divergence value of the latent random variable corresponding to the sample traffic object and the trajectory reconstruction loss value corresponding to the sample traffic object are used to construct the variational lower bound of the maximum likelihood function; and whether the variational lower bound of the constructed maximum likelihood function is maximized is determined; If the variational lower bound of the constructed maximum likelihood function is not maximized, adjusting model parameters of the feature extraction layer and the feature regression layer of the initial trajectory prediction model, and returning to the step of determining, for each sample traffic object, a sample prediction feature corresponding to the sample traffic object using the feature extraction layer of the initial trajectory prediction model and the initial sample features corresponding to the sample traffic object; If the variational lower bound of the constructed maximum likelihood function is maximized, it is determined that the initial trajectory prediction model has converged, and the target trajectory prediction model including a feature extraction layer and a feature regression layer is obtained.

5. The device according to claim 4, characterized in that The initial features corresponding to the traffic participant objects are features arranged in chronological order, which include features of multiple historical moments corresponding to the traffic participant objects; The first determination module is specifically configured to, for each traffic participant, use the feature extraction layer of the target trajectory prediction model to repeatedly perform the following steps AC based on the initial features corresponding to the traffic participant to determine the intermediate prediction features corresponding to the traffic participant; For each traffic participant, based on a graph neural network, intermediate prediction features corresponding to each static object in the intermediate prediction features corresponding to the traffic participant are fused to determine a trajectory prediction feature corresponding to the traffic participant, wherein the static objects include each static object in the current map information; Step A: Perform nonlinear mapping on the features to be processed corresponding to the traffic participant object from the feature dimension to obtain the mapped features corresponding to the traffic participant object, wherein the features to be processed are the initial features corresponding to the traffic participant object or the intermediate predicted features corresponding to the traffic participant object generated in the previous iteration; Step B: performing a feature aggregation operation on the mapping features from a time dimension to obtain the aggregated features corresponding to the traffic participant object; Step C: Fusing the aggregated features with the features of each historical moment in the features to be processed.

6. The device according to any one of claims 4 to 5, characterized in that The second determination module is specifically configured to sample, for each traffic participant, a multimodal probability distribution of latent random variables corresponding to the traffic participant to obtain a plurality of latent random variable samples corresponding to the traffic participant; The multimodal prediction trajectory corresponding to each traffic participant is determined by utilizing the feature regression layer of the target trajectory prediction model, the trajectory prediction features corresponding to each traffic participant, and a plurality of latent random variable samples corresponding to each traffic participant.

Images