A trajectory prediction method, device and equipment

Abstract

The application provides a trajectory prediction method, device and equipment, the method comprising: inputting the first target feature of the target moving object obtained into the trained intention simulator to obtain the first prediction data of K future time points; determining the target position of the future time point based on the Gaussian parameters of the candidate position corresponding to the future time point, determining the second target feature of the target moving object based on the target positions of the K future time points; inputting the second target feature into the trained trajectory predictor to obtain the second prediction data of the K future time points; determining the target mode of the future time point based on the prediction probability of each mode corresponding to the future time point, determining the prediction position of the future time point based on the Gaussian parameters of the candidate position corresponding to the future time point, and determining the prediction trajectory of the target moving object based on the target mode and the prediction position of the K future time points. Through the technical scheme of the application, an accurate and reliable prediction trajectory can be obtained, and the quality of the prediction trajectory can be improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a trajectory prediction method, apparatus and device. Background Technology

[0002] In autonomous driving scenarios, various sensors collect information from the target scene, such as information on various moving objects (e.g., motor vehicles, non-motor vehicles, pedestrians, etc.). Based on the information of the moving objects, the future trajectory of the moving objects is predicted at future moments. Then, based on the future trajectory of the moving objects at future moments, the trajectory of the vehicle is determined. Autonomous driving is achieved based on the trajectory of the vehicle, which helps to better plan the trajectory of the vehicle and thus reduce the incidence of traffic accidents.

[0003] However, there is no reasonable way in the relevant technology to predict the future trajectory of a moving object based on its information, resulting in problems such as inaccurate future trajectory prediction. Summary of the Invention

[0004] This application provides a trajectory prediction method, the method comprising:

[0005] The acquired first target features of the target moving object are input into a trained intent simulator to obtain first prediction data for K future moments; wherein, the target moving object is any moving object in the target scene, the first target features are matched with the historical trajectory of the target moving object and the map information of the target scene, and the first prediction data includes Gaussian parameters of candidate positions;

[0006] The target position at the future time is determined based on the Gaussian parameters of the candidate positions at the future time, and the second target feature of the target moving object is determined based on the target positions at K future time times.

[0007] The second target feature is input into the trained trajectory predictor to obtain the second prediction data for K future time moments. The second prediction data includes the prediction probability of each mode and the Gaussian parameters of the candidate positions.

[0008] The target mode at a future time is determined based on the predicted probability of each mode at the future time, the predicted position at the future time is determined based on the Gaussian parameters of the candidate positions at the future time, and the predicted trajectory of the target moving object is determined based on the target modes and predicted positions at K future times.

[0009] This application provides a trajectory prediction device, the device comprising:

[0010] The acquisition module is used to input the acquired first target features of the target moving object into the trained intent simulator to obtain first prediction data for K future moments; wherein, the target moving object is any moving object in the target scene, the first target features are matched with the historical trajectory of the target moving object and the map information of the target scene, and the first prediction data includes Gaussian parameters of candidate positions;

[0011] The determination module is used to determine the target position of the target moving object for each of the K future moments based on the Gaussian parameters of the candidate positions corresponding to that future moment, and to determine the second target feature of the target moving object based on the target positions of the K future moments.

[0012] The acquisition module is further configured to input the second target feature of the target moving object into the trained trajectory predictor to obtain second prediction data for K future time moments. The second prediction data includes the prediction probability of each mode and the Gaussian parameters of the candidate positions.

[0013] The determining module is further configured to, for each of the K future moments, determine the target mode of the future moment based on the prediction probability of each mode corresponding to the future moment, determine the predicted position of the future moment based on the Gaussian parameters of the candidate position corresponding to the future moment, and determine the predicted trajectory of the target moving object based on the target mode and the predicted position of the K future moments.

[0014] This application provides an electronic device, including: a processor and a machine-readable storage medium, the machine-readable storage medium storing machine-executable instructions that can be executed by the processor; the processor is used to execute the machine-executable instructions to implement the trajectory prediction method of the example above in this application.

[0015] As can be seen from the above technical solutions, in this embodiment, the first prediction data for the future time can be determined based on the intent simulator, and the target position for the future time can be determined based on the first prediction data. After obtaining the target position for the future time, the target position for the future time is not directly output. Instead, the second target feature is determined based on the target position for the future time, and the second target feature is input to the trajectory predictor to obtain the second prediction data for the future time. The predicted position for the future time is determined based on the second prediction data, and then the predicted trajectory of the target moving object is determined based on the predicted position for the future time. Thus, an accurate and reliable predicted trajectory (i.e., the future trajectory) can be obtained, and the trajectory of the target moving object in the target scene can be predicted more accurately in the future time, improving the quality of the predicted trajectory. It can predict the position of the target moving object in the future time, and based on the predicted trajectory of each moving object, it helps the vehicle to complete better decision-making, planning and control. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments of this application or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings of the embodiments of this application.

[0017] Figure 1 This is a flowchart illustrating a trajectory prediction method in one embodiment of this application;

[0018] Figure 2 This is a flowchart illustrating a trajectory prediction method in one embodiment of this application;

[0019] Figure 3 This is a schematic diagram illustrating the vectorization operation in one embodiment of this application;

[0020] Figure 4 This is a schematic diagram of a probability graph of future moments in one embodiment of this application;

[0021] Figure 5 This is the network model involved in the trajectory prediction method in one embodiment of this application;

[0022] Figure 6 This is a schematic diagram of the trajectory prediction device in one embodiment of this application;

[0023] Figure 7 This is a hardware structure diagram of an electronic device according to one embodiment of this application. Detailed Implementation

[0024] The terminology used in the embodiments of this application is for the purpose of describing particular embodiments only and is not intended to limit the application. The singular forms “a,” “the,” and “the” as used in this application and claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to any and all possible combinations comprising one or more of the associated listed items.

[0025] It should be understood that although the terms first, second, third, etc., may be used to describe various information in embodiments of this application, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" may also be interpreted as "when," "when," or "in response to a determination."

[0026] This application proposes a trajectory prediction method, which can be applied to any type of device. See [link to relevant documentation]. Figure 1 The diagram shown is a flowchart of the method, which may include:

[0027] Step 101: Input the acquired first target features of the target moving object into the trained intent simulator to obtain first prediction data for K future time points. Here, K can be a positive integer, the target moving object is any moving object in the target scene, the first target features are matched with the historical trajectory of the target moving object and the map information of the target scene, and the first prediction data includes the Gaussian parameters of the candidate positions.

[0028] Step 102: Determine the target position at the future time based on the Gaussian parameters of the candidate positions at the future time, and determine the second target feature of the moving object based on the target positions at K future time times.

[0029] Step 103: Input the second target features into the trained trajectory predictor to obtain the second prediction data for K future time moments. The second prediction data may include the prediction probability of each mode (the mode can be the direction of movement such as left turn, straight, right turn, etc.) and the Gaussian parameters of the candidate position.

[0030] Step 104: Determine the target mode for the future time based on the predicted probability of each mode corresponding to the future time, determine the predicted position for the future time based on the Gaussian parameters of the candidate positions corresponding to the future time, and determine the predicted trajectory of the target moving object based on the target modes and predicted positions of K future times.

[0031] For example, before step 101, a first target feature of the moving object can also be obtained. This can be achieved by determining the first target feature based on the historical trajectory of the moving object and the map information of the target scene. The method of obtaining this first target feature is not limited, as long as it matches the historical trajectory of the moving object and the map information of the target scene. For instance, the first target feature of the moving object can be obtained as follows: determine the initial trajectory feature of the moving object based on its historical trajectory, and determine the initial map feature based on the map information of the target scene; determine the trajectory stacking feature of the moving object based on its initial trajectory feature and the initial trajectory features of other moving objects in the target scene; determine the historical moment feature based on the initial trajectory feature and the trajectory stacking feature, and determine the first target feature of the moving object based on the historical moment feature and the initial map feature. Of course, the above is just an example and is not a limitation.

[0032] For example, determining historical moment features based on the initial trajectory features and trajectory stacking features of the target moving object may include, but is not limited to: determining the temporal fusion features of the target moving object based on the initial trajectory features and trajectory stacking features, and performing max pooling operation on the temporal fusion features along the time dimension to obtain the historical moment features of the target moving object.

[0033] For example, in step 102, for each of the K future times, the target position for that future time is determined based on the Gaussian parameter of the candidate position corresponding to that future time. This may include, but is not limited to: for each candidate position corresponding to that future time, determining the predicted probability value corresponding to that candidate position based on the candidate position and the Gaussian parameter; and based on the predicted probability value corresponding to each candidate position, determining the candidate position with the largest predicted probability value as the target position for that future time.

[0034] For example, in step 102, determining the second target feature of the target moving object based on the target position at K future times may include, but is not limited to: determining the initial future feature of the target moving object based on the target position at K future times; determining the target future feature of the target moving object based on the initial future feature and weight coefficient of the target moving object, and the initial future feature and weight coefficient of other moving objects in the target scene; and determining the second target feature of the target moving object based on the target future feature.

[0035] For example, determining the second target feature of a moving target based on the target's future features may include, but is not limited to: determining the fused features of the moving target based on the initial map features, the target's future features, and the first target features corresponding to the map information of the target scene; and determining the second target feature based on the fused features of the moving target and the trajectory stacking features of the moving target.

[0036] For example, the weight coefficient of the target moving object can be determined to be a fixed value; with the target position of the target moving object at a future time as the center, the Gaussian distribution of the target positions of other moving objects at a future time can be determined, and the weight coefficient of other moving objects can be determined based on the Gaussian distribution.

[0037] As can be seen from the above technical solutions, in this embodiment, the first prediction data for the future time can be determined based on the intent simulator, and the target position for the future time can be determined based on the first prediction data. After obtaining the target position for the future time, the target position for the future time is not directly output. Instead, the second target feature is determined based on the target position for the future time, and the second target feature is input to the trajectory predictor to obtain the second prediction data for the future time. The predicted position for the future time is determined based on the second prediction data, and then the predicted trajectory of the target moving object is determined based on the predicted position for the future time. Thus, an accurate and reliable predicted trajectory (i.e., the future trajectory) can be obtained, and the trajectory of the target moving object in the target scene can be predicted more accurately in the future time, improving the quality of the predicted trajectory. It can predict the position of the target moving object in the future time, and based on the predicted trajectory of each moving object, it helps the vehicle to complete better decision-making, planning and control.

[0038] The technical solutions described above in the embodiments of this application will be explained below in conjunction with specific application scenarios.

[0039] In autonomous driving scenarios, information about various moving objects (such as motor vehicles, non-motor vehicles, and pedestrians) within the target scene can be collected. Based on this information, the future trajectories of these objects at future moments can be predicted. The vehicle's trajectory can then be determined based on these future trajectories, ultimately enabling autonomous driving. However, there is no reasonable implementation method in related technologies for predicting the future trajectories of moving objects, leading to problems such as inaccurate predictions. For example, trajectory prediction methods using physical models (such as Kalman filtering and Markov hiding models) can predict the future trajectories of moving objects. However, these methods cannot properly model the intentions of the moving objects at future moments, resulting in inaccurate trajectory predictions.

[0040] In response to the above findings, this application proposes a trajectory prediction method. This method determines first prediction data for a future time based on an intent simulator, then determines the target position for that future time based on the first prediction data. The target position for that future time reflects the intention of the moving object. A second target feature is determined based on the target position for that future time, and the future trajectory of the moving object is predicted based on the second target feature. In other words, the future trajectory of the moving object is predicted based on its future intention, thus predicting the future trajectory based on the object's intention (i.e., the future trajectory is determined by the intention). This is a trajectory prediction method based on intent decomposition trajectory modeling, which can accurately predict future trajectories, improve the quality of predicted trajectories, and help the vehicle achieve better decision-making, planning, and control.

[0041] See Figure 2The diagram shown illustrates the process of a trajectory prediction method, which may include:

[0042] Step 201: Determine the initial trajectory features of the target moving object based on its historical trajectory, and determine the initial map features based on the map information of the target scene (such as a high-precision map).

[0043] For example, in order to achieve autonomous driving of this vehicle, the trajectory prediction method can be applied to the vehicle's on-board terminal, which predicts the trajectory of each moving object in the target scene, and then assists the vehicle in achieving autonomous driving based on the predicted trajectory of each moving object in the target scene.

[0044] The target scenario can include the area where the vehicle is traveling, such as the area the vehicle has already traveled and the area it may travel in the future. The map information of the target scenario can be a high-precision map of the area, which can include lane lines, pedestrian crossings, traffic lights, traffic signs, etc.

[0045] The target scene includes multiple moving objects (also called agents), such as motor vehicles, non-motor vehicles, and pedestrians. These moving objects are all obstacles to the vehicle. When predicting the trajectory of a moving object within the target scene, that moving object is called the target moving object. For example, if the target scene contains moving object 1, moving object 2, and moving object 3, when predicting the trajectory of moving object 1, moving object 1 is called the target moving object; when predicting the trajectory of moving object 2, moving object 2 is called the target moving object; and when predicting the trajectory of moving object 3, moving object 3 is called the target moving object.

[0046] For example, in order to predict the trajectory (i.e., future trajectory) of a target moving object over a future period of time, the historical trajectory of the target moving object over a historical period of time can be obtained first. The historical trajectory can include the historical position of the target moving object at each moment within a period T0 preceding the current moment, and the predicted trajectory can include the predicted position of the target moving object at each moment within a period T1 following the current moment.

[0047] For example, time T0 can be divided into multiple historical moments, allowing us to obtain the historical position of the target moving object at each historical moment, and these historical positions constitute the historical trajectory. Similarly, time T1 can be divided into multiple future moments (e.g., K future moments), requiring us to predict the predicted position of the target moving object at each future moment, and these predicted positions constitute the predicted trajectory.

[0048] After obtaining the historical trajectory of the target moving object, the initial trajectory features of the target moving object are determined based on the historical trajectory. For example, feature extraction is performed on the historical trajectory to obtain the initial trajectory features of the target moving object. In one possible implementation, the historical trajectory can be vectorized to obtain a historical trajectory vector, and the historical trajectory vector can be input into a trained feature extractor, which then extracts features from the historical trajectory vector to obtain the initial trajectory features.

[0049] See Figure 3 The diagram illustrates the vectorization operation on historical trajectories and high-precision maps. For two adjacent historical positions in the historical trajectory, a vector value can be determined based on the difference between the two adjacent historical positions. All vector values ​​then form the historical trajectory vector. There are no restrictions on this vectorization operation.

[0050] The feature extractor can be composed of a multilayer perceptron or other types of network models. It can be a deep learning model or a neural network model; the structure of the feature extractor is not limited, as long as it can perform feature extraction. The feature extractor can be pre-trained based on sample data, and the training process is not restricted. Based on the trained feature extractor, after obtaining the historical trajectory vector, this vector can be input into the feature extractor, which extracts features from the historical trajectory vector to obtain the initial trajectory features of the target moving object. These initial trajectory features can be denoted as A. N .

[0051] For example, to predict the trajectory of a moving target object over a future period (i.e., its future trajectory), map information of the target scene, such as a high-precision map, can be obtained. The high-precision map includes lane lines, and each lane line has multiple locations (i.e., actual physical locations). After obtaining the map information of the target scene, initial map features are determined based on this information. For instance, feature extraction is performed on the map information (such as multiple locations of lane lines, which can be understood as trajectories) to obtain initial map features. In one possible implementation, the map information can be vectorized to obtain a map information vector. This map information vector is then input into a trained feature extractor (which may be the same as or different from the feature extractor used for historical trajectories). The feature extractor then extracts features from the map information vector to obtain the initial map features.

[0052] See Figure 3The diagram illustrates the vectorization operation on a high-precision map (i.e., map information). For two adjacent positions in the lane lines of a high-precision map, a vector value can be determined based on the difference between the two adjacent positions. All vector values ​​then form the map information vector. There are no restrictions on this vectorization operation.

[0053] The feature extractor can be composed of a multilayer perceptron or other types of network models; there are no restrictions. Based on the trained feature extractor, after obtaining the map information vector, the map information vector can be input into the feature extractor, which then extracts features from the map information vector to obtain the initial map features. These initial map features can be denoted as A. M .

[0054] In summary, based on a trained feature extractor (taking the same feature extractor as an example), the historical trajectory vector can be input into the feature extractor to obtain the initial trajectory features A of the target moving object. N The map information vector is then input into the feature extractor to obtain the initial map features A. M For example, the input to the feature extractor is the historical trajectory of the target moving object and a high-precision map. This data is then vectorized using data abstraction methods to obtain abstract map data. and trajectory abstract data As input to the feature extractor, the feature extractor outputs the initial trajectory features. and initial map features

[0055] Step 202: Determine the trajectory stacking features of the target moving object based on the initial trajectory features of the target moving object and the initial trajectory features of other moving objects in the target scene. Determine the historical moment features of the target moving object based on the initial trajectory features and the trajectory stacking features of the target moving object.

[0056] For example, since the moving objects in the target scene influence each other, that is, the trajectory of one moving object will affect the trajectory of other moving objects, it is also necessary to consider the mutual influence between different moving objects. Based on this, the trajectory relationship between different moving objects can be determined by analyzing the initial trajectory features of the target moving object and the initial trajectory features of other moving objects (the determination method of the initial trajectory features of other moving objects is the same as that of the target moving object).

[0057] In one possible implementation, after obtaining the initial trajectory features of the target moving object and the initial trajectory features of other moving objects, a historical trajectory feature interactor can be used to simulate the mutual influence between different moving objects to obtain the historical moment features of the target moving object. For example, the historical trajectory feature interactor may include a historical trajectory moment interactor and a historical trajectory time series fusion processor. Specifically, the initial trajectory features of the target moving object and the initial trajectory features of other moving objects can be input to the historical trajectory moment interactor, which processes these features to obtain the trajectory stacking features of the target moving object. Alternatively, the initial trajectory features of the target moving object and the trajectory stacking features can be input to the historical trajectory time series fusion processor, which processes these features to obtain the historical moment features of the target moving object.

[0058] For example, the historical trajectory time-lapse interaction mechanism can be composed of a Transformer (a network structure incorporating an attention mechanism) or other types of network models. It can be a deep learning model or a neural network model; the structure of the historical trajectory time-lapse interaction mechanism is not limited. The historical trajectory time-lapse interaction mechanism can be pre-trained based on sample data; the training process is not limited. Based on the trained historical trajectory time-lapse interaction mechanism, the initial trajectory features of the target moving object and the initial trajectory features of other moving objects can be input into the historical trajectory time-lapse interaction mechanism. The historical trajectory time-lapse interaction mechanism outputs the trajectory stacking features of the target moving object, which can be denoted as F. N Trajectory stacking features F N This represents the interaction and influence between moving objects at each moment.

[0059] For example, the basic architecture of a historical trajectory time-lapse interactive platform is a Transformer architecture. It simulates the interaction between different moving objects by calculating the self-attention of the initial trajectory features. For instance, the historical trajectory time-lapse interactive platform can consist of multiple layers of Transformers. The inputs of the Transformer are Query, Key, and Value. Query represents the query tensor in the Transformer, Key represents the key tensor, and Value represents the value tensor. In the historical trajectory time-lapse interactive platform, the initial trajectory features of the target moving object can be... As a Query for Transformer, it can retrieve the initial trajectory features of other moving objects. Using these as the key and value of the Transformer, the output of the Transformer becomes the trajectory stacking feature F. N That is, the final tensors are stacked into Trajectory stacking feature F N This represents the interaction and influence between moving objects at each moment.

[0060] For example, the historical trajectory temporal fusion processor can be composed of Transformers or other types of network models. It can be a deep learning model or a neural network model; there are no restrictions on its structure. The historical trajectory temporal fusion processor can be pre-trained based on sample data; there are no restrictions on the training process. Based on the trained historical trajectory temporal fusion processor, the initial trajectory features A of the target moving object can be... N The stacking feature F of the trajectory of the target moving object N The input is fed into the historical trajectory time series fusion unit, which outputs the historical moment features of the target moving object. These historical moment features can be denoted as F.

[0061] For example, historical trajectory time series fusion is used to extract high-dimensional features (i.e., trajectory stacking features F) resulting from the interactions between different moving objects. N The historical trajectory time-series fusion is performed to obtain the final historical trajectory features (i.e., historical moment features F) of the target moving object. The basic architecture of the historical trajectory time-series fusion unit is the Transformer architecture. The Transformer's query comes from the features extracted after position encoding of the moment, that is, the initial trajectory features of the target moving object. As the Query of the Transformer, the Key and Value of the Transformer are the output of the historical trajectory moment interactor, which is the stacked trajectory features of the target moving object. Serving as the key and value of the Transformer. After passing through multiple layers of Transformers, the historical moment characteristics of the target moving object can be determined based on the output of the Transformer.

[0062] In one possible implementation, the output of the Transformer is a temporal fusion feature. Temporal fusion features can be used Historical moment characteristics of the target moving object

[0063] In another possible implementation, the output of the Transformer is a temporal fusion feature. Temporal fusion features can be analyzed along the time dimension. Take the maximum value to obtain the historical moment features of the target moving object. In summary, it can be seen that based on the initial trajectory features of the target moving object... Stacking features of the trajectory of the target moving object Using Transformer, the temporal fusion features of the target moving object can be determined. Then, the temporal fusion features are analyzed along the time dimension. Perform max pooling to obtain the historical moment features of the target moving object.

[0064] Step 203: Determine the first target feature of the moving object based on the historical moment feature and the initial map feature. For example, since the historical moment feature is obtained based on the historical trajectory of the moving object and the initial map feature is obtained based on the map information of the target scene, the first target feature can be matched with the historical trajectory of the moving object and the map information of the target scene.

[0065] For example, since the high-precision map of the target scene also affects the trajectory of the target moving object, that is, the trajectory needs to follow the route of the high-precision map, it is necessary to consider the relationship between the trajectory of the target moving object and the map information of the target scene. Based on this, the first target feature of the target moving object can be determined based on the historical moment features of the target moving object and the initial map features of the high-precision map.

[0066] In one possible implementation, the interaction between the moving object and the high-precision map can be simulated using a moving object-map feature interactor to obtain the first target feature of the moving object. For example, the historical time features of the moving object and the initial map features of the high-precision map can be input into the moving object-map feature interactor, which can then process the data based on the historical time features and the initial map features to obtain the first target feature of the moving object.

[0067] For example, the moving object and map feature interaction mechanism can be composed of a Transformer or other types of network models. It can be a deep learning model or a neural network model; the network structure is not limited. The moving object and map feature interaction mechanism can be pre-trained based on sample data; the training process is not restricted. Based on the trained moving object and map feature interaction mechanism, the historical time-step features F and the initial map features A can be... MThe input is given to the moving object and map feature interaction device, which outputs the first target feature of the moving object. The first target feature can be denoted as F1.

[0068] For example, a moving object and map feature interaction mechanism can use a Transformer to model the influence of map information on the trajectory of the moving object. This influence can be achieved using the Transformer's cross-attention mechanism, and can be composed of multiple Transformers stacked together. Within this interaction mechanism, initial map features can be... As a Query of Transformer, it can capture the historical features of the target moving object. As the key and value of a Transformer, after passing through multiple layers of Transformers, the output features of the Transformer can be... The primary target feature is the moving object.

[0069] Step 204: Input the first target feature of the target moving object into the trained intention simulator to obtain the first prediction data for K future time moments. The first prediction data includes the Gaussian parameters of the candidate positions.

[0070] For example, the predicted trajectory can include the predicted position of the target moving object at each time point within a time interval T1 after the current time. Therefore, time T1 can be divided into multiple future times, taking K future times as an example, where K can be a positive integer, such as 25, 30, etc. For instance, time T1 can be decomposed into a time interval of length t, i.e., K = T1 / t, where the interval between two adjacent future times is t.

[0071] For example, the intent simulator can be composed of a multilayer perceptron or other types of network models. The intent simulator can be a deep learning model or a neural network model; its structure is not limited, as long as it can output the first prediction data for K future time points. The intent simulator can be pre-trained based on sample data, and the training process is not restricted. Based on the trained intent simulator, after obtaining the first target features of the moving object, these features can be input into the intent simulator. The intent simulator then determines the first prediction data for K future time points based on these features and outputs the first prediction data for K future time points.

[0072] For each future time point, the first predicted data for that future time point includes Gaussian parameters for the candidate positions, such as σ, μ, and ρ, where σ represents the standard deviation, μ represents the mean, and ρ represents the probability of accepting the null hypothesis incorrectly, serving as a criterion for testing whether the data follows a normal distribution. Clearly, K future time points correspond to K first predicted data points, and each first predicted data point includes a set of Gaussian parameters, such as σ, μ, and ρ.

[0073] In summary, it can be seen that the input features of the intent simulator are the first target features. First target characteristics After passing through the intention simulator, we can obtain the Gaussian parameters of the position distribution on the top view at K future time points (denoted as the Gaussian parameters of the candidate positions): σ∈R n×2×K μ∈R n×2×K and ρ∈R n×1×K .

[0074] In one possible implementation, the training process of the intent simulator can be supervised using the maximum negative likelihood function, as shown in the formula below. Of course, other loss functions can also be used for supervised learning, and there are no restrictions on the loss function of the intent simulator.

[0075]

[0076] In the above formula, L G The loss value can be represented by the loss value L. G Adjust the network parameters of the intent simulator. There are no restrictions on the adjustment process; for example, gradient descent can be used to adjust the network parameters of the intent simulator. μ x μ y σ x σ y ρ represents the Gaussian parameters output by the intent simulator, and μ represents the Gaussian parameters. x and σ x μ represents the Gaussian parameter in the horizontal direction. y and σ y Y represents the Gaussian parameter in the vertical direction. x Y represents the true value of the horizontal position. y This represents the true value of the position in the vertical direction. In summary, sample data (such as historical trajectories and high-precision maps) and corresponding label data can be obtained. Based on steps 201-203, the first target feature can be obtained from the sample data, and this first target feature is input into the intent simulator to be trained to obtain μ. x μ y σ x σ y Gaussian parameters such as ρ, and label data including Y xY y Since the truth value is equal to the maximum negative likelihood function, the loss value L can be determined based on this. G Based on the loss value L G By adjusting the network parameters of the intent simulator and iterating multiple times, a trained intent simulator can be obtained.

[0077] When the first target feature is input into the intent simulator to be trained, μ at K future time points can be obtained. x μ y σ x σ y ρ, and the label data can include Y at K future times. x Y y Therefore, for each future time point, the loss value L at that future time point can be determined based on the maximum negative likelihood function. G Then, the loss value L at K future times can be used as a basis. G Calculate the overall loss value, such as all loss values ​​L. G The sum of these values ​​is used as the overall loss value, and the network parameters of the intent simulator are adjusted based on this overall loss value.

[0078] Of course, the above is just an example of training an intent simulator, and there are no limitations on it.

[0079] Step 205: For each of the K future moments, determine the target position for that future moment based on the Gaussian parameters corresponding to that future moment (i.e., the Gaussian parameters of the candidate positions).

[0080] For example, a high-precision map can correspond to multiple candidate locations. Assuming the width of the high-precision map of the target scene is W and the height is H, it can be divided into W*H candidate locations, such as candidate location (0,0), candidate location (0,1), ..., candidate location (0,W-1), candidate location (1,0), candidate location (1,1), ..., candidate location (1,W-1), ..., candidate location (H-1,0), candidate location (H-1,1), ..., candidate location (H-1,W-1).

[0081] For example, for each candidate position, the predicted probability value corresponding to the candidate position can be determined based on the candidate position and the Gaussian parameter; based on the predicted probability value corresponding to each candidate position, the candidate position with the highest predicted probability value can be determined as the target position at the future time.

[0082] For example, for a candidate position (x, y), where x belongs to [0, H-1] and y belongs to [0, W-1], the predicted probability value corresponding to the candidate position (x, y) can be determined by the following formula. Of course, the following formula is just an example.

[0083] S(x,y)=N(x-μ x ,σ x ;y-μ y ,σ y )

[0084] In the above formula, μ x μ y σ x σ y Let S(x,y) represent the Gaussian parameters, x and y represent the coordinates of the candidate position (x,y), N is a Gaussian distribution, and S(x,y) represent the predicted probability value corresponding to the candidate position (x,y). Clearly, for each candidate position, the predicted probability value corresponding to that candidate position can be obtained using the above formula.

[0085] After obtaining the predicted probability value for each candidate position, a probability map can be generated based on the predicted probability value for each candidate position. (A probabilistic map can also be called a heatmap.) A probabilistic map can include the predicted probability value for each candidate location on a high-precision map. See [link / reference]. Figure 4 The image shows an example of a probability graph for a future time. Based on the predicted probability value corresponding to each candidate position, the candidate position with the highest predicted probability value can be determined as the target position for that future time.

[0086] In summary, for each of the K future moments, the target location at that future moment can be determined; that is, the K target locations corresponding to the K future moments can be obtained.

[0087] Step 206: Determine the initial future features of the target moving object based on the target position at K future times, determine the target future features of the target moving object based on the initial future features of the target moving object, and determine the fused features of the target moving object based on the target future features of the target moving object.

[0088] For example, after obtaining the target positions at K future moments, the target positions at K future moments can reflect the intention of the target moving object at future moments. Therefore, the future trajectory of the target moving object at future moments can be predicted based on the target positions at K future moments, thereby predicting the future trajectory based on the intention of the target moving object (i.e. the future trajectory is determined by the intention), and the future trajectory can be accurately predicted.

[0089] For example, the intention of a moving target object in the future can be analyzed by a future moment interactor. For instance, the target positions at K future moments can be input into the future moment interactor, which can then determine the initial future features based on the K future moment target positions, determine the target future features based on the initial future features, and determine the fused features of the moving target object based on the target future features.

[0090] For example, the future time-moment interactor can be composed of a multilayer perceptron and a Transformer, or it can be composed of other types of network models. The future time-moment interactor can be a deep learning model or a neural network model; there are no restrictions on the network structure of the future time-moment interactor. The future time-moment interactor can be pre-trained based on sample data; there are no restrictions on this training process. Based on the trained future time-moment interactor (such as a multilayer perceptron and a Transformer), the target positions at K future time moments can be input into the trained multilayer perceptron, which determines the initial future features based on the K future target positions. After obtaining the initial future features, the target future features can be determined based on the initial future features. After obtaining the target future features, the target future features can be input into the trained Transformer, which processes them to obtain the fused features of the target moving object.

[0091] For each future time point, the probability graph corresponding to that future time point can be generated. The highest response value in the top view is taken as the target position at that future moment. The K future target positions (i.e., the K target positions) are input to the multilayer perceptron of the future moment interactor. The multilayer perceptron processes the K target positions to obtain the initial future features, which are denoted as...

[0092] After obtaining the initial future features, the target future features of the target moving object can be determined based on the initial future features of the target moving object, the weight coefficient of the target moving object, the initial future features of other moving objects (the determination method of the initial future features of other moving objects can be the same as that of the target moving object), and the weight coefficients of other moving objects.

[0093] For example, the following formula can be used to determine the future characteristics of a moving object:

[0094] F future '=F future (0)*W0+F future (1)*W1+...+F future (N)*W N

[0095] In the above formula, F future 'Represents the target's future characteristics, F' future (0) represents the initial future characteristics of the target moving object, W0 represents the weight coefficient of the target moving object, and it is assumed that the number of other moving objects is N, where N is a positive integer. future (1) represents the initial future characteristics of the first other moving object, W1 represents the weight coefficient of the first other moving object, and so on, F future (N) represents the initial future features of the Nth other moving object, W N This represents the weight coefficient of the Nth other moving object.

[0096] The weight coefficient of the target moving object can be a fixed value, such as 1. Using the target moving object's future target position as the center, a Gaussian distribution of the future target positions of other moving objects is determined, and the weight coefficients of these other moving objects are determined based on this Gaussian distribution. For example, based on the future target positions of other moving objects, a Gaussian distribution centered on the target moving object's future target position is determined, yielding the weight coefficients of the other moving objects. There are no restrictions on this determination process. Based on the Gaussian distribution, the larger the target position of other moving objects is compared with the target position of the target moving object, the smaller the weight coefficient of other moving objects. Conversely, the smaller the target position of other moving objects is compared with the target position of the target moving object, the larger the weight coefficient of other moving objects.

[0097] After obtaining the target future features of the moving object, the fused features of the moving object can be determined based on the initial map features, the target future features, and the first target features. For example, the initial map features can be used... Target future features F future The first target feature F1 is input into the Transformer of the future time-time interactor, and the Transformer is based on the initial map features. Target future features F future The first target feature F1 is processed to obtain the fused features of the moving target object. These fused features can be denoted as... For example, the initial features of the map can be... As the Query of Transformer, the target future feature F future 'Using the first target feature F1 as the key of the Transformer, and the Transformer's value as the value. After processing by the Transformer, the Transformer's output feature can be used as the fused feature.' It can also be expressed by the following formula: F fusion =MLP([F1,F future The fused features are the output features of the interactor at future time steps.

[0098] Step 207: Based on the fused features of the target moving object and the trajectory stacking features of the target moving object (see step 202 for the determination method), determine the second target feature of the target moving object.

[0099] In one possible implementation, a second target feature of the moving object can be determined using a temporal feature fusion processor. For example, the fused features of the moving object and the trajectory stacking features of the moving object can be input into the temporal feature fusion processor, which then processes the fused features and trajectory stacking features to obtain the second target feature.

[0100] For example, the temporal feature fusion processor can be composed of Transformers or other types of network models. It can be a deep learning model or a neural network model; there are no restrictions on its structure. The temporal feature fusion processor can be pre-trained based on sample data; there are no restrictions on the training process. Based on the trained temporal feature fusion processor, the fused features can be... Trajectory stacking features F N The input is fed into a temporal feature fusion unit, which outputs a second target feature. This second target feature can be denoted as...

[0101] For example, temporal feature fusion machines are used to process the fused features. Trajectory stacking features F N By fusing the components, the second target feature is obtained. The temporal feature fusion processor can use a Transformer. The Transformer's query comes from the features after the time step has been position-encoded, and the features of future time steps are also represented by position encoding, that is, the fused features. As a Query for Transformer, in addition, the trajectory stacking features are also included. As the key and value of the Transformer. After passing through multiple Transformer layers, the output features of the Transformer can be used as the second target features.

[0102] Step 208: Input the second target feature of the target moving object into the trained trajectory predictor to obtain the second prediction data for K future time moments. The second prediction data may include the prediction probability of each mode (the mode can be the direction of movement such as left turn, straight, right turn, etc.) and the Gaussian parameters of the candidate position.

[0103] For example, a trajectory predictor may include a trajectory prediction head and a trajectory scoring head. The trajectory prediction head outputs Gaussian parameters for candidate positions, and the trajectory scoring head outputs the prediction probability for each modality. The trajectory prediction head can be composed of a multilayer perceptron or other types of network models. It can be a deep learning model or a neural network model; the structure of the trajectory prediction head is not limited, as long as it can output Gaussian parameters for K future time steps. The trajectory prediction head can be pre-trained based on sample data, and this training process is not limited. Similarly, the trajectory scoring head can be composed of a multilayer perceptron or other types of network models. It can be a deep learning model or a neural network model; the structure of the trajectory scoring head is not limited, as long as it can output the prediction probability for each modality at K future time steps. The trajectory scoring head can also be pre-trained based on sample data, and this training process is not limited.

[0104] Based on the trained trajectory prediction head and trajectory scoring head, after obtaining the second target feature of the moving target object, the second target feature can be input into the trajectory predictor. The trajectory prediction head of the trajectory predictor determines the Gaussian parameters for K future time moments based on the second target feature, and the trajectory scoring head of the trajectory predictor determines the prediction probability of each mode at K future time moments based on the second target feature.

[0105] For each future time step, the second prediction data for that future time step includes Gaussian parameters of the candidate locations, such as σ, μ, and ρ. Therefore, the input features of the trajectory predictor are the second target features. Second target characteristics After passing through the trajectory prediction head of the trajectory predictor, Gaussian parameters of the position distribution on the top view at K future time points can be obtained: σ∈R n×2×K μ∈R n×2×K and ρ∈R n×1×K .

[0106] The second prediction data for this future moment also includes the prediction probability of each modality. Assuming there are P modalities, such as left turn, straight ahead, right turn, etc., the second prediction data includes the prediction probabilities of each of the P modalities, representing the prediction probability of that modality at that future moment. Therefore, the input feature of the trajectory predictor is the second target feature. Second target characteristics After the trajectory is scored by the trajectory predictor, the predicted probabilities of P modes at K future time points can be obtained. That is, the predicted probabilities of P modes at each future time point can be denoted as...

[0107] In one possible implementation, the training process of the trajectory prediction head of the trajectory predictor can be supervised by using the maximum negative likelihood function, which is shown in the following formula. Of course, other loss functions can also be used for supervised learning, and there are no restrictions on the loss function.

[0108]

[0109] In the above formula, L J The loss value can be represented by the loss value L. J The network parameters of the trajectory prediction head are adjusted, and there are no restrictions on the process of adjusting these network parameters. x μ y σ x σ y ρ represents the Gaussian parameters output by the trajectory prediction head, and μ represents the Gaussian parameters. x and σ x μ represents the Gaussian parameter in the horizontal direction. y and σ y Y represents the Gaussian parameter in the vertical direction. x Y represents the true value of the horizontal position. y Represents the true value of the position in the vertical direction.

[0110] In summary, sample data (such as historical trajectories and high-precision maps) and corresponding label data can be obtained. Based on steps 201-207, the second target feature can be obtained from the sample data, and the second target feature is input into the trajectory prediction head to be trained to obtain μ. x μ y σ x σ y Gaussian parameters such as ρ, and label data including Y x Y y Since the truth value is equal to the maximum negative likelihood function, the loss value L can be determined based on this. J Based on the loss value L J By adjusting the network parameters of the trajectory prediction head and iterating multiple times, a trained trajectory prediction head can be obtained. When the second target feature is input into the trajectory prediction head to be trained, μ at K future time points can be obtained. x μ y σ x σ y ρ, and the label data can include Y at K future times.x Y y Therefore, for each future time point, the loss value L at that future time point can be determined based on the maximum negative likelihood function. J Then, the loss value L at K future times can be used as a basis. J Calculate the overall loss value, such as all loss values ​​L. J The sum of these values ​​serves as the overall loss value, which is then used to adjust the network parameters of the trajectory prediction head. Of course, the above is merely a training example and is not intended to be limiting.

[0111] In one possible implementation, the training process of the trajectory scoring head of the trajectory predictor can be supervised using the cross-entropy loss function. Of course, other loss functions can also be used for supervised learning; there are no restrictions on the specific loss function. The loss value of the cross-entropy loss function can be denoted as L. S It can be based on the loss value L S The network parameters of the trajectory scoring head are adjusted, and the adjustment process is not restricted. In summary, sample data (such as historical trajectories and high-precision maps) and corresponding label data can be obtained. Based on steps 201-207, a second target feature can be obtained from the sample data, and this second target feature is input into the trajectory scoring head to be trained to obtain the prediction probability of each mode. The label data includes the true mode (e.g., the mode closest to the endpoint of the true trajectory is considered the true mode). Obviously, if the mode with the highest prediction probability is the same as the true mode, the prediction result is accurate; if the mode with the highest prediction probability is different from the true mode, the prediction result is inaccurate. Therefore, the loss value L can be determined based on the cross-entropy loss function (determined based on the relationship between the mode with the highest prediction probability and the true mode). S Based on the loss value L S By adjusting the network parameters of the trajectory scoring head and iterating multiple times, a trained trajectory scoring head can be obtained. When the second target feature is input into the trajectory scoring head to be trained, the predicted probability of each mode at K future time points can be obtained, and the label data can include the true modes at K future time points. Therefore, for each future time point, the loss value L at that future time point can be determined based on the cross-entropy loss function. S Then, the loss value L at K future times can be used as a basis. S Calculate the overall loss value, such as all loss values ​​L. S The sum of these values ​​serves as the overall loss value, which is then used to adjust the network parameters of the trajectory scoring head. Of course, the above is merely a training example and is not intended to impose limitations.

[0112] In one possible implementation, the trajectory prediction head and trajectory scoring head of the trajectory predictor can also be jointly trained, for example, based on the loss value L.J and loss value L S Determine the target loss value, and adjust the network parameters of the trajectory prediction head and the trajectory scoring head based on the target loss value.

[0113] In one possible implementation, the intention simulator, the trajectory prediction head of the trajectory predictor, and the trajectory scoring head can also be jointly trained, for example, based on the loss value L. G Loss value L J and loss value L S To determine the target loss value, use the following formula: L = L S +L G +L J Then, based on the target loss value, the network parameters of the intent simulator, the trajectory prediction head, and the trajectory scoring head are adjusted, and there are no restrictions on the adjustment process of these network parameters.

[0114] Step 209: For each of the K future moments, determine the target mode of the future moment based on the predicted probability of each mode corresponding to the future moment, determine the predicted position of the future moment based on the Gaussian parameters of the candidate position corresponding to the future moment, and determine the predicted trajectory of the target moving object (i.e. the future trajectory of the future moment) based on the target modes and predicted positions of the K future moments.

[0115] For example, based on the predicted probability of each mode, the mode corresponding to the highest predicted probability can be taken as the target mode at a future time, indicating that the target mode of motion may be adopted at that future time.

[0116] For example, a high-precision map can correspond to multiple candidate locations. Assuming the width of the high-precision map of the target scene is W and the height is H, it can be divided into W*H candidate locations, such as candidate location (0,0), candidate location (0,1), ..., candidate location (0,W-1), ..., candidate location (H-1,0), candidate location (H-1,1), ..., candidate location (H-1,W-1). For each candidate location, the predicted probability value corresponding to the candidate location can be determined based on the candidate location and the Gaussian parameter. Based on the predicted probability value corresponding to each candidate location, the candidate location with the highest predicted probability value can be determined as the predicted location at that future time.

[0117] For example, for a candidate position (x, y), the predicted probability value corresponding to the candidate position (x, y) can be determined by the following formula: S(x, y) = N(x - μ) x ,σ x ;y-μ y ,σ y ), μ x μy σ x σ y Let S(x,y) represent the Gaussian parameters, x and y represent the coordinates of the candidate position (x,y), N is a Gaussian distribution, and S(x,y) represent the predicted probability value corresponding to the candidate position (x,y). Clearly, the predicted probability value corresponding to each candidate position can be obtained using the above formula.

[0118] After obtaining the predicted probability value for each candidate position, a probability map can be generated based on the predicted probability value for each candidate position. (A probability map can also be called a heatmap.) A probability map can include the predicted probability value of each candidate location on a high-precision map. Based on the predicted probability value of each candidate location, the candidate location with the highest predicted probability value can be determined as the predicted location for that future time.

[0119] In summary, for each of the K future moments, the target mode and predicted position at that future moment can be determined. That is, K target positions and K target modes corresponding to the K future moments can be obtained. Thus, the predicted trajectory of the moving target object can be determined based on the target modes and predicted positions at the K future moments. For example, the predicted trajectory can include the predicted position at the 1st future moment, the predicted position at the 2nd future moment, ..., the predicted position at the Kth future moment, where the mode at the predicted position at the 1st future moment is the target mode at the 1st future moment, the mode at the predicted position at the 2nd future moment is the target mode at the 2nd future moment, ..., the mode at the predicted position at the Kth future moment is the target mode at the Kth future moment. Therefore, the predicted trajectory of the moving target object is obtained, completing the trajectory prediction process.

[0120] In one possible implementation, see Figure 5 As shown, trajectory prediction methods may involve historical trajectory feature extractors, map feature extractors, historical trajectory feature interactors, moving object and map feature interactors, intent simulators, future moment interactors, temporal feature fusion devices, and trajectory predictors.

[0121] For the historical trajectory feature extractor, the historical trajectory can be vectorized to obtain the historical trajectory vector, and the historical trajectory vector can be input into the historical trajectory feature extractor. The historical trajectory feature extractor extracts features from the historical trajectory vector to obtain the initial trajectory features.

[0122] For the map feature extractor, the map information can be vectorized to obtain a map information vector. The map information vector is then input into the map feature extractor (which may be the same as or different from the historical trajectory feature extractor). The map feature extractor extracts features from the map information vector to obtain the initial map features.

[0123] The historical trajectory feature interactor, comprising a historical trajectory time-series interactor and a historical trajectory temporal fusion unit, inputs the initial trajectory features of the target moving object and the initial trajectory features of other moving objects to the historical trajectory time-series interactor, which then outputs the stacked trajectory features of the target moving object. The initial trajectory features and the stacked trajectory features of the target moving object are then input to the historical trajectory temporal fusion unit, which outputs the historical time-series features of the target moving object.

[0124] For the moving object and map feature interaction device, historical moment features and initial map features can be input into the moving object and map feature interaction device, which will then process them based on the historical moment features and initial map features to obtain the first target feature of the target moving object.

[0125] For the intent simulator, the first target feature of the moving target object can be input to the intent simulator. The intent simulator determines the first prediction data for K future time moments based on the first target feature and outputs the first prediction data for K future time moments. The first prediction data includes the Gaussian parameters of the candidate positions.

[0126] For the future time-time interactor, the target positions of K future times can be input into the future time-time interactor. The future time-time interactor determines the initial future features based on the target positions of K future times, determines the target future features based on the initial future features, and determines the fused features based on the target future features.

[0127] For the temporal feature fusion processor, the fused features of the target moving object and the trajectory stacked features of the target moving object can be input into the temporal feature fusion processor, which then processes the fused features of the target moving object and the trajectory stacked features of the target moving object to obtain the second target feature.

[0128] For the trajectory predictor, the second target feature of the moving target object can be input into the trajectory predictor. The trajectory predictor processes the second target feature to obtain the second prediction data for K future time moments. The second prediction data can include the prediction probability of each mode and the Gaussian parameters of the candidate positions.

[0129] As can be seen from the above technical solutions, the embodiments of this application can obtain accurate and reliable predicted trajectories (i.e., future trajectories), more accurately predict the trajectory of target moving objects in the target scene at future moments, improve the quality of predicted trajectories, and predict the position of target moving objects at future moments. Based on the predicted trajectories of each moving object, it helps the vehicle to complete better decision-making, planning, and control. A trajectory prediction method based on intent decomposition modeling is designed, which can predict the position of target moving objects at future moments, while helping the vehicle to complete better decision-making, planning, and control. The intent at future moments is expressed through heatmaps. The features at future moments are refined through a probability-based feature fusion method. The features are further refined through a scene interaction method based on cross-attention, ultimately improving performance.

[0130] Based on the same concept as the methods described above, this application proposes a trajectory prediction device, see [link to relevant documentation]. Figure 6 The diagram shown is a structural schematic of the trajectory prediction device, which may include:

[0131] The acquisition module 61 is used to input the acquired first target features of the target moving object into the trained intention simulator to obtain first prediction data for K future moments; wherein, the target moving object is any moving object in the target scene, the first target features are matched with the historical trajectory of the target moving object and the map information of the target scene, and the first prediction data includes Gaussian parameters of candidate positions;

[0132] The determination module 62 is used to determine the target position of the target moving object for each of the K future moments based on the Gaussian parameters of the candidate positions corresponding to the future moment, and to determine the second target feature of the target moving object based on the target positions of the K future moments.

[0133] The acquisition module 61 is further configured to input the second target feature of the target moving object into the trained trajectory predictor to obtain second prediction data for K future moments, the second prediction data including the prediction probability of each mode and the Gaussian parameters of the candidate positions;

[0134] The determining module 62 is further configured to, for each of the K future moments, determine the target mode of the future moment based on the prediction probability of each mode corresponding to the future moment, determine the predicted position of the future moment based on the Gaussian parameters of the candidate position corresponding to the future moment, and determine the predicted trajectory of the target moving object based on the target mode and the predicted position of the K future moments.

[0135] For example, the determining module 62 is further configured to determine the initial trajectory features of the target moving object based on the historical trajectory of the target moving object, and determine the initial map features based on the map information of the target scene; determine the trajectory stacking features of the target moving object based on the initial trajectory features of the target moving object and the initial trajectory features of other moving objects in the target scene; determine the historical time features based on the initial trajectory features of the target moving object and the trajectory stacking features, and determine the first target feature based on the historical time features and the initial map features.

[0136] For example, when determining the historical moment features based on the initial trajectory features and the stacked trajectory features of the target moving object, the determining module 62 is specifically used to: determine the temporal fusion features of the target moving object based on the initial trajectory features and the stacked trajectory features of the target moving object, and perform max pooling operation on the temporal fusion features along the time dimension to obtain the historical moment features of the target moving object.

[0137] For example, when the determining module 62 determines the target position for the future time based on the Gaussian parameters of the candidate positions corresponding to the future time, it specifically performs the following: for each candidate position corresponding to the future time, it determines the predicted probability value of the candidate position based on the candidate position and the Gaussian parameters; based on the predicted probability value corresponding to each candidate position, it determines the candidate position with the largest predicted probability value as the target position for the future time.

[0138] For example, when the determining module 62 determines the second target feature of the target moving object based on the target positions at the K future times, it is specifically used to: determine the initial future features of the target moving object based on the target positions at the K future times; determine the target future features of the target moving object based on the initial future features and weight coefficients of the target moving object, and the initial future features and weight coefficients of other moving objects in the target scene; and determine the second target feature based on the target future features.

[0139] For example, when determining the second target feature based on the target future feature, the determining module 62 is specifically used to: determine the fused feature of the target moving object based on the map initial feature, the target future feature and the first target feature corresponding to the map information of the target scene; and determine the second target feature based on the fused feature of the target moving object and the trajectory stacking feature of the target moving object.

[0140] For example, the determining module 62 is further configured to determine the weight coefficient of the target moving object as a fixed value; and to determine the Gaussian distribution of the target positions of other moving objects at future times, centered on the target position of the target moving object at a future time, and to determine the weight coefficient of other moving objects based on the Gaussian distribution.

[0141] Based on the same application concept as the above method, this application proposes an electronic device, see [link to application]. Figure 7 As shown, the electronic device includes a processor 71 and a machine-readable storage medium 72, the machine-readable storage medium 72 storing machine-executable instructions that can be executed by the processor 71; the processor 71 is used to execute the machine-executable instructions to implement the trajectory prediction method disclosed in the above example of this application.

[0142] Based on the same concept as the above method, this application also provides a machine-readable storage medium storing a plurality of computer instructions, which, when executed by a processor, can implement the trajectory prediction method disclosed in the above examples of this application.

[0143] The aforementioned machine-readable storage medium can be any electronic, magnetic, optical, or other physical storage device that can contain or store information, such as executable instructions, data, etc. For example, machine-readable storage media can be: RAM (Random Access Memory), volatile memory, non-volatile memory, flash memory, storage drives (such as hard disk drives), solid-state drives, any type of storage disk (such as optical discs, DVDs, etc.), or similar storage media, or combinations thereof.

[0144] The systems, devices, modules, or units described in the above embodiments can be implemented by a computer entity or by a product with a certain function. A typical implementation device is a computer, which can be a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email sending and receiving device, game console, tablet computer, wearable device, or any combination of these devices.

[0145] For ease of description, the above devices are described separately by function as various units. Of course, in implementing this application, the functions of each unit can be implemented in one or more software and / or hardware.

[0146] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, embodiments of this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0147] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0148] Furthermore, these computer program instructions can also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in the process. Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0149] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0150] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A trajectory prediction method, characterized in that, The method includes: The acquired first target features of the target moving object are input into a trained intent simulator to obtain first prediction data for K future moments; wherein, the target moving object is any moving object in the target scene, the first target features are matched with the historical trajectory of the target moving object and the map information of the target scene, and the first prediction data includes Gaussian parameters of candidate positions; The target position at a future time is determined based on the Gaussian parameters of the candidate positions at future times. A second target feature of the target moving object is determined based on the target positions at K future times. Specifically, the initial future features of the target moving object are determined based on the target positions at the K future times. The target future features of the target moving object are determined based on the initial future features and weight coefficients of the target moving object, the initial future features and weight coefficients of other moving objects in the target scene. The fused features of the target moving object are determined based on the target future features, the initial map features corresponding to the map information of the target scene, and the first target feature. The second target feature is determined based on the fused features and the trajectory stacking features of the target moving object. The trajectory stacking features of the target moving object are determined based on the initial trajectory features of the target moving object and the initial trajectory features of other moving objects in the target scene. The second target feature is input into the trained trajectory predictor to obtain the second prediction data for K future time moments. The second prediction data includes the prediction probability of each mode and the Gaussian parameters of the candidate positions. The target mode at a future time is determined based on the predicted probability of each mode at the future time, the predicted position at the future time is determined based on the Gaussian parameters of the candidate positions at the future time, and the predicted trajectory of the target moving object is determined based on the target modes and predicted positions at K future times.

2. The method according to claim 1, characterized in that, Before inputting the acquired first target features of the moving target object into the trained intention simulator to obtain the first prediction data for K future time moments, the method further includes: The initial trajectory features of the target moving object are determined based on its historical trajectory, and the initial map features are determined based on the map information of the target scene. The trajectory stacking features of the target moving object are determined based on the initial trajectory features of the target moving object and the initial trajectory features of other moving objects in the target scene; Historical moment features are determined based on the initial trajectory features of the target moving object and the trajectory stacking features, and first target features are determined based on the historical moment features and the initial map features.

3. The method according to claim 2, characterized in that, The determination of historical moment features based on the initial trajectory features and the trajectory stacking features of the target moving object includes: Based on the initial trajectory features and the trajectory stacking features of the target moving object, the temporal fusion features of the target moving object are determined, and max pooling is performed on the temporal fusion features along the time dimension to obtain the historical moment features of the target moving object.

4. The method according to claim 1, characterized in that, Determining the target position at the future time based on the Gaussian parameters of the candidate positions at the future time includes: For each candidate position corresponding to the future time, a predicted probability value is determined based on the candidate position and the Gaussian parameter; based on the predicted probability value corresponding to each candidate position, the candidate position with the highest predicted probability value is determined as the target position for the future time.

5. The method according to claim 1, characterized in that, The method further includes: The weight coefficient of the target moving object is determined to be a fixed value; with the target position of the target moving object at a future time as the center, the Gaussian distribution of the target positions of the other moving objects at a future time is determined, and the weight coefficient of the other moving objects is determined based on the Gaussian distribution.

6. A trajectory prediction device, characterized in that, The device includes: The acquisition module is used to input the acquired first target features of the target moving object into the trained intent simulator to obtain first prediction data for K future moments; wherein, the target moving object is any moving object in the target scene, the first target features are matched with the historical trajectory of the target moving object and the map information of the target scene, and the first prediction data includes Gaussian parameters of candidate positions; The determination module is used to determine the target position of the target moving object for each of the K future moments based on the Gaussian parameters of the candidate positions corresponding to that future moment, and to determine the second target feature of the target moving object based on the target positions of the K future moments. The acquisition module is further configured to input the second target feature of the target moving object into the trained trajectory predictor to obtain second prediction data for K future time moments. The second prediction data includes the prediction probability of each mode and the Gaussian parameters of the candidate positions. The determining module is further configured to, for each of the K future moments, determine the target mode of the future moment based on the prediction probability of each mode corresponding to the future moment, determine the predicted position of the future moment based on the Gaussian parameters of the candidate position corresponding to the future moment, and determine the predicted trajectory of the target moving object based on the target mode and the predicted position of the K future moments. Specifically, when the determining module determines the second target feature of the target moving object based on the target positions at the K future times, it is used to: determine the initial future features of the target moving object based on the target positions at the K future times; determine the target future features of the target moving object based on the initial future features of the target moving object and the weight coefficient of the target moving object, the initial future features of other moving objects in the target scene and the weight coefficients of other moving objects; determine the fused features of the target moving object based on the target future features of the target moving object, the initial map features corresponding to the map information of the target scene and the first target feature; and determine the second target feature based on the fused features and the trajectory stacking features of the target moving object, wherein the trajectory stacking features of the target moving object are determined based on the initial trajectory features of the target moving object and the initial trajectory features of other moving objects in the target scene.

7. The apparatus according to claim 6, Its features are, The determining module is further configured to: determine the initial trajectory features of the target moving object based on its historical trajectory; determine the initial map features based on the map information of the target scene; determine the trajectory stacking features of the target moving object based on the initial trajectory features of the target moving object and the initial trajectory features of other moving objects in the target scene; determine historical time features based on the initial trajectory features of the target moving object and the trajectory stacking features; and determine the first target feature based on the historical time features and the initial map features. Specifically, when the determining module determines the historical moment features based on the initial trajectory features and the trajectory stacking features of the target moving object, it is used to: determine the temporal fusion features of the target moving object based on the initial trajectory features and the trajectory stacking features of the target moving object, and perform max pooling operation on the temporal fusion features along the time dimension to obtain the historical moment features of the target moving object; Specifically, when the determining module determines the target position for the future time based on the Gaussian parameters of the candidate positions corresponding to the future time, it is used to: for each candidate position corresponding to the future time, determine the predicted probability value of the candidate position based on the candidate position and the Gaussian parameters; and based on the predicted probability value corresponding to each candidate position, determine the candidate position with the largest predicted probability value as the target position for the future time. The determining module is further configured to determine the weight coefficient of the target moving object as a fixed value; and to determine the Gaussian distribution of the target positions of other moving objects at future times, centered on the target position of the target moving object at a future time, and to determine the weight coefficient of other moving objects based on the Gaussian distribution.

8. An electronic device, characterized in that, include: A processor and a machine-readable storage medium, the machine-readable storage medium storing machine-executable instructions that can be executed by the processor; The processor is configured to execute machine-executable instructions to implement the method of any one of claims 1-5.

Images