A target trajectory prediction method

Abstract

This invention discloses a target trajectory prediction method, comprising the following steps: Based on the target's historical trajectory, multi-level motion features and corresponding training labels are extracted using differential encoding; a paired interaction graph is established based on the multi-level motion features, and motion-interaction-based dot product attention is calculated; the trajectory code is calculated based on the obtained dot product attention and multi-level motion features, and decoded from temporal, spatial, and motion feature dimensions to obtain the predicted trajectory; during training, backpropagation training is performed using the decoded trajectory and training labels, and during testing / inference, the predicted position trajectory of the target at future times is output. This invention utilizes target trajectories to construct and extract interactive behaviors between targets in complex scenes; it achieves prediction of the target's future trajectory from multiple dimensions of temporal, spatial, and motion features, resulting in higher prediction accuracy compared to existing methods.

Description

Technical Field

[0001] This invention relates to the application of target trajectory prediction, specifically to a target trajectory prediction method under complex interactive conditions. Background Technology

[0002] With the development of artificial intelligence and autonomous driving technologies, analyzing the historical trajectory of a target and predicting its future trajectory plays an important role in the identification-tracking-prediction-planning technology path for assisted driving and autonomous driving, and is also of great significance for improving social safety and operational efficiency.

[0003] Because the motion of a target is influenced by various factors (such as motion characteristics, environmental information, and interactive behaviors), the target's trajectory is highly arbitrary. Existing methods are commonly used in simple scenarios with single conditions (a pedestrian trajectory prediction method based on spatiotemporal graph attention networks, CN115376103A), but their performance is poor when predicting nonlinear trajectories of multiple targets in complex scenarios (such as intersections, roundabouts, etc.). Furthermore, with the increase in participants and the escalation of scene complexity, determining the impact of non-explicit interactive behaviors between targets on target trajectory prediction is a widespread problem. Existing methods often utilize additional information such as driving intentions and high-precision maps (a method for joint prediction of surrounding vehicle driving intentions and trajectories for autonomous vehicles, CN115158364A) for trajectory prediction. However, due to the limitations in obtaining additional information such as driving intentions, relying solely on the target's historical location information to analyze the interaction characteristics between targets and then predict the target's future trajectory has a wide range of applicability and high application value. Summary of the Invention

[0004] Purpose of the invention: In order to solve the problems existing in the prior art, the purpose of this invention is to provide a target trajectory prediction method that utilizes the historical location information of the target to construct interactive influences, so as to provide trajectory prediction information for assisted driving and autonomous driving systems.

[0005] Technical solution: A target trajectory prediction method, comprising the following steps:

[0006] (1) Using the trajectory position data collected by sensors such as LiDAR and cameras, multi-level motion features are obtained by differential coding techniques; during the model training process, multi-level labels are obtained by using a similar differential coding method for the label position data.

[0007] (2) Establish a pairwise interaction relationship graph, use the multi-level motion features of each target to calculate the dot product attention value based on motion-interaction, and combine the multi-level motion features to calculate the trajectory code.

[0008] (3) Using a stereo decoder, decoding is performed from the time, space and feature dimensions; in space, the influence of hypothetical predicted trajectories of different targets on other targets is calculated; in the feature dimension, the influence of high-level features on low-level features is calculated.

[0009] Preferably, in step (1), the multi-level motion features include δ-layer results, and each layer of features is obtained by encoding after δ differencing of the target's historical position trajectory sequence X; further, in step (1), the training labels of the dual are obtained by performing the same δ differencing on the target's future position trajectory sequence Y.

[0010] Preferably, in step (2), the pairwise paired interaction graph is a fully connected graph established for all targets in the same scene. The calculation of the relationship between targets in the graph is as follows: using the multi-level trajectory encoding of the current target i Calculate query Q δ Utilize the remaining target Ω in the scene i Multi-level trajectory coding Calculate the shared key K and the multi-level value V δ Through matrix Q δ The attention value is calculated by the dot product of K and then compared with the value V. δ Multiply to obtain the remaining target Ω i Motion-interaction effects on the current target.

[0011] Preferably, the decoding process in step (3) adds spatial decoding and motion feature decoding to the general temporal decoding; spatially, the position Y of the current target i at time t is decoded. i,t Depends on the other targets Ω in the current scenario i Hypothetical predicted trajectory Y at time t-1 Ωi,t-1 Again, the influence of motion-interaction attention is assessed through encoding and computation; in terms of motion feature dimensions, the low-level feature units in the recurrent unit of the decoder at time t for target i. The computation depends on higher-level feature units.

[0012] Beneficial effects: Compared with the prior art, the present invention has the following significant advancements: It utilizes only the position trajectory data of targets in complex scenarios, combined with a high-efficiency deep learning model, to construct the interaction influence between targets in complex scenarios from the perspective of the inherent motion and social interactivity of the targets; The present invention comprehensively analyzes the formation mechanism of target trajectories from the dimensions of space, time and motion features, and generates predicted trajectories that meet the non-collision requirements from the perspective of hypothetical prediction, thus meeting the needs of interactive behavior modeling and target trajectory prediction in complex scenarios. Attached Figure Description

[0013] Figure 1This is a flowchart of the method for predicting target trajectory according to the present invention;

[0014] Figure 2 This is the overall framework of the target trajectory prediction network. Detailed Implementation

[0015] The present invention will be further described below with reference to the accompanying drawings and specific examples.

[0016] Figure 1 This is a schematic diagram of the implementation process of the present invention, and the specific steps are shown in the attached figures 101-106.

[0017] Input the position trajectory sequence X of all targets in the same scene from a bird's-eye view / top-down view, such as vehicles, pedestrians, bicycles, etc. Figure 2 As shown in the extractor section, multi-level motion features are obtained through differential and LSTM (Long Short-Term Memory) encoding.

[0018]

[0019]

[0020] Here, Diff(·) represents the difference operation, and φ(·) represents the LSTM-based encoder. The training labels for the δ-level are calculated using the same difference method as follows:

[0021]

[0022] Training labels are used during the training process and the final result to calculate the loss and perform backpropagation gradient optimization.

[0023] like Figure 2 As shown in the motion-interactive attention diagram, this applies to the current target i and the remaining targets Ω in the current scene. i Establish a pairwise interaction graph G=, where the connections between targets in the graph are achieved through attention. Represented by multi-level trajectory encoding of the current target i. Calculate query Q δ Utilize the remaining target Ω in the scene i Multi-level trajectory coding Calculate the shared key K and the multi-level value V δ :

[0024]

[0025]

[0026]

[0027] Where FC(·) represents a fully connected network layer, enc key (·) represents the encoder, and concat(·) represents the cascading operation.

[0028] Through matrix Q δ The attention value is calculated by the dot product of K and then compared with the value V. δ Multiply to obtain the remaining target Ω i Motion-interaction effects on the current target:

[0029]

[0030]

[0031] Where d is the scaling factor.

[0032] like Figure 2 As shown in the stereo decoder, common methods are based on the following time-series decoding approach:

[0033]

[0034] in z represents the hidden variable of the decoder at time t. i This represents the trajectory encoding obtained by the encoder.

[0035] This invention further introduces spatial decoding and motion feature decoding methods as follows:

[0036]

[0037] It includes the time dimension (t-1→t) and the spatial dimension (Ω). i →i), and decoding of the motion feature dimension (δ+1→δ). This represents the trajectory code obtained by the encoder. Represents relative to the previous moment The system status, Represents relative to The high-level latent variables, with the spatial dimension representing the latent variables of the hypothetical predicted trajectories of the remaining targets. The calculation is as follows:

[0038]

[0039] Where a ij,0 Represents the motion-interaction attention extracted in step (2). This represents the hypothetical predicted trajectory of the remaining targets obtained during the decoding process.

[0040] The decoder's hidden variable sequence is then processed by an activation function to obtain the output prediction position sequence:

[0041]

[0042] The performance comparison of this invention with other trajectory prediction methods is shown in Tables 1 and 2:

[0043] The selected datasets include the NGSIM dataset, the highD dataset, the INTERACTION dataset, and the Argoverse dataset, which contain complex interactive scenarios such as highways, intersections, roundabouts, and urban roads, involving targets such as vehicles, pedestrians, and bicycles.

[0044] The evaluation metric is the root mean square error (RMSE), with a smaller value indicating better performance. The comparison methods are common prediction models in the field, including: Social LSTM: Human Trajectory Prediction in Crowded Spaces (S-LSTM), Convolutional Social Pooling for Vehicle Trajectory Prediction (CS-LSTM), Multiple Futures Prediction (MFP), Multi-Agent Tensor Fusion for Contextual Trajectory Prediction (MATF GAN), Non-local Social Pooling for Vehicle Trajectory Prediction (NLS-LSTM), Attention Based Vehicle Trajectory Prediction (MHA-LSTM), and ST-LSTM: Spatio-Temporal Graph based Long Short-Term Memory Network for Vehicle Trajectory Prediction (ST-LSTM).

[0045] Table 1. Performance comparison of the present invention with other methods on NGSIM and highD datasets.

[0046]

[0047] Table 2. Performance comparison of the present invention with other methods on the INTERACTION and Argoverse datasets.

[0048]

[0049] The technical means disclosed in this invention are not limited to those disclosed in the above embodiments, but also include technical solutions composed of any combination of the above technical features. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this invention, and these improvements and modifications are also considered within the scope of protection of this invention.

Claims

1. A target trajectory prediction method, characterized in that, Comprising the following steps: (1) Establishing a dual multi-level motion feature model training and reasoning method; using a motion extractor to perform difference and encoding on all target trajectories in the input scene to obtain multi-level motion features; During the training process, the same operation is taken to obtain multi-level training labels for the target trajectories as training labels; (2) Using the multi-level motion features extracted in (1), a paired interaction graph of the targets in the scene is established; the dot product attention value based on motion-interaction is calculated, and the trajectory encoding is calculated in combination with the multi-level motion features; (3) Using a stereo decoder, decoding is performed in the time, space and feature dimensions; the influence of different target hypothetical prediction trajectories on other targets is calculated in the space; the influence of high-level features on low-level features is calculated in the feature dimension; In step (3), in terms of feature dimension, the low-level feature units in the recurrent unit of the decoder. The computation depends on higher-level feature units. Specifically as follows: This includes decoding of the motion feature dimension (δ+1→δ). This represents the trajectory code obtained by the encoder. Represents relative to the previous moment The system status, Represents relative to High-level hidden variables; In spatial dimensions, decode the trajectory Y of a single target i at time t i,t Compute the remaining targets Ω i Hypothetical trajectory generated during the decoding process at time t-1 Motion-interaction influence for target i; spatial dimension represents a latent variable of the remaining target's hypothetical predicted trajectory is calculated as follows: where a ij,0 represents the motion-interaction attention extracted in step (2), represents the hypothetical predicted trajectory obtained for the remaining targets during the decoding process.

2. The target trajectory prediction method of claim 1, wherein, The step (1) extracts the motion features h of the delta level from the input target historical trajectory X and future trajectory Y δ and tags and establishes a parallel delta strip backpropagation channel, as follows: Wherein Diff(·) represents the difference operation, and φ(·) represents the LSTM-based encoder; at the same time, the training labels of the δ level are calculated by using the same difference method as follows: The training labels are used to calculate the loss with the final result during the training process, so as to perform gradient optimization by back propagation.

3. The target trajectory prediction method of claim 1, wherein, The point product attention calculation of the motion-interaction in the step (2) utilizes the multi-level trajectory encoding of the target i Computing the query Q δ , utilizing the multi-level trajectory encoding of the remaining targets Ω i in the scene Computing the shared key K and the multi-level value V δ , and then using matrix point product calculation to obtain the encoding amount The specific process is as follows: where FC(·) represents a fully connected network layer, enc key (·) represents an encoder, concat(·) represents a concatenation operation; The attention value is calculated by the matrix Q δ and the dot product of K, and multiplied by the value V δ to obtain the remaining target Ω i Motion-interactive influence on the current target: Wherein d is a scaling factor.

Images