Interactive perception trajectory prediction method for heterogeneous agents in shared space
By introducing a framework of Transformer encoder and cross-LSTM decoder, combined with learnable class embedding and graph attention enhanced by relative position, the problems of heterogeneity processing and low computational efficiency in existing technologies are solved, and efficient trajectory prediction of heterogeneous agents is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-15
- Publication Date
- 2026-03-24
AI Technical Summary
Existing technologies ignore the heterogeneity between agents, resulting in an inability to accurately perceive heterogeneous characteristics and predict future interactions, and also have low computational efficiency.
Employing a Transformer-based encoder-decoder framework, this approach dynamically and sparsely updates the interactions between agents through cascaded spatiotemporal interaction modeling and cross-LSTM decoders. By combining learnable class embeddings and graph attention enhanced by relative position, it enables trajectory prediction for heterogeneous agents.
It achieves accurate characterization of the behavioral patterns of heterogeneous intelligent agents and accurate prediction of future dynamic interactions, reduces the computational burden, and meets real-time requirements.
Smart Images
Figure CN121723084A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of automatic driving, in particular to an interaction perception trajectory prediction method for heterogeneous agents in a shared space. BACKGROUND
[0002] The prior art ignores the heterogeneity between agents, and most methods either ignore the heterogeneity of traffic participants in the scene or remove heterogeneous traffic participants that are not the focus of the study, thereby modeling all traffic participants in the scene as homogeneous. The former ignores the different kinematic differences brought by heterogeneity, and the latter destroys the interaction relationship contained in the scene, without strategically using shared features such as GANs, GNNs and Transformers. Most of these methods assume homogeneous agents.
[0003] Most of the prior art only models interactions in the encoder based on historical data, and rarely obtains evolving dependencies in the decoder, such as Social GAN, STGCNN, Multiclass-SGCN, etc.
[0004] Some methods in the prior art model interactions frame by frame, such as Social LSTM, PV-Collision, etc., ignoring the fact that interactions in real-world scenarios are usually sparse and exhibit temporal persistence.
[0005] Therefore, how to establish a trajectory prediction method that can accurately perceive heterogeneity, predict future interactions and has high computational efficiency has become a technical problem that technicians in the field urgently need to solve. SUMMARY
[0006] Therefore, the purpose of the present application is to provide an interaction perception trajectory prediction method for heterogeneous agents in a shared space, which can accurately perceive heterogeneity, predict future interactions and has high computational efficiency.
[0007] The technical problems solved by the present application adopt the following technical solutions:
[0008] An interaction perception trajectory prediction method for heterogeneous agents in a shared space comprises the following steps:
[0009] Step S1, encoder-historical trajectory and interaction feature extraction, applying a Transformer-based spatial modeling at each time step, processing historical trajectory data, and extracting and fusing the motion features of each agent and the spatial interaction features of the historical period;
[0010] Step S2, Decoder - Future Trajectory Iterative Prediction and Interaction Modeling: Based on the encoder output state, the future trajectory is iteratively predicted, and the interaction between agents is dynamically and sparsely updated during the prediction process to simulate the evolution of future interactions.
[0011] Furthermore, in step S1, the specific methods for encoder-historical trajectory and interaction feature extraction include:
[0012] Step S11: Input preprocessing and motion feature calculation
[0013] The input is the history of all agents in event T. h coordinate sequence Furthermore, each agent i has its kinematic characteristics defined at time step t as shown in formula (1):
[0014]
[0015] in, Indicates displacement. This moment Compared to the previous moment The heading angle of the position;
[0016] Step S12, Spatial embedding and category embedding
[0017] The original information from the previous step By mapping to a high-dimensional feature space through two layers of MLP, the basic spatial embedding is obtained. At the same time, the corresponding class embedding E(C(i)) is looked up from a learnable embedding table according to the class C(i) of the agent. The basic spatial embedding and the class embedding vector are added together to obtain the final fused feature. As shown in formula (2):
[0018]
[0019] in, W represents the spatial embedding at time step t. s Let E(C(i)) represent the weights of the MLP, where E(C(i)) is the learnable class embedding indexed by the agent class C(i).
[0020] Step S13, Cascaded Spatiotemporal Interaction Modeling
[0021] By aggregating the interaction features of all time steps throughout the entire observation period, spatiotemporal interaction information is formed throughout the entire history.
[0022] Furthermore, in step S13, the specific methods for cascaded spatiotemporal interaction modeling include:
[0023] Step S131, Spatial Interaction Modeling
[0024] At each historical time step t, the software fuses the features of all agents in the scene. Stacked into matrix Z t This is input into a spatial Transformer module; this spatial Transformer module uses a self-attention mechanism to compute the interactions between agents; a spatial mask M is introduced. t It ensures that interactions are only considered between agents within a certain distance D. If the distance between agents i and j is less than the threshold D, attention is calculated normally; if the distance exceeds D, the attention weight is set to -∞ after Softmax, i.e., attention between agents exceeding the threshold D is blocked, focusing on local effective interactions, and the output is the feature enhanced by the interaction. As shown in formula (3):
[0025]
[0026] Where F(*) represents the update function of the Transformer, and Q... t For querying the matrix, K t The key matrix, For K t The transpose of V t M is a value matrix. t ∈R N×N It is a spatial mask, each element This can be expressed by formula (4):
[0027]
[0028] in, D represents the distance between agent i and agent j at time step t, and D represents the threshold.
[0029] Step S132, Time-dependent aggregation
[0030] For each agent i, interact with the enhanced feature sequences at all historical time steps. Input an LSTM network and extract the hidden state at the last time step of the LSTM. The hidden states of all agents represent the historical behavior of the agent. The initial hidden state matrix H0 is formed, which contains the spatiotemporal interaction information of each agent throughout the history. It is used as the initial state of the decoder, as shown in Equation (5):
[0031]
[0032] Among them, W e Indicates the weights of the LSTM. This represents the interactive enhanced spatial embedding sequence of all historical time steps.
[0033] Furthermore, in step S2, the specific methods for decoder-future trajectory iterative prediction and interactive modeling include:
[0034] Step S21, decoder initialization
[0035] Set the initial hidden state matrix H0 output by the encoder as the initial state of the decoder LSTM;
[0036] Step S22, based on the dynamic interaction across the entire prediction range explained by cross-LSTM;
[0037] Step S23, Output and Model Training
[0038] Repeat step S22 until all T are completed. f Each prediction step outputs the predicted trajectories of all agents; during the training phase, a negative log-likelihood loss function is used to ensure that the Gaussian distribution predicted by the model can cover the true trajectory with a high probability; for heterogeneous datasets, different weights (α1, α2) are assigned to different classes in the total loss to balance learning.
[0039] The model is optimized by using the backpropagation algorithm to optimize all parameters in the model, so that the model can make accurate, safe and physically consistent predictions.
[0040] Furthermore, in step S22, the specific methods for interpreting dynamic interactions across the entire prediction range using LSTM include:
[0041] Step S221, predict the location distribution
[0042] For each agent i, use its current LSTM hidden state Predict the parameters of a bivariate Gaussian distribution using an MLP;
[0043]
[0044] in, For the parameters of a bivariate Gaussian distribution, It is the predicted mean. It is the standard deviation. The uncertainty in the prediction of the x-coordinate The uncertainty in the prediction of the y-coordinate It is the correlation coefficient between the motion along the x-axis and y-axis;
[0045] Step S222, Update spatial embedding
[0046] Use the predicted location Generate the spatial embedding of the current prediction step using an MLP. Add its category embedding again, as shown in formula (7):
[0047]
[0048] in, W' is the mean of the predicted positions in formula (6). s Let E(C(i)) be the learnable parameters of the multilayer perceptron, and let E(C(i)) be the learnable category embedding vector corresponding to the category C(i) to which agent i belongs.
[0049] To account for constantly changing spatial relationships, the interaction module processes the current hidden state and predicted location to generate interaction features R. t As shown in formula (8):
[0050]
[0051] Where N is the number of agents, H t Let be the set or matrix of hidden states of all agents at time step t;
[0052] Step S223, Heterogeneous Interaction Modeling
[0053] The software checks whether the current time step t is an integer multiple of the preset interactive update frequency K.
[0054] If so, activate the interactive module;
[0055] If not, skip this step, use the interaction features or zero vector from the previous step, and proceed to the next time step t+1.
[0056] Step S224, Gating status update
[0057] If interaction modeling was performed in this step, the resulting interaction features will be... With LSTM hidden state Fusion is performed through a gated fusion unit; this gated mechanism learns a weight. Used to balance the influence of its own state and external interactions, outputting the updated hidden state. If no interaction is performed, then As shown in formula (13):
[0058]
[0059] Where σ(·) is the Sigmoid activation function, W r and Used to generate gating signals Will and Projected onto the gated space Used to update the original hidden state when updating the hidden state. Perform the transformation.
[0060] Step S225, LSTM state update
[0061] Updated hidden state and the spatial embedding of the current step The hidden state is fed into the LSTM unit together to calculate the hidden state at the next time step. As shown in formula (14):
[0062]
[0063] Among them, W d This is the complete set of learnable parameters for the LSTM unit in the decoder.
[0064] Furthermore, in step S223, the interactive module is activated; including:
[0065] (1) Construct a dynamic graph with all agents as nodes and their current LSTM hidden states. Node features; based on the latest predicted location Establish edges for node pairs whose distance is within the threshold D;
[0066] (2) For graph attention enhanced by relative position, compute the relative position vector d of each node i and its neighbor j. ij And encoded into edge embeddings e via MLP ij As shown in formula (9):
[0067] e ij =MLP(d ij W edge (9)
[0068] in, W is the relative position vector between nodes i and j. edge It is a learnable weight matrix used for edge embedding;
[0069] When calculating attention, the states of neighboring nodes are considered. With edge embedding e ij The sums together generate a key and a value, which combine the neighbor's own state and its relationship with its target, as shown in formula (10):
[0070]
[0071] Where, q i For the query vector, k ij Let v be the key vector. ij Let W' be a value vector. Q ,W'K and W' V All are learnable parameters;
[0072] Interaction characteristics r of agent i i The calculation is as follows:
[0073]
[0074] Where d represents the element dimension of the query;
[0075] Through weighted aggregation, the interaction feature r is obtained. i Finally, the interaction features of all agents are collected into the interaction matrix R, as shown in formula (12):
[0076]
[0077] Furthermore, in step S23, all parameters in the model are optimized, including MLP weights, LSTM weights, class embedding table, and attention weights.
[0078] The present invention discloses a method for predicting the interactive sensing trajectory of heterogeneous intelligent agents in a shared space, which has the following beneficial effects:
[0079] This invention constitutes a complete, efficient software solution specifically for predicting heterogeneous interaction trajectories in shared spaces, systematically solving the three core problems of existing technologies in terms of heterogeneity handling, dynamic interaction capture, and computational efficiency:
[0080] 1. Achieve accurate characterization of the behavioral patterns of heterogeneous intelligent agents: By introducing new representation learning methods, the model can fundamentally understand and distinguish the unique motion intentions of different intelligent agents such as pedestrians and vehicles.
[0081] 2. Achieve accurate prediction of future dynamic interactions: By introducing a decoder design, the model can not only review the past, but also perceive and respond to changes in future interaction relationships between agents in real time.
[0082] 3. Construct a high-performance and high-efficiency prediction framework: By optimizing the calculation strategy, while ensuring improved prediction accuracy, the computational burden of the model is significantly reduced, so that it meets the real-time requirements. Attached Figure Description
[0083] Figure 1 This is a flowchart of the method of the present invention.
[0084] Figure 2 This is an encoder-decoder framework diagram.
[0085] Figure 3 This is a schematic diagram of cross-LSTM interactive modeling.
[0086] Figure 4 This is the decoding flowchart.
[0087] Figure 5 This is a schematic diagram of the interaction modeling process.
[0088] Figure 6 It is a box plot showing the velocity and acceleration distributions of the three main categories.
[0089] Figure 7 It is a scene proportion diagram of different combinations of intelligent agent categories in the dataset.
[0090] Figure 8 This is a qualitative comparison chart of trajectory prediction between the method proposed in this invention and MART. Detailed Implementation
[0091] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0092] This invention introduces learnable category embeddings, which are more flexible and enable the model to dynamically learn and distinguish the behavioral features of different categories of agents. Utilizing a prior kinematic feature analysis method, combined with a specific scenario, vehicles and cyclists with similar movement patterns are grouped into a single category. This invention proposes an encoder-decoder interaction modeling framework, where the encoder acquires historical information and the decoder predicts future trajectories, while simultaneously using cross-LSTM to interpret dynamic interactions across the entire prediction range. This invention introduces the concept of interaction persistence; compared to dense interaction methods, sparse but persistent interactions can simultaneously improve prediction accuracy and computational efficiency.
[0093] The technical solution of this invention is a method for predicting the interactive perception trajectory of heterogeneous intelligent agents in a shared space. This method employs an encoder-decoder framework, such as... Figure 2 As shown, the encoder performs cascaded spatiotemporal interaction modeling to extract historical features, while the decoder explicitly models the interaction and utilizes the persistence of the interaction to reduce the frequency of interaction, thereby improving accuracy and efficiency.
[0094] This method introduces learnable category embeddings, enabling the model to dynamically learn and distinguish the behavioral features of different categories of agents. It employs a category-aware representation learning method to classify vehicles and cyclists with similar motion patterns into a single category and applies relative position-enhanced attention to extract spatiotemporal interaction features, significantly improving computational efficiency.
[0095] like Figure 1 As shown, this invention provides a method for predicting the interactive sensing trajectory of heterogeneous intelligent agents in a shared space, comprising the following steps:
[0096] Step S1, Encoder - Historical Trajectory and Interaction Feature Extraction: At each time step, apply Transformer-based spatial modeling, process historical trajectory data, extract and fuse the motion features of each agent and the spatial interaction features of historical time periods.
[0097] The specific methods for encoder-historical trajectory and interaction feature extraction include:
[0098] Step S11: Input preprocessing and motion feature calculation
[0099] The input is the history of all agents in event T. h coordinate sequence Furthermore, each agent i has its kinematic characteristics defined at time step t as shown in formula (1):
[0100]
[0101] in, Indicates displacement. This moment Compared to the previous moment The heading angle of the position;
[0102] Function: It transforms the original absolute coordinates into kinematic features that include velocity and direction, providing a motion description for the model and enhancing the model's ability to capture motion dynamics in time and space.
[0103] Step S12, Spatial embedding and category embedding
[0104] The original information from the previous step By mapping to a high-dimensional feature space through two layers of MLP, the basic spatial embedding is obtained. At the same time, the corresponding class embedding E(C(i)) is looked up from a learnable embedding table according to the class C(i) of the agent. The basic spatial embedding and the class embedding vector are added together to obtain the final fused feature. As shown in formula (2):
[0105]
[0106] in, W represents the spatial embedding at time step t. s Let E(C(i)) represent the weights of the MLP, where E(C(i)) is the learnable class embedding indexed by the agent class C(i).
[0107] Function: To map motion features to a high-dimensional space and incorporate the agent's category information.
[0108] Step S13, Cascaded Spatiotemporal Interaction Modeling
[0109] By aggregating the interaction features of all time steps throughout the entire observation period, spatiotemporal interaction information is formed throughout the entire history.
[0110] In step S13, the specific methods for cascaded spatiotemporal interaction modeling include:
[0111] Step S131, Spatial Interaction Modeling
[0112] like Figure 4 As shown, at each historical time step t, the software fuses the features of all agents in the scene. Stacked into matrix Z t The input is then fed into a spatial Transformer module; this module uses a self-attention mechanism to compute the interactions between agents; to improve efficiency, a spatial mask M is introduced. t It ensures that interactions are only considered between agents within a certain distance D. If the distance between agents i and j is less than the threshold D, attention is calculated normally; if the distance exceeds D, the attention weight is set to -∞ after Softmax, i.e., attention between agents exceeding the threshold D is blocked, focusing on local effective interactions, and the output is the feature enhanced by the interaction. As shown in formula (3): This step reflects the spatial sparsity of the interaction and reduces the amount of computation.
[0113]
[0114] Where F(*) represents the update function of the Transformer, and Q... t For querying the matrix, K t The key matrix, For K t The transpose of V t M is a value matrix. t ∈R N×N It is a spatial mask, each element This can be expressed by formula (4):
[0115]
[0116] in, D represents the distance between agent i and agent j at time step t, and D represents the threshold.
[0117] Function: At each time step t, model the interactions between all agents.
[0118] Step S132, Time-dependent aggregation
[0119] For each agent i, interact with the enhanced feature sequences at all historical time steps. Input an LSTM network and extract the hidden state at the last time step of the LSTM. The hidden states of all agents represent the historical behavior of the agent. The initial hidden state matrix H0 is formed, which contains the spatiotemporal interaction information of each agent throughout the history. It is used as the initial state of the decoder, as shown in Equation (5):
[0120]
[0121] Among them, W e Indicates the weights of the LSTM. This represents the interactive enhanced spatial embedding sequence of all historical time steps.
[0122] Function: To aggregate the interaction features of all time steps within the entire observation period to form an overall historical state representation.
[0123] Step S2, Decoder - Future Trajectory Iterative Prediction and Interaction Modeling: Based on the encoder output state, future trajectories are iteratively predicted, and the interactions between agents are dynamically and sparsely updated during the prediction process to simulate the evolution of future interactions, such as... Figure 3 As shown, LSTM updates the hidden state for prediction and merges spatial interactions between cells.
[0124] Step S2, the specific methods for decoder-future trajectory iterative prediction and interactive modeling include:
[0125] Step S21, decoder initialization
[0126] Set the initial hidden state matrix H0 output by the encoder as the initial state of the decoder LSTM;
[0127] Function: To provide an initial state containing historical interaction information for predicting loops.
[0128] Step S22, interpret the dynamic interactions across the entire prediction range based on the cross-LSTM; this loop is executed for each future time step t until the prediction ends.
[0129] Specific methods for interpreting dynamic interactions across the entire prediction range using LSTM include:
[0130] Step S221, predict the location distribution
[0131] For each agent i, use its current LSTM hidden state Predict the parameters of a bivariate Gaussian distribution using an MLP;
[0132]
[0133] in, For the parameters of a bivariate Gaussian distribution, It is the predicted mean. It is the standard deviation. The uncertainty in the prediction of the x-coordinate The uncertainty in the prediction of the y-coordinate It is the correlation coefficient between the motion along the x-axis and y-axis;
[0134] Function: At each step of decoding, predict the probability distribution of the future position of the agent.
[0135] Step S222, Update spatial embedding
[0136] Use the predicted location Generate the spatial embedding of the current prediction step using an MLP. Add its category embedding again, as shown in formula (7):
[0137]
[0138] in, W' is the mean of the predicted positions in formula (6). s Let E(C(i)) be the learnable parameters of the multilayer perceptron, and let E(C(i)) be the learnable category embedding vector corresponding to the category C(i) to which agent i belongs.
[0139] Function: Update the agent's spatial embedding based on the predicted location to reflect its new state.
[0140] To account for constantly changing spatial relationships, the interaction module processes the current hidden state and predicted location to generate interaction features R. t As shown in formula (8):
[0141]
[0142] Where N is the number of agents, H t Let be the set or matrix of hidden states of all agents at time step t;
[0143] Step S223, Heterogeneous Interaction Modeling
[0144] The detailed flowchart is as follows: Figure 4 As shown. Since interactions between agents do not occur in every frame, the software checks whether the current time step t is an integer multiple of the preset interaction update frequency K;
[0145] If so, then activate the interactive module; such as Figure 5 As shown, the hidden states output by the LSTM are used as nodes in the graph, and the predicted positions of these nodes form relative information e. ij Attention mechanisms facilitate message passing between graph nodes to enhance interactive representations. These include:
[0146] (1) Construct a dynamic graph with all agents as nodes and their current LSTM hidden states. Node features; based on the latest predicted location Establish edges for node pairs whose distance is within the threshold D;
[0147] (2) For graph attention enhanced by relative position, compute the relative position vector d of each node i and its neighbor j. ij And encoded into edge embeddings e via MLP ij As shown in formula (9):
[0148] e ij =MLP(d ij W edge (9)
[0149] in, W is the relative position vector between nodes i and j. edge It is a learnable weight matrix used for edge embedding;
[0150] The purpose of this step is to encode the relative spatial relationship between the two agents into a vector that a computer can understand.
[0151] When calculating attention, the states of neighboring nodes are considered. With edge embedding e ij The sums together generate a key and a value, which combine the neighbor's own state and its relationship with its target, as shown in formula (10):
[0152]
[0153] Where, q i For the query vector, k ij Let v be the key vector. ij Let W' be a value vector. Q ,W' K and W' V All of these are learnable parameters; this design allows the target node to focus on its relative position with neighboring nodes.
[0154] Interaction characteristics r of agent i i The calculation is as follows:
[0155]
[0156] Where d represents the element dimension of the query.
[0157] Through weighted aggregation, the interaction feature r is obtained. i Finally, the interaction features of all agents are collected into the interaction matrix R, as shown in formula (12):
[0158]
[0159] If not, skip this step, use the interaction features or zero vector from the previous step, and proceed to the next time step t+1.
[0160] Function: During the decoding process, based on the future states predicted by the agents, continuously model the dynamic evolution of their interactions.
[0161] Step S224, Gating status update
[0162] If interaction modeling was performed in this step, the resulting interaction features will be... With LSTM hidden state Fusion is performed through a gated fusion unit; this gated mechanism learns a weight. Used to balance the influence of its own state and external interactions, outputting the updated hidden state. If no interaction is performed, then As shown in formula (13):
[0163]
[0164] Where σ(·) is the Sigmoid activation function, W r and Used to generate gating signals Will and Projected onto the gated space Used to update the original hidden state when updating the hidden state. Perform the transformation;
[0165] Function: To smoothly fuse the hidden state and external interaction features of LSTM.
[0166] Step S225, LSTM state update
[0167] Updated hidden state and the spatial embedding of the current step The hidden state is fed into the LSTM unit together to calculate the hidden state at the next time step. As shown in formula (14):
[0168]
[0169] Among them, W d This is the complete set of learnable parameters for the LSTM unit in the decoder.
[0170] Function: To pass the state that incorporates the latest interaction information to the future, and to embed the current spatial information into the LSTM network to update the state for the next time step.
[0171] Step S23, Output and Model Training
[0172] Repeat step S22 until all T are completed. f Each prediction step outputs the predicted trajectories of all agents; during the training phase, a negative log-likelihood loss function is used to ensure that the Gaussian distribution predicted by the model can cover the real trajectory with a high probability; for heterogeneous datasets, different weights (α1, α2) are assigned to different categories (such as pedestrians, vehicles, and cyclists) in the total loss to balance the learning.
[0173] Function: The output results generate the final trajectory prediction;
[0174] The model is optimized using the backpropagation algorithm, refining all parameters to ensure accurate, safe, and physically consistent predictions. These optimized parameters include MLP weights, LSTM weights, class embedding tables, and attention weights.
[0175] The system structure of the present invention is as follows: Figure 1 As shown, its core is a neural network model that integrates a cascaded spatiotemporal interactive encoder and a cross-LSTM interactive decoder.
[0176] The encoder is used to extract spatiotemporal features and interaction relationships from historical trajectories; the decoder dynamically models future interactions during the prediction process, introduces the assumption of interaction persistence, and reduces redundant calculations.
[0177] Furthermore, as shown in Table 6, the ablation experiments reveal that the interaction modeling module and the category embedding module in both the encoder and decoder are indispensable components of the system, and removing any of these modules will lead to a significant decrease in prediction performance.
[0178] Table 6. Evaluation of the impact of each core module on predictive performance in ablation studies.
[0179]
[0180] The metrics in Table 6, ADE / FDE, are better the lower they are. Bold values indicate the best performance.
[0181] The system operates through an end-to-end forward reasoning process, as detailed below:
[0182] (1) Historical information coding
[0183] The encoder analyzes historical trajectories, extracts the motion features, category information, and interaction relationships of each agent, and generates the initial state H0.
[0184] (2) Future trajectory prediction
[0185] The decoder starts from H0 and predicts future positions frame by frame.
[0186] Each frame first predicts the position based on the current state;
[0187] If the current frame is an interaction update step, then an interaction graph is constructed, the interaction features enhanced by relative position are calculated, and the state is updated by gating. Table 7 proves the effectiveness of the gating update mechanism.
[0188] Table 7. Impact of gating update operations on the decoder
[0189]
[0190] The indicator in Table 7, ADE / FDE, is better the lower it is. Bold values indicate the best values.
[0191] If it is not an interactive update step, then the state of the previous frame is directly used to continue prediction;
[0192] Update the LSTM state to the next frame.
[0193] (3) Trajectory generation
[0194] Repeat the above process until a complete future trajectory is generated.
[0195] The technical effects and advantages of this invention are as follows:
[0196] 1) High prediction accuracy
[0197] ① As shown in Table 1, on the HBS dataset, ADE is 0.435m and FDE is 0.994m, which is better than all baseline models (such as MART, Social GAN, etc.). ADE is improved by about 13.0% and FDE by about 10.1%.
[0198] Table 1 shows the prediction performance on the HBS dataset.
[0199]
[0200] The indicator in Table 1, ADE / FDE (meters), is better the lower it is. Bold / underlined values indicate best / second-best.
[0201] ② As shown in Table 3, on the GCS dataset, ADE is 15.4 pixels and FDE is 30.7 pixels, which also reach the optimal level, proving its superior generalization ability.
[0202] Table 3 shows the prediction performance on the GCS dataset.
[0203]
[0204] The parameters in Table 3: ADE / FDE (pixels), the lower the better. Bold / underlined values indicate best / second best.
[0205] 2) High security
[0206] ① As shown in Table 2, the collision rate COL on HBS is only 0.42%, which is significantly lower than other methods.
[0207] Table 2 shows the collision rate (COL) results on the HBS dataset.
[0208]
[0209] In Table 2, the indicator COL (%) is lower, the better. Bold values indicate the best values.
[0210] ② The interactive modeling capability in the decoder enables the system to anticipate future conflicts and generate avoidance trajectories in advance, such as... Figure 7 The qualitative results are expressed.
[0211] 3) High computational efficiency
[0212] As shown in Table 5, the optimal configuration (F4) achieves a 66.7% improvement in inference speed compared to frame-by-frame interaction (F1) through sparse interaction modeling. For the scheme that performs interaction modeling every three time steps (F3), it achieves the best performance on the HBS dataset, with ADE and FDE improvements of 2.9% and 4.1% respectively, and an inference speed improvement of 29.6% compared to frame-by-frame interaction (F1). On the denser GCS dataset, the optimal configuration (F4) performs even better, with ADE and FDE improvements of 0.8% and 1.0% respectively, and an inference speed improvement of 66.7%.
[0213] Table 5. Impact of reduced interactive modeling frequency across LSTM encoders on prediction performance and efficiency.
[0214]
[0215] The bolded values in Table 5 represent the best values.
[0216] As shown in Table 4, the model has only 312.9K parameters, which is much lower than MART (874.1K), making it more suitable for edge deployment.
[0217] Table 4 Comparison of Model Parameter Sizes
[0218]
[0219] As shown in Table 8, the best overall performance can be achieved by assigning appropriate loss weights to different categories (pedestrians and vehicles / cyclists), which reflects the model's ability to learn in a balanced manner from heterogeneous data during the training phase.
[0220] Table 8 shows the impact of α1 and α2 on prediction performance on the HBS dataset.
[0221]
[0222] Table 8 shows the impact of α1 and α2 on prediction performance on the HBS dataset: ADE / FDE (meters), with lower values being better. Bold / underlined values indicate best / second best.
[0223] 4) Smooth and natural trajectory
[0224] Non-frame-by-frame interactive modeling avoids trajectory jitter and generates smoother trajectories that better conform to human behavior patterns, such as... Figure 8 As shown, Figure 8 (a) is a complex interaction scenario involving pedestrians, vehicles, and cyclists. Figure 8 (b) For scenarios involving pedestrian groups, nonlinear trajectories, and stationary agents, the model can accurately predict complex behaviors such as nonlinear motion and stationary states, and the generated trajectories are more consistent with real-world interaction logic and human behavior patterns.
[0225] Application scenario of this invention: an autonomous vehicle approaches an intersection without traffic lights, a pedestrian is crossing, and an oncoming bicycle is turning left.
[0226] The encoder identifies the vehicle, pedestrians, bicycles, and their historical trajectories;
[0227] The decoder injects the vehicle's intention to slow down and give way during the first interactive update;
[0228] Subsequent non-interactive frames continue this intent, and during the second interactive update, the risk is assessed to be eliminated, generating a slow-moving trajectory.
[0229] Effect: Generates a safe trajectory that decelerates before proceeding, with no collision prediction.
[0230] Compared with existing technologies:
[0231] 1. Learnable category embeddings: Learnable category embeddings are more flexible, such as... Figure 6 and Figure 7 As shown, the model dynamically learns feature representations of heterogeneous agents and performs strategic grouping based on data-driven approaches (combining vehicles and riders). Figure 6 (a) shows the speed distribution of pedestrians, vehicles, and cyclists. Figure 6 (b) represents the acceleration distribution of pedestrians, vehicles and cyclists. Figure 7The proportion of scenarios with different combinations of agent categories in the dataset illustrates... Figure 7 (a) Training set and Figure 7 (b) The percentage of scenarios in the test set that contain one, two, or three agent classes.
[0232] 2. Encoder Cascade Spatiotemporal Modeling: A cascaded structure of Transformer (spatial) and LSTM (temporal) is adopted to extract historical spatiotemporal interaction features more effectively.
[0233] 3. Decoder across LSTM structure: An independent interaction modeling module is introduced into the decoder, which is carried out in conjunction with the LSTM time-series update, realizing explicit modeling of future dynamic interactions.
[0234] 4. Graph attention enhanced by relative position: In the interaction model, relative position information is encoded and incorporated into the attention mechanism, making the interaction computation more sensitive to spatial relationships.
[0235] 5. Sparse interactive update mechanism: The interaction persistence hypothesis is proposed and verified. Interactive calculations are performed in the decoder at a frequency lower than the frame rate, which improves prediction accuracy and efficiency.
[0236] 6. Gating fusion mechanism: Gating units are used in the decoder to balance its own state and the impact of interactions, making state updates more reliable.
[0237] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A method for predicting the interactive sensing trajectory of heterogeneous intelligent agents in a shared space, characterized in that, Includes the following steps: Step S1, Encoder - Historical Trajectory and Interaction Feature Extraction: At each time step, apply Transformer-based spatial modeling, process historical trajectory data, extract and fuse the motion features of each agent and the spatial interaction features of historical time periods. Step S2, Decoder - Future Trajectory Iterative Prediction and Interaction Modeling: Based on the encoder output state, the future trajectory is iteratively predicted, and the interaction between agents is dynamically and sparsely updated during the prediction process to simulate the evolution of future interactions.
2. The method for predicting the interactive sensing trajectory of heterogeneous intelligent agents in a shared space according to claim 1, characterized in that, In step S1, the specific methods for encoder-historical trajectory and interaction feature extraction include: Step S11: Input preprocessing and motion feature calculation The input is the history of all agents in event T. h coordinate sequence Furthermore, each agent i has its kinematic characteristics defined at time step t as shown in formula (1): in, Indicates displacement. This moment Compared to the previous moment The heading angle of the position; Step S12, Spatial embedding and category embedding The original information from the previous step By mapping to a high-dimensional feature space through two layers of MLP, the basic spatial embedding is obtained. At the same time, the corresponding class embedding E(C(i)) is looked up from a learnable embedding table according to the class C(i) of the agent. The basic spatial embedding and the class embedding vector are added together to obtain the final fused feature. As shown in formula (2): in, W represents the spatial embedding at time step t. s Let E(C(i)) represent the weights of the MLP, and E(C(i)) be the learnable class embedding indexed by the agent class C(i) of the agent. Step S13, Cascaded Spatiotemporal Interaction Modeling By aggregating the interaction features of all time steps throughout the entire observation period, spatiotemporal interaction information is formed throughout the entire history.
3. The method for predicting the interactive sensing trajectory of heterogeneous intelligent agents in a shared space according to claim 2, characterized in that, In step S13, the specific methods for cascaded spatiotemporal interaction modeling include: Step S131, Spatial Interaction Modeling At each historical time step t, the software fuses the features of all agents in the scene. Stacked into matrix Z t This is input into a spatial Transformer module; this spatial Transformer module uses a self-attention mechanism to compute the interactions between agents; a spatial mask M is introduced. t It ensures that interactions are only considered between agents within a certain distance D. If the distance between agents i and j is less than the threshold D, attention is calculated normally; if the distance exceeds D, the attention weight is set to -∞ after Softmax, i.e., attention between agents exceeding the threshold D is blocked, focusing on local effective interactions, and the output is the feature enhanced by the interaction. As shown in formula (3): Where F(*) represents the update function of the Transformer, and Q... t For querying the matrix, K t The key matrix, For K t The transpose of V t M is a value matrix. t ∈R N×N It is a spatial mask, each element This can be expressed by formula (4): in, D represents the distance between agent i and agent j at time step t, and D represents the threshold. Step S132, Time-dependent aggregation For each agent i, interact with the enhanced feature sequences at all historical time steps. Input an LSTM network and extract the hidden state at the last time step of the LSTM. The hidden states of all agents represent the historical behavior of the agent. The initial hidden state matrix H0 is formed, which contains the spatiotemporal interaction information of each agent throughout the history. It is used as the initial state of the decoder, as shown in Equation (5): Among them, W e Indicates the weights of the LSTM. This represents the interactive enhanced spatial embedding sequence of all historical time steps.
4. The method for predicting the interactive sensing trajectory of heterogeneous intelligent agents in a shared space according to claim 3, characterized in that, Step S2, the specific methods for decoder-future trajectory iterative prediction and interactive modeling include: Step S21, decoder initialization Set the initial hidden state matrix H0 output by the encoder as the initial state of the decoder LSTM; Step S22, based on the dynamic interaction across the entire prediction range explained by cross-LSTM; Step S23, Output and Model Training Repeat step S22 until all T are completed. f Each prediction step outputs the predicted trajectories of all agents; during the training phase, a negative log-likelihood loss function is used to ensure that the Gaussian distribution predicted by the model can cover the true trajectory with a high probability; for heterogeneous datasets, different weights (α1, α2) are assigned to different classes in the total loss to balance learning. The model is optimized by using the backpropagation algorithm to optimize all parameters in the model, so that the model can make accurate, safe and physically consistent predictions.
5. The method for predicting the interactive sensing trajectory of heterogeneous intelligent agents in a shared space according to claim 4, characterized in that, In step S22, the specific methods for interpreting dynamic interactions across the entire prediction range using LSTM include: Step S221, predict the location distribution For each agent i, use its current LSTM hidden state Predict the parameters of a bivariate Gaussian distribution using an MLP; in, For the parameters of a bivariate Gaussian distribution, It is the predicted mean. It is the standard deviation. The uncertainty in the prediction of the x-coordinate The uncertainty in the prediction of the y-coordinate It is the correlation coefficient between the motion along the x-axis and y-axis; Step S222, Update spatial embedding Use the predicted location Generate the spatial embedding of the current prediction step using an MLP. Add its category embedding again, as shown in formula (7): in, W' is the mean of the predicted positions in formula (6). s Let E(C(i)) be the learnable parameters of the multilayer perceptron, and let E(C(i)) be the learnable category embedding vector corresponding to the category C(i) to which agent i belongs. To account for constantly changing spatial relationships, the interaction module processes the current hidden state and predicted location to generate interaction features R. t As shown in formula (8): Where N is the number of agents, H t Let be the set or matrix of hidden states of all agents at time step t; Step S223, Heterogeneous Interaction Modeling The software checks whether the current time step t is an integer multiple of the preset interactive update frequency K. If so, activate the interactive module; If not, skip this step, use the interaction features or zero vector from the previous step, and proceed to the next time step t+1. Step S224, Gating status update If interaction modeling was performed in this step, the resulting interaction features will be... With LSTM hidden state Fusion is performed through a gated fusion unit; this gated mechanism learns a weight. Used to balance the influence of its own state and external interactions, outputting the updated hidden state. If no interaction is performed, then As shown in formula (13): Where σ(·) is the Sigmoid activation function, W r and Used to generate gating signals Will and Projected onto the gated space Used to update the original hidden state when updating the hidden state. Perform the transformation; Step S225, LSTM state update Updated hidden state Spatial embedding of the current step The hidden state is fed into the LSTM unit together to calculate the hidden state at the next time step. As shown in formula (14): Among them, W d This is the complete set of learnable parameters for the LSTM unit in the decoder.
6. The method for predicting the interactive sensing trajectory of heterogeneous intelligent agents in a shared space according to claim 5, characterized in that, In step S223, the interactive module is activated; this includes: (1) Construct a dynamic graph with all agents as nodes and their current LSTM hidden states. Node features; based on the latest predicted location Establish edges for node pairs whose distance is within the threshold D; (2) For graph attention enhanced by relative position, compute the relative position vector d of each node i and its neighbor j. ij And encoded into edge embeddings e via MLP ij As shown in formula (9): e ij =MLP(d ij ;IN edge ) (9) in, W is the relative position vector between nodes i and j. edge It is a learnable weight matrix used for edge embedding; When calculating attention, the states of neighboring nodes are considered. With edge embedding e ij The sums together generate a key and a value, which combine the neighbor's own state and its relationship with its target, as shown in formula (10): q i =h i W' Q , k ij =(h j +e ij )W' K , (10) v ij =(h j +e ij )W ' V , Where, q i For the query vector, k ij Let v be the key vector. ij Let W' be a value vector. Q ,W' K and W' V All are learnable parameters; Interaction characteristics r of agent i i The calculation is as follows: Where d represents the element dimension of the query; Through weighted aggregation, the interaction feature r is obtained. i Finally, the interaction features of all agents are collected into the interaction matrix R, as shown in formula (12):
7. The method for predicting the interactive sensing trajectory of heterogeneous intelligent agents in a shared space according to claim 6, characterized in that, In step S23, all parameters in the model are optimized, including MLP weights, LSTM weights, class embedding table, and attention weights.