Heterogeneous Graph Neural Network for Autonomous Motion Forecasting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing autonomous systems face challenges in accurately and efficiently predicting the motion of multiple agents in crowded environments, as current techniques often compromise accuracy for computational efficiency or require extensive training data to generalize across various scenarios.
Innovation Solution
A method involving the creation of a heterogeneous graph using map data and historical trajectories of agents and map elements, which is then modeled by a graph neural network to generate agent goal locations, allowing for accurate and timely motion forecasting invariant to the viewpoint of the autonomous system.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If each agent independently processes the scene to build its own understanding, then each agent can act with its own understanding, but accuracy is compromised for computational needs especially in situations with a large number of agents
Solution Approach 1:
The patent segments the scene processing by creating a shared viewpoint encoding that is independent of any single agent's perspective. This encoding is then reused by multiple agents, dividing the computational workload while maintaining accurate scene understanding across all agents.
Solution Approach 2:
The shared viewpoint encoding serves multiple agents simultaneously, making it a universal representation that can be used by any agent in the scene. This multi-functional encoding structure allows the same processed scene understanding to benefit all agents without requiring separate independent processing for each.
2Device complexity
If a single viewpoint is used for predictions of each agent in the scene, then processing is simplified, but predictions are not invariant to the single viewpoint and require far more training data to generalize to rarely seen or novel autonomous system poses
Solution Approach 1:
Instead of encoding the scene from each agent's individual viewpoint, the patent inverts the approach by encoding the scene from a shared viewpoint that is independent of any single agent. This inversion allows the encoding to be reused by all agents while maintaining invariance to specific viewpoint transformations.
Solution Approach 2:
The patent changes the parameter space by using relative positional encodings and transformations that are invariant to the autonomous system's pose. By transforming the encoding parameters to be viewpoint-independent, the system achieves better generalization without requiring additional training data for novel poses.
Data Source
AI summary
Motion forecasting for autonomous systems includes obtaining map data of a geographic region and historical trajectories of agents located in the geographic region. The map data includes map elements. The agents and the map elements have a corresponding physical locations in the geographic region. Motion forecasting further includes building, from the historical trajectories and the map data, a heterogeneous graph for the agents and the map elements. The heterogeneous graph defines the corresponding physical locations of the agents and the map elements relative to each other of the agents and the map elements. Motion forecasting further includes modelling, by a graph neural network, agent actions of an agent of the agents using the heterogeneous graph to generate an agent goal location, and operating an autonomous system based on the agent goal location.