Trajectory Prediction Fusion for Multimodal Agent Behavior

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing trajectory prediction systems for autonomous vehicles struggle to accurately capture the interactions between heterogeneous inputs, leading to inaccurate future trajectory predictions and mode collapse, particularly when dealing with multimodal agent behaviors.

Innovation Solution

A trajectory prediction system that encodes input world state using sparse polylines and raw agent state information, combined with a multi-context gating fusion component and learned latent anchor embeddings, to generate more accurate trajectory predictions by effectively capturing interactions and multi-modal behaviors.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Device complexity

If a single input representation is used to encode all perception signals and map information, then the system complexity is reduced, but the ability to capture interactions between different heterogeneous elements is lost

Engineering Contradiction:

Improveinput representation complexityVSAvoidinteraction information between heterogeneous elements

Core Design Contradiction:

Device complexityVSLoss of information

Solution Approach 1:

The system segments the input representation into multiple specialized encoders: a polyline encoder for road features, an agent state encoder for dynamic objects, and an interaction encoder for relationships between elements. Each encoder processes specific types of data independently, preserving the unique characteristics and interactions of heterogeneous elements while maintaining manageable system complexity through modular architecture.

Inventive Principle:

Principle #1Segmentation

2Productivity

If conventional encoding methods are used for trajectory prediction, then the system is computationally efficient, but mode collapse occurs and multi-modal agent behaviors are not captured

Engineering Contradiction:

Improvecomputational efficiencyVSAvoidtrajectory prediction accuracy and multi-modal behavior capture

Core Design Contradiction:

ProductivityVSReliability

Solution Approach 1:

The system introduces an interaction encoder as an intermediary component that processes relationships between agents and their environment before feeding into the trajectory prediction module. This intermediary layer captures multi-modal behaviors and contextual interactions without requiring computationally intensive post-processing, maintaining efficiency while improving prediction reliability through enriched feature representations.

Inventive Principle:

Principle #24Intermediary (Mediator)

3Ease of operation

If all input signals are represented in a single unified format, then the processing pipeline is simplified, but the heterogeneous nature of static and dynamic inputs is lost

Engineering Contradiction:

Improveprocessing pipeline simplicityVSAvoidhandling of heterogeneous input types

Core Design Contradiction:

Ease of operationVSAdaptability or versatility

Solution Approach 1:

The system implements a universal processing framework where multiple specialized encoders (polyline encoder, agent state encoder, interaction encoder) produce standardized embedding vectors that can be uniformly processed by downstream trajectory prediction modules. This multi-functional architecture maintains pipeline simplicity through consistent output formats while preserving the heterogeneous nature of inputs through specialized processing stages.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Data Source

PatentUS12420846B2Agent trajectory prediction using context-sensitive fusion

Publication Date: 2025.09.23 WAYMO LLC

AI summary

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for agent trajectory prediction using context-sensitive fusion.