Trajectory Forecast Heatmaps Using Gated Relation Encoding

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Current algorithms for autonomous driving struggle to accurately predict the future trajectory of targets, especially in real-world scenarios, due to limitations in analyzing human-human and human-space interactions, which restricts their feasibility for far future predictions.

Innovation Solution

A method involving a Gated Relation Encoder (GRE) component, Trajectory Prediction Network (TPN), and Spatial Refinement Network (SRN) that processes images to identify spatial and spatial-temporal features, generates relational states, and produces heatmaps indicating a projected trajectory, using a combination of 2D and 3D fully convolutional networks, LSTM, and a relation gate module to refine predictions.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If algorithms analyze human-human interactions with social pooling layer, then social interaction understanding is improved, but trajectory prediction accuracy for far future deteriorates

Engineering Contradiction:

Improvesocial interaction understandingVSAvoidtrajectory prediction accuracy

Core Design Contradiction:

Adaptability or versatilityVSMeasurement precision

Solution Approach 1:

The algorithm segments the analysis into multiple interaction boundary layers: a first interaction boundary for nearby targets and local surroundings, and a second interaction boundary for far future predictions. This segmentation allows different analysis depths for different prediction timeframes, resolving the contradiction between social interaction understanding and far future prediction accuracy.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The algorithm applies different analysis qualities to different spatial regions: dense social interaction analysis (scene context fusion layer) for nearby targets within the first interaction boundary, and sparse analysis for distant targets within the second interaction boundary. This local quality differentiation maintains prediction accuracy across varying distances and timeframes.

Inventive Principle:

Principle #3Local quality

2Productivity

If algorithms restrict interaction boundary to nearby targets, then computational efficiency is improved, but far future prediction capability deteriorates

Engineering Contradiction:

Improvecomputational efficiencyVSAvoidfar future prediction capability

Core Design Contradiction:

ProductivityVSReliability

Solution Approach 1:

The algorithm dynamically adjusts the interaction boundary based on prediction requirements: using a first (smaller) interaction boundary for near-term predictions to maintain computational efficiency, and expanding to a second (larger) interaction boundary for far future predictions to ensure reliability. This dynamic adaptation resolves the contradiction between efficiency and capability.

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The algorithm adds a temporal dimension to the interaction boundary concept, creating time-varying boundaries: a first interaction boundary for short-timeframe predictions and a second interaction boundary for long-timeframe predictions. This dimensional extension allows the system to maintain both computational efficiency for immediate predictions and prediction capability for distant future scenarios.

Inventive Principle:

Principle #17Another dimension (Dimensionality change)

3Measurement precision

If algorithms use scene context fusion layer for human-space interactions, then local environment understanding is improved, but model feasibility for real-world scenarios deteriorates

Engineering Contradiction:

Improvelocal environment understandingVSAvoidreal-world scenario feasibility

Core Design Contradiction:

Measurement precisionVSAdaptability or versatility

Solution Approach 1:

The algorithm creates a universal framework that handles multiple interaction types (human-human and human-space) through a unified multi-layer architecture. The scene context fusion layer processes human-space interactions while the social pooling layer handles human-human interactions, with both layers operating within a single algorithmic framework that adapts to various real-world scenarios, resolving the contradiction between specialized understanding and general feasibility.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Data Source

PatentUS11062141B2Methods and apparatuses for future trajectory forecast

Publication Date: 2021.07.13 HONDA MOTOR CO LTD

AI summary

Aspects of the present disclosure may include methods, apparatuses, and computer readable media for receiving a plurality of images having a plurality of interactions associated with a plurality of objects, identifying a plurality of spatial features from the plurality of images, identifying a plurality of spatial-temporal features from the plurality of spatial features, obtaining a past trajectory data of the target, generating a relational state of the target and the plurality of interactions based at least on the past trajectory data, the plurality of spatial-temporal features, or the plurality of descriptive relations, and generating a plurality of heatmaps indicating a projected trajectory of the target.