3D Occupancy Prediction Using Forward-Backward View Transformation

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing 3D occupancy prediction methods for autonomous vehicles are inefficient in terms of time, quality, and computing resources, which hinders effective motion planning and obstacle avoidance.

Innovation Solution

A neural network-based 3D occupancy prediction model that combines forward projection and backward projection neural networks to generate a 3D voxel representation and birds-eye-view (BEV) representation, which are then combined to infer occupancy of voxels, utilizing pre-training for depth estimation and semantic segmentation.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If traditional 3D occupancy prediction methods are used, then computing resources and time are consumed, but prediction accuracy and efficiency remain insufficient

Engineering Contradiction:

Improveoccupancy prediction accuracyVSAvoidprediction efficiency

Core Design Contradiction:

Measurement precisionVSProductivity

Solution Approach 1:

The prediction process is divided into two independent stages: forward projection (generating 3D voxel representation from 2D images) and backward projection (generating BEV representation from 3D voxels). This segmentation allows each stage to be optimized independently and processed in parallel, improving both accuracy and efficiency without requiring full 3D reconstruction for every prediction task.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The method transforms the prediction problem from direct 3D space analysis to a two-stage process involving 2D image space and bird's-eye-view space. By projecting between different dimensional representations (2D→3D→2D), the system achieves more accurate occupancy prediction while reducing computational complexity compared to direct 3D methods.

Inventive Principle:

Principle #17Another dimension (Dimensionality change)

2Reliability

If complex neural network architectures are used to improve prediction quality, then computing resources increase

Engineering Contradiction:

Improveoccupancy prediction reliabilityVSAvoidcomputing resource consumption

Core Design Contradiction:

ReliabilityVSUse of energy by moving object

Solution Approach 1:

The method extracts only the essential features needed for occupancy prediction by separating the forward projection (feature extraction from images) and backward projection (feature aggregation to BEV) stages. This extraction approach avoids processing unnecessary data and reduces computing resource consumption while maintaining prediction reliability.

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The forward projection network pre-processes 2D image features into 3D voxel representations before the backward projection stage. This preliminary action organizes and structures the data in advance, reducing the computational burden on subsequent processing stages and lowering overall energy consumption while improving prediction reliability.

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS12515706B2Occupancy prediction using forward-backward view transformation

Publication Date: 2026.01.06 NVIDIA CORP

AI summary

Apparatuses, systems, and techniques of using one or more machine learning processes (e.g., neural network(s)) to predict occupancy using an image input. In at least one embodiment, image data is processed using a neural network to predict occupancy in a 3D voxel space. In at least one embodiment, image data is processed using a neural network to detect objects in a 3D space.