Surround View Occlusion Reconstruction Using Temporal 3D Scene Data

Solution Overview

Problem

Autonomous and semi-autonomous navigation systems face challenges in accurately interpreting sensor data due to occlusions caused by objects obstructing the view, which can lead to incomplete or imperfect understanding of the operating environment.

Innovation Solution

A system and method for occlusion reconstruction using temporal information, which involves generating a three-dimensional computerized representation of the environment, synthesizing a virtual camera view, and utilizing historical image data to estimate and fill in occluded areas with pixel data from immobile objects, employing techniques like depth interpretation, semantic segmentation, and epipolar reprojection.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If objects are used to obstruct the view in the operating environment, then the view is blocked and occlusion occurs, but this prevents accurate interpretation of sensor data and creates incomplete understanding of the environment

Engineering Contradiction:

Improveaccuracy of navigationVSAvoidoccluded view information

Core Design Contradiction:

ReliabilityVSLoss of information

Solution Approach 1:

The system performs preliminary actions by capturing image data from multiple time points (t-n, t-1, t) before the occlusion becomes problematic. Historical iterations of image data are stored and used to reconstruct occluded areas, allowing the system to anticipate and compensate for view blockages before they affect navigation accuracy.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system creates a virtual camera view that copies and synthesizes information from multiple historical image iterations. By generating a three-dimensional computerized representation and using epipolar reprojection, the system creates a virtual copy of the environment that can be reconstructed even when direct view is blocked, replacing missing information with synthesized data.

Inventive Principle:

Principle #26Copying

2Measurement precision

If the system uses only current image data to interpret the environment, then processing is simple and fast, but occlusions cause incomplete and inaccurate environmental understanding

Engineering Contradiction:

Improveenvironmental understanding accuracyVSAvoidsystem complexity

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

The system performs preliminary actions by capturing and storing image data from multiple time points before the occlusion becomes problematic. Historical iterations of image data are stored and used to reconstruct occluded areas, allowing the system to anticipate and compensate for view blockages before they affect navigation accuracy.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system transitions from two-dimensional image data to a three-dimensional computerized representation of the environment. By adding the temporal dimension (multiple time points) and spatial dimension (three-dimensional representation), the system can reconstruct occluded areas using depth information and historical data, significantly improving measurement precision.

Inventive Principle:

Principle #17Another dimension (Dimensionality change)

3Reliability

If the system reconstructs occlusions using historical image data and three-dimensional representation, then navigation accuracy improves, but processing time and computational resources increase

Engineering Contradiction:

Improvenavigational guidance reliabilityVSAvoidprocessing time

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The system performs preliminary actions by capturing and storing image data from multiple time points before the occlusion becomes problematic. Historical iterations of image data are stored and used to reconstruct occluded areas, allowing the system to anticipate and compensate for view blockages before they affect navigation accuracy.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system creates a virtual camera view that copies and synthesizes information from multiple historical image iterations. By generating a three-dimensional computerized representation and using epipolar reprojection, the system creates a virtual copy of the environment that can be reconstructed even when direct view is blocked, replacing missing information with synthesized data.

Inventive Principle:

Principle #26Copying

Data Source

PatentUS12475582B2System and method for occlusion reconstruction in surround views using temporal information

Publication Date: 2025.11.18 GM GLOBAL TECHNOLOGY OPERATIONS LLC

AI summary

A system for occlusion reconstruction in surround views using temporal information is provided. The system includes an active camera device generating image data describing a first view of an operating environment and a computerized visual data controller. The controller includes programming to analyze the image data to generate a three-dimensional computerized representation of the operating environment, utilize the image data and the representation of the operating environment to synthesize a virtual camera view of the operating environment from a desired viewpoint, and identify an occlusion in the virtual camera view. The controller further includes programming to utilize historical iterations of the image data identify immobile objects within the operating environment and utilize the historical iterations to estimate filled information for the occlusion. The controller further includes programming to reconstruct the occlusion with the filled information using pixel data from the immobile objects and utilize the representation to provide navigational guidance.