Single-Image Traversable Space Prediction Beyond Occlusions
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for predicting traversable space in a scene are limited by their reliance on visible surfaces, failing to account for occluded surfaces and requiring resource-intensive computations, while existing models that predict beyond the line-of-sight often use expensive voxel-based techniques or require complete, static training environments.
Innovation Solution
A machine-learned model trained from stereo video sequences and semantic segmentation data predicts both visible and occluded traversable surfaces using a lightweight representation, capable of determining traversable space from a single RGB image, accounting for object footprints and movement.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If voxel-based techniques are used to predict traversable space beyond line-of-sight, then occluded traversable space can be detected, but computational cost increases significantly
Solution Approach 1:
The patent uses a lightweight neural network model that copies the essential functionality of expensive voxel-based methods but implements it through learned image-to-image mapping. The model learns to predict occluded traversable space directly from visible image data without requiring full 3D voxel reconstruction, achieving similar detection capability with much lower computational overhead.
Solution Approach 2:
The patent changes the representation parameters from dense 3D voxel grids to lightweight 2D depth maps and segmentation masks. This parameter transformation allows the model to capture occluded traversable space information in a compressed format that requires significantly fewer computational resources while maintaining detection accuracy.
2Measurement precision
If complete static training environments are used, then training data quality improves, but adaptability to dynamic real-world scenes decreases
Solution Approach 1:
The patent trains the model on dynamic video sequences rather than static images, allowing the network to learn temporal patterns and motion cues. This enables the model to adapt to moving objects and changing scenes while maintaining training data quality through the use of temporally coherent multi-frame inputs and optical flow information.
Solution Approach 2:
The patent performs preliminary actions during training by pre-processing video sequences to extract depth information, semantic segmentation, and motion cues. These pre-computed features are used to supervise the training of the lightweight model, enabling it to learn from rich training data while maintaining fast inference speed during deployment.
3Device complexity
If line-of-sight based scene analysis is used, then computational simplicity is maintained, but detection of occluded traversable surfaces is lost
Solution Approach 1:
The patent introduces an intermediary neural network model that acts as a bridge between simple line-of-sight analysis and complex 3D reconstruction. The model uses learned priors from training data to infer occluded traversable space from visible cues alone, maintaining algorithmic simplicity while recovering information beyond the line-of-sight through data-driven reasoning.
Data Source
AI summary
A model predicts the geometry of both visible and occluded traversable surfaces from input images. The model may be trained from stereo video sequences, using camera poses, per-frame depth, and semantic segmentation to form training data, which is used to supervise an image to image network. In various embodiments, the model is applied to a single RGB image depicting a scene to produce information describing traversable space of the scene that includes occluded traversable. The information describing traversable space can include a segmentation mask of traversable space (both visible and occluded) and non-traversable space and a depth map indicating an estimated depth to traversable surfaces corresponding to each pixel determined to correspond to traversable space.