Single-Image Traversable Surface Geometry With Occlusion Prediction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for predicting traversable space in a scene from a single image are limited, as they often rely on visible surfaces and fail to detect occluded traversable areas, requiring resource-intensive computations and incomplete or synthetic training environments.
Innovation Solution
A model that predicts the geometry of both visible and occluded traversable surfaces using stereo video sequences, camera poses, per-frame depth, and semantic segmentation to form training data, which is used to supervise an image-to-image network, allowing for efficient determination of traversable space from a single RGB image.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If existing methods predict traversable space using line-of-sight algorithms, then visible surfaces can be detected, but occluded traversable surfaces behind objects cannot be detected
Solution Approach 1:
The system performs preliminary depth prediction and semantic segmentation on the input image before path planning. By predicting depth maps and identifying traversable regions in advance, the system prepares information about occluded areas before they are needed for navigation decisions, reducing information loss without adding complex real-time processing
Solution Approach 2:
The system creates a copied representation of the scene in the form of a depth map and semantic segmentation mask. This copy contains predicted information about occluded traversable surfaces that is not directly visible in the original image, allowing the system to plan paths without needing complex 3D reconstruction
2Loss of information
If voxel or mesh parameterization is used to predict beyond line-of-sight, then occluded traversable space can be estimated, but computational cost becomes expensive
Solution Approach 1:
The system uses lightweight, disposable predictions of depth maps and semantic segmentation masks instead of expensive, persistent voxel or mesh representations. These predictions are generated once from the input image and discarded after path planning, providing necessary information about occluded areas without the high computational cost of full 3D reconstructions
Solution Approach 2:
The system extracts only the essential information needed for path planning - specifically, depth predictions and semantic segmentation masks indicating traversable regions - from the input image. This extraction approach avoids the computational burden of complete voxel or mesh parameterization while still providing information about occluded traversable space
3Measurement precision
If complete training environments with full spatial coverage are used, then model accuracy improves, but training data requirements and computational resources increase
Solution Approach 1:
The system uses partial action by training on sequences that cover only the necessary portion of the environment for path planning. Rather than requiring complete 360-degree coverage, the model learns from sequences that provide sufficient information about traversable and non-traversable regions, reducing training data requirements while maintaining practical accuracy
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A model predicts the geometry of both visible and occluded traversable surfaces from input images. The model may be trained from stereo video sequences, using camera poses, per-frame depth, and semantic segmentation to form training data, which is used to supervise an image to image network. In various embodiments, the model is applied to a single RGB image depicting a scene to produce information describing traversable space of the scene that includes occluded traversable. The information describing traversable space can include a segmentation mask of traversable space (both visible and occluded) and non-traversable space and a depth map indicating an estimated depth to traversable surfaces corresponding to each pixel determined to correspond to traversable space.