Mixed Reality Navigation for Occluded Obstacle Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing navigation systems fail to effectively warn users of obstacles or hazards in their environment, particularly when the system's field of view is obstructed or when navigating new spaces.
Innovation Solution
A model-based mixed reality navigation system that uses depth cameras and environmental mapping to integrate real and virtual objects, filling in occluded regions based on expected geometries and using cloud data to enhance obstacle detection and interaction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the system uses depth cameras and environmental mapping to detect obstacles, then obstacle detection capability is improved, but the system fails to detect obstacles in occluded regions where the field of view is blocked
Solution Approach 1:
The system performs preliminary mapping of the environment using depth cameras to create a 3D model of the space, including potential obstacle locations. This preliminary action allows the system to predict and fill in occluded regions before the user actually encounters them, maintaining continuous awareness even when directly observing occluded areas.
Solution Approach 2:
The system creates virtual copies of the physical environment through 3D mapping and rendering. These virtual representations include inferred obstacles in occluded regions, allowing the system to display warning indicators in the user's field of view that correspond to real-world hazards not directly visible to the cameras.
2Measurement precision
If the system integrates real and virtual objects with correct interaction, then virtual object positioning is improved, but the complexity of the navigation system increases
Solution Approach 1:
The system merges the real-world environment captured by depth cameras with virtual obstacle representations into a unified 3D model. This integration allows virtual objects to be positioned accurately relative to real-world features while maintaining a single coherent navigation system that processes both real and virtual elements together.
Solution Approach 2:
The system introduces a 3D environmental map as an intermediary data structure that bridges real camera inputs and virtual object rendering. This intermediary representation simplifies the integration process by providing a common coordinate system and data format for both real and virtual objects, reducing overall system complexity despite the sophisticated integration required.
Data Source
Figure 1
Figure 2A
Figure 2B
AI summary
Head mounted display, HMD, with forward facing camera(s) for Mixed Reality or Merged Reality (MR), sometimes called Augmented Reality (AR). The user wearing the HMD can walk around the real world seeing virtual objects, static or moving, superimposed over actual objects. The available scenes is used to recognize objects and then to fill in the occluded or partially scanned regions based on the expected geometries of those objects.