Partial Passthrough Video Overlay for VR Immersion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing virtual reality systems require users to remove or deactivate their devices to see the real world, which disrupts the virtual reality experience and can lead to collisions with real-world objects.
Innovation Solution
A partial passthrough system that allows users to view a video feed of their environment overlaid on the virtual environment using cameras and sensors, triggered by events such as hand gestures or proximity to real-world objects.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If users remove or deactivate their virtual reality device to see the real world, then they can view their environment, but their virtual reality experience is disrupted and they risk colliding with real-world objects
Solution Approach 1:
The patent segments the visual field into virtual and real-world portions, allowing users to view only specific portions of the real world (e.g., exit signs, doors, hazards) while maintaining immersion in the virtual environment. This is achieved through selective passthrough modes that display real-world video feeds in specific regions or contexts within the virtual display.
Solution Approach 2:
The system introduces an intermediary layer (video feed of real world) that mediates between the virtual environment and the user's need to see the real world. Instead of removing the VR device entirely, the real-world video feed is integrated as an intermediary element within the virtual display, allowing simultaneous access to both virtual and real-world information.
2Reliability
If users maintain full virtual reality immersion, then they have an uninterrupted virtual experience, but they cannot see real-world objects or people
Solution Approach 1:
The system dynamically adjusts the level of real-world visibility based on user needs and context. Passthrough modes can be activated temporarily for specific tasks (e.g., locating an exit, interacting with a physical object) and then deactivated to restore full virtual immersion. The system adapts between full VR mode and partial passthrough mode seamlessly.
Solution Approach 2:
The patent implements periodic or temporary activation of passthrough modes rather than continuous display. Users can briefly activate real-world visibility when needed and return to full virtual immersion afterward. This periodic switching maintains overall VR experience continuity while providing periodic access to real-world information.
3Loss of information
If users activate full video passthrough to see the real world, then they can view their environment, but they lose the virtual reality experience completely
Solution Approach 1:
Instead of providing uniform full passthrough, the system applies real-world visibility locally to specific regions or contexts within the virtual environment. For example, real-world video may be displayed only in peripheral regions, or only when specific conditions are met (e.g., approaching a wall, needing to locate an object). This local application preserves virtual experience quality while providing necessary real-world information.
Solution Approach 2:
The patent implements partial passthrough rather than complete passthrough, providing just enough real-world visibility to satisfy user needs without completely sacrificing the virtual experience. The degree of passthrough can be adjusted to provide minimal necessary information while maintaining overall VR immersion.
Data Source
AI summary
Systems and methods for providing partial passthrough video to a user of a virtual reality device are disclosed herein. Providing the partial passthrough video can include detecting a hand passthrough trigger event and identifying a hand passthrough video feed. Providing partial passthrough video can further include aligning the hand passthrough video feed with a virtual environment presented to a user by the virtual environment and, based on the aligning of the hand passthrough video feed with the virtual environment, overlaying the hand passthrough video feed on the virtual environment.