Mixed Reality Object Rendering for VR Spatial Awareness
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In immersive virtual reality (VR) environments, users face challenges in interacting with real-world objects and maintaining spatial awareness due to occlusion by VR systems, which can lead to disorientation and potential collisions with real-world objects.
Innovation Solution
The integration of a passthrough view of the real-world environment within the VR environment, allowing users to identify real-world objects and render them as mixed reality (MR) objects, enabling interaction and maintaining spatial awareness without disrupting the immersion.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a virtual boundary wall is displayed to alert users of safe zones, then user safety is improved, but immersion and visual quality are deteriorated due to grid-like appearances breaking the VR experience
Solution Approach 1:
The patent creates a copy of the real-world object (e.g., desk, chair) as a virtual MR object within the VR environment. This copy serves the dual purpose of maintaining immersion (since it looks like a natural VR object) while still providing the safety boundary function. The virtual copy replicates the visual appearance of the real object, allowing users to interact with it naturally without noticing it's a safety mechanism.
Solution Approach 2:
The MR object acts as an intermediary between the real-world safety boundary and the virtual VR environment. Instead of directly displaying a grid-like boundary wall that breaks immersion, the system uses an intermediate representation (the MR object) that mediates between the physical world and virtual space, providing safety information in an immersive way.
2Reliability
If traditional virtual boundary walls are used to define safe zones, then user safety is improved, but device complexity and ease of operation are worsened due to manual boundary drawing requirements
Solution Approach 1:
The system performs automatic object detection and MR object generation without requiring manual user input for boundary setup. The camera system automatically identifies real-world objects, and the system autonomously creates corresponding virtual MR objects. This self-service approach eliminates the need for users to manually draw boundaries or configure safety zones, reducing complexity while maintaining safety.
Solution Approach 2:
The system performs preliminary object detection and MR object creation automatically before the user needs to interact with the boundary. By pre-establishing the safety boundaries through automatic detection, the system eliminates the need for complex manual setup procedures, making the process simpler and more intuitive for users.
3Illumination intensity
If VR systems fully occlude the real-world environment for immersion, then immersion quality is improved, but spatial awareness and user safety are worsened due to inability to see real-world objects
Solution Approach 1:
The patent merges the real-world environment with the virtual VR environment by integrating MR objects that correspond to real-world objects into the VR scene. This combination allows users to maintain immersion in the virtual environment while simultaneously being aware of real-world objects through their virtual representations, thus preserving both immersion quality and spatial awareness.
Solution Approach 2:
The system creates virtual copies of real-world objects (MR objects) that are integrated into the VR environment. These copies provide users with spatial awareness information about real-world objects while maintaining the immersive VR experience. Users can see and interact with these copies in the virtual space, which correspond to actual objects in the physical environment.
Data Source
AI summary
In one embodiment, a method includes displaying, for one or more displays of a virtual VR device, a first output image comprising a passthrough view of a real-world environment. The method includes identifying, using one or more images captured by one or more cameras of the VR display device, a real-world object in the real-world environment. The method includes receiving a user input indicating a first dimension corresponding to the real-world object. The method includes automatically determining, based on the first dimension, a second and third dimension corresponding to the real-world object. The method includes rendering, for the one or more displays of the VR display device, a second output image of a VR environment. The VR environment includes a MR object that corresponds to the real-world object. The MR object is defined by the determined first, second, and third dimensions.