Shared Mixed Reality Rendering with Live Participant Visibility
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Participants in a shared virtual reality (VR) experience in the same physical room cannot see each other in their true physical form, and existing solutions either use synthetic avatars or require a prepared room with green screens.
Innovation Solution
A system utilizing VR headsets with forward-facing color camera stereo pairs and auxiliary processors to calculate and render a mixed reality scene based on the participants' positions and orientations, allowing them to see each other and physical objects in their actual form while replacing the room with a rendered VR scene.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If participants wear VR headsets to share a virtual reality experience, then immersion in the virtual environment is improved, but the ability to see each other and physical objects in their true form deteriorates
Solution Approach 1:
The patent introduces forward-facing color cameras as intermediaries that capture images of physical participants and objects, which are then integrated into the virtual environment. These cameras act as mediators between the physical and virtual worlds, allowing participants to see each other's true forms while maintaining virtual immersion.
Solution Approach 2:
The patent merges the virtual environment with real-time camera feeds of physical participants and objects. By combining the pre-rendered virtual scene with live video input, the system creates a hybrid experience where both virtual and physical elements coexist, allowing participants to see each other in their true form while immersed in the virtual setting.
2Adaptability or versatility
If green screen algorithms are used to replace room walls with virtual environments, then virtual environment immersion is improved, but the complexity of the system and preparation required increases
Solution Approach 1:
The patent extracts the green screen requirement entirely from the system. Instead of requiring participants to be in a prepared room with green walls, the system uses forward-facing cameras to capture the natural room environment and selectively replaces only the background portions that need virtual replacement, leaving physical participants and objects visible in their natural setting.
Solution Approach 2:
The patent performs preliminary rendering of the virtual environment offline before the VR session begins. This pre-rendered virtual scene is then combined with real-time camera feeds during the session, eliminating the need for real-time green screen processing and reducing system complexity during actual use.
3Ease of operation
If synthetic avatars are used to represent participants, then social interaction capability is improved, but the authenticity of physical presence deteriorates
Solution Approach 1:
The patent creates a hybrid representation where physical participants are copied into the virtual environment through camera feeds rather than being fully replaced by synthetic avatars. This copying approach maintains the authenticity of physical presence while enabling social interaction within the virtual space, as participants see each other's actual images rather than artificial representations.
Data Source
AI summary
A system for implementing a shared mixed reality experience to participants in a same physical room having a plurality of VR headsets, each of which is adapted to be worn by a participant in the room. Each VR headset having a forward-facing color camera stereo pair. The system includes a computer in communication with at least one of the VR headsets. The system includes a memory in communication with the computer. The memory storing an original 3D digital representation of the room. The computer calculates a rendered VR scene for the at least one of the headsets from a point of view of each of the participant's two eyes, based on the VR headset's current position and orientation. A method for implementing a shared mixed reality experience to participants in a same physical room.