Reconstructing Obstructed Face Portions in VR Headsets
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current 3D virtual reality systems obstruct users' faces with headsets, leading to a subpar user experience due to loss of facial data, which is undesirable in applications requiring real-time image merging.
Innovation Solution
A method and system that reconstruct obstructed face portions by obtaining off-line 3D data of a user's head without a face-obstructing object, applying real-time 3D transformations, and merging the reconstructed data into the real-time 3D data to create a seamless virtual reality environment.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a virtual reality headset is worn to provide immersive 3D environment, then the virtual reality experience is improved, but the user's face becomes obstructed and facial data is lost
Solution Approach 1:
The system captures and stores a complete 3D model of the user's face before they wear the VR headset. This preliminary 3D scan creates a digital template that can be applied later to reconstruct facial portions that are obscured by the headset during the virtual reality experience.
Solution Approach 2:
The invention creates a digital copy of the user's face in the form of a 3D model. This copy is then used to generate synthetic facial images that replace the obscured portions of the real face, allowing other users to see a complete representation of the wearer's face even when the headset blocks part of it.
2Productivity
If real-time 3D data is captured to merge user images into the virtual environment, then the seamless integration is improved, but the face-obstructing object causes data loss
Solution Approach 1:
The system merges two data sources: the real-time 3D data captured during VR usage and the pre-captured complete 3D facial model. By combining these datasets, the system produces a complete facial representation that fills in the gaps caused by headset obstruction, enabling seamless integration into the virtual environment.
Solution Approach 2:
The pre-captured 3D facial model serves as an intermediary that bridges the gap between the limited real-time data and the complete facial appearance. This intermediary model provides the missing facial information that cannot be directly captured due to headset obstruction.
3Manufacturing precision
If the near eye display is integrated within the headset to provide stereoscopic viewing, then the 3D virtual environment quality is improved, but at least a portion of the face is obstructed
Solution Approach 1:
The invention transitions from 2D image capture to 3D facial modeling. By capturing and processing data in three dimensions, the system can reconstruct facial portions from multiple angles and depths, compensating for the physical obstruction caused by the headset's near-eye display integration.
Data Source
AI summary
A method and a system for reconstructing obstructed face portions are provided herein. The method may include the following steps: obtaining off-line 3D data, being 3D data of a head of a person not wearing a face-obstructing object, being an object which obstructs a portion of the face of the person; obtaining in real time, real-time 3D data, being 3D data of said head, wherein said person wears said face-obstructing object; applying a 3D transformation to at least a portion of the off-line 3D data, based on the real-time 3D data, to yield reconstructed real time 3D data, being real-time 3D data related to the obstructed face portions; and merging the reconstructed real time 3D data into the real-time 3D data. The system may implement the aforementioned steps over a computer processor.