Face Reconstruction for Headset-Occluded Game Player Images
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Computer simulations, such as VR or AR games, often occlude players' faces due to headsets, hindering social interaction and enjoyment by obscuring full face visibility.
Innovation Solution
A machine learning model, utilizing RNNs like LSTM, reconstructs full-face images by overlaying computer-generated transparent headsets and incorporating additional information to refine the images, ensuring full-face visibility.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If players wear headsets in computer simulations, then immersion and simulation experience are improved, but face visibility and social interaction are worsened due to occlusion
Solution Approach 1:
The system creates a synthetic copy of the player's face by capturing images from multiple cameras (including inward-facing cameras on the headset) and using machine learning models to generate a reconstructed face image that replaces the occluded view, allowing players to see each other's faces while still wearing headsets
Solution Approach 2:
The system introduces an intermediary processing layer between the physical reality (player wearing headset) and the visual display (opponent's face), using machine learning models to translate occluded views into visible reconstructions, enabling social interaction without removing the headset
2Loss of information
If machine learning models are used to reconstruct full-face images, then face visibility is improved, but computational complexity and processing requirements are worsened
Solution Approach 1:
The system performs preliminary actions by capturing multiple images from different cameras before the actual reconstruction is needed, storing these images for later processing, and pre-training machine learning models to handle the reconstruction task efficiently during gameplay
Solution Approach 2:
The system divides the complex face reconstruction task into separate modules: image capture from multiple cameras, image processing and alignment, machine learning-based reconstruction, and display integration, allowing each component to be optimized independently
Data Source
AI summary
An image of a computer game player wearing a headset that occludes part of the face is input to a trained machine learning (ML) model, which outputs in response a full-face image that is not occluded for use in, e.g., social network settings related to the game.