VR Scene Equipment Removal via GAN and Skeletal Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current virtual reality experiences are limited in providing shared, realistic interactions among multiple participants without the cumbersome appearance of virtual reality equipment within the virtual environment.
Innovation Solution
A system utilizing high-fidelity three-dimensional scanners and low-fidelity high-speed sensors to capture detailed scans and skeletal modeling data, generating realistic animations of participants' movements and appearances, excluding virtual reality equipment from the virtual reality scene, and enhancing augmented reality experiences by projecting supplementary content onto surfaces based on participants' gaze and pose.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If virtual reality equipment is used to provide virtual reality experiences, then rich sensory feedback and interactive experiences are achieved, but the equipment appears cumbersome and unwanted within the virtual reality scene
Solution Approach 1:
The patent extracts and removes the unwanted virtual reality equipment from the virtual reality scene while preserving the functional capabilities. The system captures images of participants wearing VR equipment, then uses image processing and generative adversarial networks to generate modified images where the equipment is removed, leaving only the participant's natural appearance in the virtual environment.
Solution Approach 2:
The patent creates a realistic copy or representation of participants without the VR equipment. By using generative adversarial networks, the system generates synthetic images that copy the appearance of participants as if they were not wearing VR equipment, thereby providing a realistic representation that eliminates the cumbersome equipment appearance while maintaining the virtual reality experience quality.
2Measurement precision
If detailed three-dimensional scans are collected to generate realistic participant representations, then realism is enhanced, but data processing time and computational resources increase
Solution Approach 1:
The patent performs preliminary actions by capturing detailed three-dimensional scans of participants before the virtual reality experience begins. This pre-processing allows the system to have participant models ready in advance, reducing real-time processing requirements during the actual virtual reality session.
Solution Approach 2:
The patent uses dynamic skeletal modeling to track participant movements in real-time, updating the pre-generated detailed models with current pose information. This dynamic approach allows the system to maintain high realism through detailed scans while reducing processing time by only updating movement data rather than re-scanning all details continuously.
Data Source
AI summary
A system for providing a virtual reality experience obtains, from a first sensor, image data of a first subject upon entry to an area. The system obtains pose information for the first subject during a period in which the subject is within the area. A graphical asset is generated from the image data. A skeletal model is maintained based on the pose information. A virtual reality scene is generated, using the asset and skeletal model, from the perspective of a second subject.