Vantage Generation for Immersive VR Playback
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current virtual and augmented reality systems face challenges in generating high-quality, immersive experiences due to computationally intensive viewpoint video generation, leading to lag and reduced immersion caused by six degrees of freedom, parallax, and view-dependent lighting.
Innovation Solution
A system that designates multiple vantage points within a viewing volume, retrieves and combines images from these points to generate combined images, which are then reprojected onto three-dimensional shapes, incorporating color and depth data, and adjusted for occlusion and view-dependent shading, allowing for efficient generation of viewpoint video.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If viewpoint video is generated directly from captured video data, then high-quality immersive experience with six degrees of freedom, parallax, and view-dependent lighting is achieved, but computational intensity increases resulting in lag
Solution Approach 1:
The system pre-generates and stores multiple vantages (pre-computed viewpoint videos) from different locations within the viewing volume before the user views the content. When the user looks in a particular direction, the system quickly retrieves and composites the relevant pre-computed vantages rather than generating the viewpoint video in real-time, thereby maintaining high immersive quality while eliminating computational lag.
2Manufacturing precision
If multiple vantages are generated and stored for all possible viewing positions, then high-resolution stereo imagery is available for any viewing position and orientation, but data storage requirements and processing complexity increase
Solution Approach 1:
The viewing volume is divided into multiple discrete locations, and vantages are generated only for these specific segmented positions rather than for every possible continuous position. During playback, the system selects and composites vantages from these predetermined locations based on the user's actual viewing position, achieving high resolution accuracy where needed while reducing overall system complexity.
Solution Approach 2:
The system generates high-quality vantages only at specific discrete locations within the viewing volume where they are most needed, rather than uniformly across all positions. The composite image technique selectively combines vantages from nearby locations, providing high viewing resolution accuracy at the user's actual position while avoiding the complexity of pre-generating and storing data for all possible positions.
Data Source
AI summary
Video data of an environment may be prepared for presentation to a user in a virtual reality or augmented reality experience. According to one method, a plurality of locations distributed throughout a viewing volume may be designated, at which a plurality of vantages are to be positioned to facilitate viewing of the environment from proximate the locations. For each location, a plurality of images of the environment, captured from viewpoints proximate the location, may be retrieved. For each location, the images may be reprojected to a three-dimensional shape and combined to generate a combined image. The combined image may be applied to one or more surfaces of the three-dimensional shape to generate a vantage. The vantages may be stored such that the vantages can be used to generate viewpoint video of the scene, as viewed from a virtual viewpoint corresponding to an actual viewer's viewpoint within the viewing volume.