VR Viewpoint Video Generation from 3DoF Orientation Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Virtual reality and augmented reality experiences with limited degrees of freedom, such as those using smartphones, struggle to provide immersive views due to the lack of accurate position tracking, leading to a constrained viewing experience.
Innovation Solution
The system maps orientation data from a 3DoF device to an estimated viewer location using a calibrated three-dimensional shape, allowing for the generation and display of viewpoint video that simulates six degrees of freedom, by defining a ray at the viewer orientation and locating its intersection with a three-dimensional shape, and using vantages distributed throughout a viewing volume to reproject images accurately.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If viewpoint video is generated directly from captured video data, then viewing experience immersion is improved, but computational intensity increases causing lag
Solution Approach 1:
The system performs preliminary actions by pre-calculating and storing vantage point videos at multiple predetermined locations within the viewing volume before the actual viewing experience. This allows the mobile device to simply retrieve and display pre-computed video data rather than performing real-time computational intensive viewpoint generation, thereby maintaining immersion while eliminating computational lag.
2Adaptability or versatility
If six degrees of freedom position tracking is implemented, then viewing experience freedom is improved, but device complexity increases
Solution Approach 1:
The system creates a virtual copy of the six degrees of freedom tracking capability through software computation. Instead of requiring physical 6DoF sensors in the mobile device, the system uses a calibrated three-dimensional shape model to mathematically map orientation data to position information, effectively copying the functionality of dedicated 6DoF headsets through computational geometry rather than hardware sensors.
3Measurement precision
If orientation data mapping to viewer location is performed, then position estimation accuracy is improved, but computational requirements increase
Solution Approach 1:
The system performs preliminary computation by pre-calculating the mapping relationship between orientation data and viewer location during a calibration phase. The calibrated three-dimensional shape model stores pre-computed geometric relationships that allow the mobile device to quickly transform orientation measurements into position estimates using simple lookups rather than performing computationally intensive real-time calculations, thereby improving precision while minimizing energy consumption.
Data Source
AI summary
A virtual reality or augmented reality experience may be presented for a viewer through the use of input including only three degrees of freedom. The input may include orientation data indicative of a viewer orientation at which a head of the viewer is oriented. The viewer orientation may be mapped to an estimated viewer location. Viewpoint video may be generated of a scene as viewed from a virtual viewpoint with a virtual location corresponding to the estimated viewer location, from along the viewer orientation. The viewpoint video may be displayed for the viewer. In some embodiments, mapping may be carried out by defining a ray at the viewer orientation, locating an intersection of the ray with a three-dimensional shape, and, based on a location of the intersection, generating the estimated viewer location. The shape may be generated via calibration with a device that receives input including six degrees of freedom.