VR Point Cloud Projection for Parallax and Occlusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current virtual reality systems face challenges in integrating video that can be viewed from multiple user heights and interpupillary distances while maintaining scene continuity and proper parallax, especially with position tracking, without missing pixels due to occlusion.
Innovation Solution
A method and system for generating and playing back virtual reality multimedia using a point cloud representation, capturing color and depth data, and projecting it in a two-dimensional format that can be rendered on a display, allowing for head movement tracking and interpupillary distance adjustments.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If fixed interpupillary distance solutions are used with stereoscopic images, then video integration is achieved, but the system cannot accommodate multiple user heights and interpupillary distances while maintaining proper parallax
Solution Approach 1:
The patent transitions from traditional 2D video projections to a 3D point cloud representation, where each point in space contains color and depth information. This dimensional transformation enables the system to accommodate multiple viewing positions, heights, and interpupillary distances while maintaining scene continuity and proper parallax effects, as the point cloud structure inherently preserves spatial relationships from all angles.
2Ease of operation
If traditional video streams are used per eye, then stereoscopic viewing is achieved, but missing pixels occur due to occlusion when users move
Solution Approach 1:
The system pre-captures and stores complete 3D point cloud data of the entire scene, including all points that may become visible when the user moves to different positions. This preliminary capture of comprehensive spatial information ensures that no pixels are missing during playback, regardless of user movement or head tracking, as all potential viewing angles are already represented in the point cloud structure.
3Reliability
If point cloud data is captured and projected, then scene continuity and proper parallax are maintained, but data processing complexity increases
Solution Approach 1:
The patent segments the 3D scene into discrete points, where each point contains color and depth information. This segmentation transforms the continuous complex scene into manageable individual elements that can be efficiently stored, transmitted, and rendered. The point cloud segmentation maintains scene continuity while reducing processing complexity compared to traditional mesh-based 3D representations.
Data Source
AI summary
A method is provided of generating virtual reality multimedia at a developer computing device having a processor interconnected with a memory. The method comprises: capturing, at the processor, a point cloud representing a scene, the point cloud data including color and depth data for each of a plurality of points corresponding to locations in the capture volume; generating, at the processor, a two-dimensional projection of a selected portion of the point cloud, the projection including the color and depth data for the selected portion; and storing the two-dimensional projection in the memory.