Spherical 3D Video Rendering Using Pixel Shift Depth Maps
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current spherical video rendering for virtual reality lacks depth component, resulting in sub-optimal immersion experiences, especially when users tilt their heads or look in certain directions, due to the reliance on two-dimensional images and ineffective 3D rendering techniques that do not account for the user's ability to move and view from different angles.
Innovation Solution
A method involving the generation of a spherical depth map by comparing images from pairs of cameras, stitching these maps together, and applying pixel shift values to create a modified spherical image, which is then combined with the spherical color video to produce a natural and immersive 3D video experience, utilizing a single color video and depth video instead of two, thereby reducing bandwidth requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If spherical video is rendered as 2D monoscopic images, then bandwidth requirements are reduced, but depth perception and immersion experience are lost
Solution Approach 1:
The patent transforms 2D spherical video into 3D stereoscopic video by introducing depth information through pixel shift values. Each pixel is shifted along the viewing direction based on its depth value, effectively adding a third dimension to the original 2D images while maintaining spherical geometry. This allows the system to provide depth perception without requiring multiple complete 360-degree image sets.
Solution Approach 2:
The patent applies depth-based pixel shifting locally to specific regions of the spherical video rather than uniformly processing the entire image. By calculating pixel shift values based on depth maps for specific directions (up, down, left, right), the system selectively modifies only the portions of the video that need depth enhancement, reducing overall processing complexity and bandwidth requirements compared to full 3D rendering.
2Loss of information
If traditional 3D rendering techniques are used for spherical video, then depth perception is provided, but the solution does not work when users tilt their heads or look in certain directions
Solution Approach 1:
The patent implements a dynamic rendering system that adapts to user head movements and viewing directions in real-time. Instead of pre-rendering fixed 3D views, the system calculates pixel shift values dynamically based on the user's current gaze direction and head orientation. This allows the spherical 3D video to maintain depth perception and natural appearance regardless of user movement, as the rendering parameters are continuously adjusted to match the user's perspective.
3Loss of information
If multiple cameras are used to capture spherical video for 3D rendering, then depth information is captured, but device complexity increases
Solution Approach 1:
The patent segments the spherical video capture into multiple directional views (up, down, left, right) that can be captured by fewer cameras positioned strategically. Rather than requiring a full array of cameras for complete spherical coverage, the system divides the spherical field of view into segments that can be captured by limited camera pairs, reducing hardware complexity while still enabling comprehensive depth map generation through stitching algorithms.
Data Source
AI summary
Systems and methods are disclosed for spherical three dimensional video rendering for virtual reality. A method includes receiving a spherical two-dimensional (2D) input image and a corresponding spherical depth map to be used in a creation of a spherical three-dimensional (3D) video, determining, by a processing device, a pixel shift value for each pixel of the spherical 2D input image based on the spherical depth map, and generating, based on the spherical 2D input image and the pixel shift values, a modified spherical image, wherein the modified spherical image in combination with the spherical 2d input image comprises at least one frame in the spherical 3D video.