360 Depth Content Rendering via Shader-Based Depth Interpretation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional methods of storing 3D content in conventional video file formats introduce artifacts and provide a poor user experience, especially for stereoscopic 3D 360 panoramic content, due to issues handling head movements and peripheral vision, and result in large file sizes.
Innovation Solution
Storing and playing back 360 depth content, which includes 360 panoramic image data and corresponding depth information, using a modified 3D rendering pipeline that employs vertex or fragment shaders to interpret and render the depth information, allowing for a more immersive experience and reduced file size by incorporating depth data into the video encoding process.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If stereoscopic video content is stored in conventional video file formats, then the content can be displayed on standard devices, but visual artifacts are introduced and user experience deteriorates
Solution Approach 1:
The patent transitions from conventional 2D video storage to a depth-aware representation by incorporating depth maps as a separate dimension. Each video frame is paired with a depth map that encodes distance information, enabling the system to distinguish between foreground and background elements. This dimensional enhancement allows for artifact reduction through depth-based filtering and view synthesis without requiring multiple full-resolution stereoscopic video streams.
Solution Approach 2:
The patent introduces depth maps as an intermediary data structure that mediates between the original video content and the final rendered output. These depth maps serve as a bridge that enables various processing operations such as view synthesis, focal plane rendering, and artifact removal. The depth information acts as a mediator that guides the rendering pipeline to reconstruct accurate 3D scenes from single-camera input.
2Adaptability or versatility
If stereoscopic 3D 360 panoramic content is displayed, then immersive experience is provided, but peripheral vision quality deteriorates and head movement handling becomes problematic
Solution Approach 1:
The patent implements dynamic view synthesis that adapts to the user's head movements and viewing direction. Rather than using fixed stereoscopic pairs, the system dynamically generates appropriate left and right eye views based on the user's current orientation. The depth maps enable real-time view synthesis that maintains high quality across the entire 360-degree field of view, allowing users to look up, down, left, right, or anywhere in between without experiencing peripheral vision degradation or artifacts.
3Reliability
If separate video content is encoded for each viewpoint, then accurate 3D representation is achieved, but file size increases significantly
Solution Approach 1:
The patent extracts only the essential depth information from the full video data, storing it in a compressed depth map format. Instead of encoding complete video streams for multiple viewpoints, the system extracts depth cues from single-camera video and stores them as separate depth map channels. This extraction approach retains the critical 3D structural information while discarding redundant visual data, achieving accurate view synthesis with significantly reduced storage requirements.
Solution Approach 2:
The patent changes the representation parameters from full-color video data to depth-encoded maps. By representing scenes in terms of depth values rather than full RGB information, the system achieves more efficient compression. The depth maps use fewer bits per pixel while still enabling accurate 3D reconstruction through view synthesis algorithms that leverage the depth information to generate realistic multi-view content.
Data Source
AI summary
As user device can receive and display 360 panoramic content in a 360 depth format. 360 depth content can comprise 360 panoramic image data and corresponding depth information. To display 360 depth content, the user device can generate a 3D environment based on the 360 depth content and the current user viewpoint. A content display module on the user device can render 360 depth content using a standard 3D rendering pipeline modified to render 360 depth content. The content display module can use a vertex shader or fragment shader of the 3D rendering pipeline to interpret the depth information of the 360 depth content into the 3D environment as it is rendered.