3D Scene Encoding via Depth Layer Sorting for Variable-Perspective Playback
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing 3D video formats fail to achieve efficient compression, transmission, and playback while providing variable-perspective playback, which is essential for maintaining viewer comfort and realism in 6 degree-of-freedom displays as users move their heads.
Innovation Solution
A method and system that compress, transmit, and display 3D scenes by sorting features into scene layers with depth, color, and transparency maps, allowing for efficient compression and variable-perspective playback, suitable for streaming to 6 DoF displays like VR and AR headsets.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If existing 3D video formats are used, then compression and transmission are achieved, but variable-perspective playback capability is lost
Solution Approach 1:
The patent segments the 3D scene into multiple depth layers, each representing a different depth range. This segmentation allows independent processing and compression of each layer while preserving the ability to reconstruct variable perspectives by selectively rendering layers based on camera position and head pose, thus resolving the contradiction between compression efficiency and variable-perspective capability.
Solution Approach 2:
The patent introduces a depth dimension by organizing scene features into multiple depth layers beyond the traditional 2D image plane. This dimensional organization enables efficient compression through depth-based grouping while maintaining variable-perspective playback by allowing dynamic selection and rendering of appropriate layers based on viewing conditions.
2Manufacturing precision
If high-quality 3D scene data is transmitted, then playback quality is improved, but bandwidth consumption increases
Solution Approach 1:
The patent extracts only the essential depth and perspective information needed for variable-perspective playback by organizing scene data into depth layers. This extraction allows transmission of compressed representation data rather than full high-resolution 3D data, maintaining playback quality while significantly reducing bandwidth consumption.
Solution Approach 2:
The patent changes the parameter representation by encoding scene geometry in terms of depth layer parameters rather than full pixel data. This parameter-based encoding maintains visual quality for variable perspectives while reducing data quantity, as only key depth and transformation parameters need to be transmitted rather than complete image data for all possible views.
3Ease of operation
If variable-perspective rendering is enabled, then viewer comfort is improved, but processing complexity increases
Solution Approach 1:
The patent performs preliminary organization of scene features into depth layers during encoding. This preliminary action simplifies subsequent variable-perspective rendering at playback, as the layered structure pre-computes depth relationships, reducing the processing complexity required for real-time perspective changes while maintaining viewer comfort.
Solution Approach 2:
The patent implements dynamic perspective rendering by allowing selective activation and rendering of depth layers based on current camera position and head pose. This dynamic approach optimizes processing complexity by only rendering necessary layers for the current view rather than processing all scene data, while still providing smooth variable-perspective playback for viewer comfort.
Data Source
AI summary
A computer-implemented method for encoding a scene volume includes: (a) identifying features of a scene volume that are within a camera perspective range with respect to a default camera perspective; (b) converting the identified features into rendered features; and (c) sorting the rendered features into a plurality of scene layers, each including corresponding depth, color, and transparency maps for the respective rendered features. Further, (a), (b), and (c) may be repeated, operating on temporally ordered scene volumes, to produce and output a sequence encoding a video. Corresponding systems and non-transitory computer-readable media are disclosed for encoding a 3D scene and for decoding an encoded 3D scene. Efficient compression, transmission, and playback of video describing a 3D scene can be enabled, including for virtual reality displays with updates based on a changing perspective of a user viewer for variable-perspective playback.