Volumetric Video Encoding via View-Dependent Patch Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current technologies face challenges in efficiently encoding and decoding volumetric video content due to its large data requirements, which hinder storage and transmission efficiency, especially for immersive 6DoF experiences that demand high bit-rates and storage capacities.
Innovation Solution
The method involves defining a reference viewing box and an intermediate viewing box within a 3D scene, encoding a central view and peripheral patches, and using metadata to describe these boxes, allowing for differential encoding and decoding that reduces data complexity and storage needs without increasing encoding or decoding complexity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If volumetric video content is encoded with high quality for 6DoF experiences, then viewing quality and immersion are improved, but data size and storage requirements increase significantly
Solution Approach 1:
The 3D scene is divided into multiple view-dependent image sets, each corresponding to a specific viewing bounding box. Within each set, images are segmented into patches that can be selectively encoded and transmitted. This segmentation allows the system to provide high-quality viewing experience for specific viewpoints while avoiding the need to encode and store all possible views at full quality, thus reducing overall data size.
Solution Approach 2:
The patent applies different encoding qualities to different regions of the 3D scene based on their importance and view-dependency. Central patches that are visible from multiple viewpoints are encoded with higher quality, while peripheral patches are encoded with lower quality or omitted entirely. This local quality differentiation maintains viewing quality for critical regions while reducing data size for less important areas.
2Adaptability or versatility
If complete 6DoF volumetric video is provided, then user freedom and immersion are improved, but encoding complexity and computational resources increase
Solution Approach 1:
The patent implements a dynamic encoding scheme where the level of detail and number of view-dependent image sets are adjusted based on the user's current viewpoint and movement within the 3D scene. As the user moves, the system dynamically selects which patches to encode and transmit, providing 6DoF freedom when needed while reducing encoding complexity when the user is stationary or viewing from limited angles.
Solution Approach 2:
Instead of encoding complete volumetric data for all possible viewpoints, the patent encodes only the necessary portions (patches) that are visible from the current or predicted viewpoints. This partial action approach provides sufficient 6DoF experience for practical use cases while avoiding the excessive computational complexity of encoding all possible views.
3Reliability
If multiple view-dependent image sets are encoded for different bounding boxes, then parallax and 3DoF+ experience are improved, but transmission bandwidth and processing requirements increase
Solution Approach 1:
The patent merges multiple view-dependent image sets into a unified data structure organized by viewing bounding boxes. Patches from different viewpoint sets are combined and encoded together, allowing the system to transmit consolidated data that provides parallax effects while reducing redundant information. This merging improves transmission efficiency by eliminating duplicate data across multiple image sets.
Solution Approach 2:
The patent transitions from encoding complete 2D images to encoding 3D patches with spatial coordinates and view-dependency metadata. This dimensional change allows the system to organize data in a multi-dimensional structure that efficiently represents multiple viewpoints, improving transmission efficiency by encoding only the essential 3D geometric information rather than redundant 2D image data for each viewpoint.
Data Source
AI summary
A method for encoding a volumetric video content representative of a 3D scene is disclosed. The method comprises obtaining a reference viewing box and an intermediate viewing box defined within the 3D scene. For the reference viewing bounding box, the volumetric video reference subcontent is encoded as a central image and peripheral patches for parallax. For the intermediate viewing bounding box, the volumetric video intermediate sub-content is encoded as intermediate central patches which are differences between the intermediate central image and the reference central image.