Volumetric Content Atlas Encoding for Lower-Bitrate 6DoF Video
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies face challenges in efficiently encoding and decoding volumetric video content due to the large amount of data required, leading to bitrate issues in storage, transmission, and decoding performance, particularly when transitioning between 3DoF and 6DoF experiences.
Innovation Solution
The method involves creating attribute and geometry atlas images that pack patch pictures of a three-dimensional scene, along with metadata that includes projection parameters and depth values, allowing for efficient inverse projection of pixels based on these parameters, and encoding these images and metadata in a data stream.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If volumetric video content is encoded using traditional methods, then complete 6DoF navigation is achieved, but bitrate increases significantly causing storage and transmission issues
Solution Approach 1:
The patent divides the volumetric scene into multiple patches (e.g., 8 patches in a 2x2 grid) that are independently encoded and packed into atlas images. This segmentation allows selective transmission of only the necessary patches based on viewing conditions, reducing the overall bitrate while maintaining 6DoF navigation capability when needed.
Solution Approach 2:
The patent encodes multiple patches at higher quality than strictly necessary for basic 3DoF viewing, preparing excess data that can be utilized when users need 6DoF navigation. This partial over-encoding ensures that when full navigation is required, the data is already available without requiring real-time re-encoding, thus managing bitrate efficiently.
2Quantity of substance
If 3DoF video is used for immersive content, then storage and transmission are simplified, but user experience becomes frustrating due to lack of parallax and translation freedom
Solution Approach 1:
The patent creates a dynamic encoding system that can adapt between 3DoF and 6DoF modes based on user needs. The atlas image structure allows the system to provide 3DoF viewing when sufficient, but can switch to 6DoF navigation when users interact with the content, offering parallax and translation freedom on demand without continuously consuming high bitrate resources.
3Productivity
If multiple patch pictures are packed into atlas images, then decoding performance improves, but encoding complexity increases
Solution Approach 1:
The patent merges multiple patch pictures into a single atlas image structure, combining the encoding of multiple views into one unified image format. This merging process leverages existing efficient image encoding technologies, avoiding the need for complex multi-view video encoding while still enabling high decoding performance through the packed patch structure.
Data Source
AI summary
Methods and apparatus for encoding and decoding a volumetric scene are disclosed. A set of attribute and geometry patches is obtained by projecting samples of the volumetric scene onto the patches according to projection parameters. If the geometry patch is comparable to a planar layer located at a constant depth according to the projection parameters, only the attribute patch is packed in an attribute atlas image and the depth value is encoded in metadata. Otherwise, both attribute and geometry patches are packed in an atlas. At the decoding, if metadata for an attribute patch indicates that its geometry may be determined from the projection parameters and a constant depth, the attributes are inverse projected on a planar layer. Otherwise, attributes are inverse projected according to the associated geometry patch.