Volumetric Video Atlas Buffer for Consistent Prediction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
MPEG-I Immersive Video (MIV) technologies face challenges in achieving closed-loop prediction between encoders and decoders due to differences in video encoding and decoding processes, leading to compression inefficiencies and limited 6-DoF capabilities in volumetric video rendering.
Innovation Solution
Implementing a shared reconstructed atlas buffer and a warping-based prediction mechanism between the MIV encoder and decoder, allowing for independent video encoding and decoding processes while ensuring consistent predictions, and utilizing different granularities such as slice/tile-level, sub-picture-level, and layer-level for improved compression efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If basic view texture and depth pixels are uncompressed in the encoder, then prediction accuracy is improved, but bitrate increases
Solution Approach 1:
The atlas is divided into multiple patches that are processed independently. Each patch can be encoded with appropriate compression while maintaining prediction accuracy where needed. This segmentation allows selective application of compression techniques across different regions of the atlas.
Solution Approach 2:
The patent applies different compression parameters and techniques to different parts of the atlas. By adjusting compression strength, prediction modes, and encoding parameters dynamically, the system maintains prediction accuracy for critical regions while achieving overall bitrate reduction.
2Quantity of substance
If compression is applied at the encoder, then bitrate is reduced, but prediction consistency between encoder and decoder deteriorates
Solution Approach 1:
The patent implements feedback mechanisms where the encoder uses decoded reconstructed atlases from the decoder to generate predictions. This closed-loop approach ensures that both encoder and decoder operate with consistent reference data, maintaining prediction consistency while allowing compression to be applied.
Solution Approach 2:
The encoder creates copies of reconstructed atlas data and uses these copies for prediction operations. By working with copied reconstructed data rather than original uncompressed data, the system maintains consistency with the decoder's reconstructed versions while enabling compression.
3Adaptability or versatility
If different granularities are used for the shared reconstructed atlas buffer, then adaptability to different video codecs is improved, but device complexity increases
Solution Approach 1:
The patent implements a dynamic buffer management system where the granularity of the shared reconstructed atlas buffer can be adjusted based on the specific video codec being used. The system dynamically selects appropriate buffer organization levels (patch-level, tile-level, or full-atlas-level) to match the requirements of different codecs, maintaining adaptability while managing complexity through automated selection.
Data Source
Figure 1
Figure 2a~2b
Figure 2c~2d
AI summary
There are disclosed methods, apparatuses and computer program products for volumetric video compression. In accordance with an embodiment, the method for encoding comprises obtaining a basic view and one or more additional views of a volumetric video content; forming patches from the basic view and one or more additional views; packing patches into one or more regions of an atlas; encoding the one or more regions of the atlas; reconstructing patches from the encoded atlas; selecting a reference view; forming residual data based on the reconstructed patches and the reference view; and encoding the residual data. In accordance with an embodiment, the method for decoding comprises decoding residual data regarding reconstructed patches and a reference view; obtaining the reference view; forming patches of an atlas from the decoding residual data and the reference view; and reconstructing a basic view and one or more additional views of a volumetric video content from the patches.