Multiplane Image Packing for Efficient Volumetric Video Transmission
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies face challenges in efficiently transmitting and rendering multiplane images (MPI) due to the complexity of packing and compressing texture and alpha layers, which affects the quality and flexibility of rendering novel views.
Innovation Solution
The method involves packing texture and alpha layers of MPI into 2D video frames, generating a video bitstream and metadata bitstream, and using compatible encoders and decoders to ensure efficient transmission and rendering of multiplane images.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If texture and alpha layers of MPI are packed into 2D video frames and compressed, then transmission efficiency and bandwidth utilization are improved, but the complexity of packing arrangement and metadata management increases
Solution Approach 1:
The patent segments the multiplane image into separate texture layers and alpha layers, which are then packed into 2D video frames. This segmentation allows independent compression and transmission of different layer types, improving overall transmission efficiency while enabling flexible packing arrangements that can be managed through structured metadata
Solution Approach 2:
The patent introduces a metadata bitstream as an intermediary that carries packing arrangement parameters and MPI-specific information. This metadata acts as a mediator between the compressed video bitstream and the decoder, providing necessary instructions for correct reconstruction without increasing the complexity of the compression process itself
2Adaptability or versatility
If multiple packing arrangements are supported for flexible rendering, then adaptability for different rendering scenarios is improved, but encoder and decoder complexity increases
Solution Approach 1:
The patent implements a universal packing framework that can accommodate multiple packing arrangements (spatial packing, temporal packing, interleaved packing) within a single encoder/decoder system. The standardized metadata format serves as a universal interface that describes any packing arrangement, allowing the same hardware/software to support various rendering scenarios without requiring separate specialized components for each packing type
Solution Approach 2:
The patent enables dynamic selection of packing arrangements based on rendering requirements. The encoder can adaptively choose the most suitable packing arrangement for different scenarios (e.g., spatial packing for high resolution, temporal packing for motion compensation), and the decoder can dynamically interpret different metadata formats, providing flexibility without fixed hardware constraints
3Loss of energy
If MPI layers are compressed into video bitstream, then data transmission and storage are optimized, but reconstruction accuracy and image quality may deteriorate
Solution Approach 1:
By segmenting MPI into separate texture and alpha layers before compression, the patent preserves the structural integrity and semantic meaning of each layer type. This segmentation allows for targeted compression strategies that maintain reconstruction quality, as each layer can be compressed according to its specific characteristics rather than being mixed together
Solution Approach 2:
The patent utilizes parameter changes in the metadata bitstream to control the compression process. Packing arrangement parameters, layer merging information, and dynamic range adjustment parameters allow optimization of compression ratios while maintaining image quality. The metadata provides the necessary instructions for reversible transformation, ensuring accurate reconstruction when decoding
Data Source
AI summary
Methods and apparatus for transmission of volumetric images in the MPI format. According to an example embodiment, texture and alpha layers of multiplane images are packed, as tiles, into a sequence of video frames. The sequence of video frames is then compressed to generate a video bitstream, which is transmitted together with a metadata bitstream specifying at least the parameters of the packing arrangement for the tiles in the sequence of video frames. Example packing arrangements include various selectable spatial and temporal arrangements for texture layers, alpha layers, and camera views. In some examples, the metadata bitstream is implemented using a SEI message and includes parameters selected from the group consisting of a size of the reference view, the number of layers in the multiplane image, the number of simultaneous views, one or more characteristics of the packing arrangement, layer merging information, dynamic range adjustment information, and reference view information.