HEIF Multi-Plane Image Container for Texture and Opacity Coding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image file formats like HEIF do not effectively support multi-plane imaging (MPI) which requires encoding and decoding of texture and opacity information across multiple planes, leading to inefficiencies in rendering and processing volumetric content.
Innovation Solution
The proposed solution extends the HEIF file format to support MPI by encoding texture and opacity information using 2D video codecs like HEVC and VVC, and storing MPI metadata to enable efficient decoding and rendering of multi-plane images.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If existing image file formats like HEIF are used, then compatibility and standardization are maintained, but support for multi-plane imaging and volumetric content is insufficient
Solution Approach 1:
The patent segments the multi-plane image data into separate texture planes and opacity planes, with each plane stored as independent image data. This segmentation allows the existing HEIF format to accommodate MPI by organizing data in a structured manner without requiring fundamental format changes, thus improving adaptability while managing complexity.
Solution Approach 2:
The patent introduces a depth dimension to the traditional 2D image format by storing multiple planes at different depth levels. Each plane represents a specific depth slice of the volumetric content, enabling the HEIF format to represent 3D scenes through stacked 2D planes, thereby enhancing versatility without completely redesigning the container format.
2Manufacturing precision
If multi-plane images are encoded with texture and opacity information, then rendering quality and volumetric content capability are improved, but encoding and decoding complexity increases
Solution Approach 1:
The patent merges the texture and opacity information into a unified HEIF container structure, where both types of data are stored using the same codec framework. This merging approach allows standard video decoding infrastructure to handle MPI data, reducing the need for specialized decoding hardware or software while maintaining rendering quality.
Solution Approach 2:
The patent utilizes parameter sets within the HEIF format to store metadata about the multi-plane image structure, including depth information, plane dimensions, and compression parameters. By encoding this structural information as parameters rather than embedded data, the system maintains rendering precision while simplifying the decoding process through parameter-driven configuration.
3Adaptability or versatility
If multiple planes of texture and opacity are stored, then volumetric content representation is enhanced, but data size and storage requirements increase
Solution Approach 1:
The patent employs the universal HEIF container format to store multiple plane data, leveraging the same encoding infrastructure used for standard 2D images. This universality allows efficient storage of volumetric content using existing compression algorithms and container mechanisms, avoiding the need for specialized storage formats and reducing overall data size compared to proprietary MPI formats.
Solution Approach 2:
The patent nests multiple image planes within a single HEIF container structure, organizing texture and opacity planes in a hierarchical manner. This nesting approach allows compact storage of volumetric data by packing multiple data structures within the existing container framework, thereby enhancing representation capability while minimizing storage requirements through efficient space utilization.
4Adaptability or versatility
If HEIF format is extended to support MPI, then compatibility with existing systems is maintained, but implementation complexity increases
Solution Approach 1:
The patent introduces metadata structures as intermediaries between the HEIF container and the multi-plane image data. These metadata elements provide a standardized interface that describes the MPI structure, enabling existing HEIF parsers to accommodate MPI by first interpreting the metadata and then processing the corresponding plane data, thus maintaining compatibility while managing implementation complexity.
Solution Approach 2:
The patent extends HEIF by introducing new parameter sets and configuration options that can be optionally included in the container. These parameter changes allow the format to adapt to MPI requirements without breaking backward compatibility, as the extended parameters are optional and can be ignored by older systems. This approach enables gradual adoption and simplifies implementation by allowing selective support for MPI features.
Data Source
AI summary
Methods, systems, and bitstream syntax are described for a file container that supports the storage and transmission of multi-plane images. Examples are provided for coding texture and opacity information using HEVC or VVC coding and the HEIF container. Examples of carrying coded MPI images according to V3C and an example HEIF-based player are also presented.