Multi-Plane Image Conversion to Layered Meshes for 3D Compression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Three-dimensional images and immersive content are resource-intensive to store and render, posing challenges in efficient file size and computational demands.
Innovation Solution
Generate multi-depth images (MDI) from multi-plane images (MPI) by splitting MPI into sub-volumes, calculating meshes and RGBA textures, and using machine learning to optimize rendering errors, resulting in a compressed representation with fewer layers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multi-plane images are stored with many fronto-parallel planes to maintain image quality, then image quality is preserved, but file size and storage requirements increase significantly
Solution Approach 1:
The patent segments the continuous depth space into discrete depth bins, grouping multiple fronto-parallel planes into fewer depth layers. This segmentation reduces the number of separate image planes from potentially hundreds to a manageable number of depth bins, thereby compressing the file size while preserving visual quality through selective depth sampling.
Solution Approach 2:
The patent creates a simplified copy of the multi-plane image structure by generating a depth map that represents the essential depth information. Instead of storing and processing all original planes, the system uses this compressed depth representation to guide rendering and compositing operations, reducing storage requirements while maintaining image quality.
2Measurement precision
If multi-plane images with many layers are used to represent three-dimensional content, then rendering accuracy across viewpoints is improved, but computational complexity and rendering time increase
Solution Approach 1:
The patent divides the complex multi-plane rendering task into simpler sub-tasks by segmenting planes into depth bins. Each depth bin can be processed independently with simplified compositing operations, reducing the computational complexity of rendering while maintaining accuracy through the preserved depth relationships across viewpoint changes.
Solution Approach 2:
The patent changes the parameter representation from storing individual plane data to storing depth map information and associated metadata. This parameter transformation reduces the data structure complexity and enables more efficient rendering algorithms that operate on the compressed representation rather than the full multi-plane dataset.
3Adaptability or versatility
If multiple fronto-parallel planes are composited to form three-dimensional content, then viewpoint synthesis capability is enhanced, but memory requirements and processing power increase
Solution Approach 1:
The patent creates a compressed depth map copy that captures the essential geometric information needed for viewpoint synthesis. This simplified representation enables virtual viewpoint rendering with reduced computational requirements, as the depth map can be used to efficiently calculate new views without requiring all original plane data to be actively processed in memory.
Solution Approach 2:
The patent performs preliminary processing to extract and store depth information in a compressed format before rendering operations. By pre-computing and storing the depth map representation, the system prepares the data in advance for efficient viewpoint synthesis, reducing the real-time processing power needed during actual rendering and viewpoint generation.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Methods, systems, and media for generating compressed images are provided. In some embodiments, the method comprises: identifying a multi-plane image (MPI) that represents a three-dimensional image; splitting the MPI into a plurality of sub-volumes; calculating, for each sub-volume of the MPI, a depthmap; converting each depthmap to a mesh, wherein each mesh corresponds to a layer of a plurality of layers associated with a multi-depth image (MDI) to be rendered; calculating, for each layer of the plurality of layers, an image that indicates a color and a transmittance of each voxel included in the layer; storing the meshes corresponding to the plurality of layers of the MDI and the images corresponding to the plurality of layers of the MDI as the MDI; and, in response to receiving a request for the three-dimensional image from a user device, transmitting the MDI to the user device, wherein the user device is configured to render the MDI by mapping, for each layer of the MDI, the image corresponding to the layer as a texture on the mesh corresponding to the layer.