Tiled Multiplane Images With Adaptive Depth Planes for View Synthesis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing multiplane images (MPIs) are highly redundant and require a large number of depth layers, making them inefficient for large images and unsuitable for mobile and AR/VR applications.
Innovation Solution
The method splits a single large MPI into small, tiled regions with adaptive depth planes, using a learning-based framework to compute per-tile MPIs for efficient view synthesis, allowing high-quality results with fewer depth planes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If a traditional multiplane image (MPI) is used to represent the scene, then complex appearance effects and soft edges can be modeled, but the representation becomes highly redundant and requires a large number of depth layers
Solution Approach 1:
The patent divides the image into multiple tiles or patches, and for each tile, only a small number of depth planes are learned to represent the local scene. This segmentation approach reduces the overall number of depth planes needed while maintaining synthesis quality in each local region.
Solution Approach 2:
The patent applies different numbers of depth planes to different tiles based on local scene complexity. Tiles with higher depth complexity use more planes, while simpler regions use fewer planes, optimizing the trade-off between quality and efficiency.
2Manufacturing precision
If a traditional multiplane image (MPI) is used, then plausible view synthesis results can be achieved, but the representation is ill-suited for large images
Solution Approach 1:
By dividing large images into smaller tiles, the patent enables scalable processing where each tile can be independently processed with a manageable number of depth planes, making the approach suitable for large-resolution images.
3Manufacturing precision
If a traditional multiplane image (MPI) is used, then depth complexity can be represented, but the computational efficiency is reduced for mobile devices and VR headsets
Solution Approach 1:
The tile-based approach with localized depth planes reduces computational complexity compared to global MPI methods, enabling real-time rendering on mobile and VR devices while maintaining depth representation accuracy.
Data Source
AI summary
A method for generating tiled multiplane images from a source image is disclosed. The method includes obtaining color and depth images. The method also includes extracting a feature map using a first neural network. The method also includes generating masks for a tile using a second neural network based on a corresponding tile of the feature map and corresponding sections of the color and depth images. The method also includes computing depths of a planes corresponding to the tile based on the masks. The method also includes generating a per-tile multiplane image for the tile based on the masks. The method also includes rendering an image using per-tile multiplane images and depths. A system for generating tiled multiplane images from a source image is also disclosed.