Multiplane Image Tiling with Orientation for 3D View Synthesis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Volumetric video capture generates large amounts of data, making management and storage expensive and challenging, while existing techniques struggle with view synthesis from non-standard camera positions and restrict content to flat planes, leading to data inefficiencies.
Innovation Solution
The method involves decomposing multi-plane images into oriented tiles with added orientation information, reducing the number of slices and tiles by incorporating depth accuracy and tile orientation, allowing for faster processing and reduced data storage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If volumetric video capture is used to achieve view synthesis from any angle, then immersive experience quality is improved, but data management and storage complexity increases
Solution Approach 1:
The patent segments the 3D scene into multiple depth slices arranged in a stacked representation, where each slice contains 2D image data at a specific depth plane. This segmentation allows the system to process and store volumetric data in manageable chunks while maintaining the ability to synthesize views from any camera position, thereby reducing data management complexity without sacrificing view synthesis capability.
Solution Approach 2:
The patent introduces a depth dimension by organizing image slices along the Z-axis to create a 3D stacked representation. This dimensional transformation enables efficient storage of volumetric data by leveraging depth information, allowing view synthesis from arbitrary positions while reducing the computational and storage complexity compared to traditional volumetric methods.
2Adaptability or versatility
If traditional MPI representation is used to store 3D scene data, then view synthesis is possible, but the number of slices and tiles increases data storage requirements
Solution Approach 1:
The patent merges multiple depth slices into a single stacked representation structure, where slices are arranged and processed together as a unified 3D volume. This merging reduces the total number of independent slices and tiles that need to be stored and managed separately, thereby decreasing data storage requirements while preserving the ability to synthesize views from any angle.
Solution Approach 2:
By organizing slices along the depth dimension and processing them as a stacked 3D representation, the patent efficiently packs data in the depth-Z dimension. This dimensional organization reduces redundancy and minimizes the total number of tiles required compared to traditional 2D MPI representations, thus reducing storage volume while maintaining view synthesis capability.
3Manufacturing precision
If more slices and tiles are used to maintain view synthesis quality, then image quality is improved, but processing time increases
Solution Approach 1:
The patent segments the volumetric data into a stacked representation of depth slices, allowing selective processing of only the relevant slices needed for a particular view synthesis task. This segmentation enables the system to maintain high view synthesis quality by including necessary depth information while reducing processing time by excluding irrelevant slices and tiles from the processing pipeline.
Solution Approach 2:
By organizing data in a 3D stacked representation with explicit depth ordering, the patent enables efficient culling and selective processing of slices based on the virtual camera position. This dimensional organization allows the system to maintain high view synthesis quality by processing only the necessary depth range, thereby reducing overall processing time compared to traditional methods that must process all tiles uniformly.
Data Source
AI summary
A device, an apparatus and associated methods are provided. In one embodiment, the method comprises obtaining a multi-plane image (MPI) representation of a three dimensional (3D) scene. The MPI representation includes a plurality of slices of content from the 3D scene, each slice corresponding to a different depth relative to a position of a first virtual camera. Each slice is decomposed into regular tiles; and the orientation of each tile is determined.