Kernel-Based Image Deformation Representation for 3D View Synthesis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional image deformation representation methods, such as image-domain warping, are inefficient for storing and transmitting 3D video content due to large data sizes and inability to handle semi-transparent and reflective objects, leading to high latency and limited support for view synthesis.
Innovation Solution
A kernel-based representation with variable support and order for image deformation, combined with Variable Scale and Differential Category Coding, allows for efficient compression and handling of semi-transparent objects by overlaying kernels with opacity and inter-coding, reducing data size and enabling real-time view synthesis.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If conventional image-domain warping with depth maps is used for view synthesis, then arbitrary view generation is enabled, but the data size and storage requirements become very large
Solution Approach 1:
The patent transforms the depth map representation from pixel-space values to a compact parametric form using polynomial functions. Instead of storing depth values for every pixel, the invention represents depth surfaces as polynomials with a small number of coefficients, fundamentally changing the parameter space from O(N) pixel values to O(K) polynomial coefficients where K << N.
Solution Approach 2:
The invention extracts and separates the essential deformation information from the full depth map. By identifying that only a small number of polynomial coefficients are needed to represent the depth surface, the method extracts these critical parameters while discarding redundant pixel-level data, achieving compression without losing view synthesis capability.
2Ease of manufacture
If conventional depth maps with single depth values per pixel are used, then storage is simplified, but semi-transparent and reflective objects cannot be handled
Solution Approach 1:
The patent extends the depth representation from scalar values to multi-layer polynomial structures. Each layer represents a different depth component or object layer, allowing the system to represent semi-transparent and reflective objects by stacking multiple polynomial depth layers with different opacity weights, thus adding a dimensional aspect to the depth representation.
Solution Approach 2:
The invention creates a composite depth representation by combining multiple polynomial layers, each representing different object layers or depth components. These layers are composited with varying opacity and transparency parameters, enabling the representation of complex materials like glass, water, and reflective surfaces while maintaining the compact polynomial form.
3Measurement precision
If high-resolution depth maps are used for accurate view synthesis, then synthesis quality improves, but transmission bandwidth and latency increase
Solution Approach 1:
The patent changes the parameter representation from high-resolution pixel-space depth values to low-resolution polynomial coefficients. This parameter transformation maintains synthesis quality because the polynomial functions can be evaluated at any resolution during view synthesis, while the transmitted data size is reduced from millions of pixel values to a handful of coefficients.
Solution Approach 2:
The invention performs preliminary computation to fit polynomial functions to the depth data during encoding. This pre-processing step extracts the essential deformation characteristics into compact polynomial forms, so that during transmission and decoding, only the lightweight polynomial parameters need to be transmitted and evaluated, avoiding the need to transmit and process full-resolution depth maps in real-time.
Data Source
AI summary
A technique for displaying 3D videos creates a representation of image deformation, such as depth maps, in terms of a function for overlaying kernels with variable support and order. By optimizing the kernel type, order and support, which are allowed to be varied across different region of the deformation, it is able to approximate the image deformations in terms of the kernel parameters. Since the number of kernel parameters is usually much smaller than that of the number of pixels, this allows a significant reduction in the storage size of the image deformation.