Autostereoscopic View Synthesis via Steerable Pyramid Filtering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current 3D content production pipelines struggle with generating multiple views required for multi-view autostereoscopic displays, as they are impractical and costly due to the need for dense camera rigs, and existing methods fail to effectively handle scenes with defocus blur, motion blur, transparent materials, and specularities, while also suffering from interperspective aliasing and viewer comfort issues.
Innovation Solution
A method and system that takes a stereoscopic stream as input, applies spatial band pass filters to synthesize additional views and perform antialiasing, allowing for real-time rendering of multi-view autostereoscopic 3D video displays without requiring changes to existing pipelines or using dense depth map reconstruction, leveraging steerable pyramid decomposition and phase-based techniques for view interpolation and filtering.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If dense camera rigs are used to capture TV-quality scenes, then view quality is improved, but device complexity and cost increase significantly
Solution Approach 1:
The patent segments the view synthesis process into multiple processing stages: depth map generation from stereo pairs, steerable pyramid decomposition of input views, and view synthesis through filtering operations. This segmentation allows complex multi-view generation to be achieved through simpler, modular processing steps rather than requiring complex physical camera rigs.
Solution Approach 2:
The patent creates synthetic copies of views through computational methods. Instead of capturing multiple views with physical cameras, the system generates additional views by synthesizing them from a limited number of input views using depth information and filtering operations, effectively copying and transforming the light field computationally.
2Adaptability or versatility
If view interpolation is used to generate additional views, then multi-view content is improved, but image quality deteriorates due to inaccurate depth and missing scene regions
Solution Approach 1:
The patent performs preliminary depth map generation and scene analysis before view interpolation. By pre-computing accurate depth information from stereo pairs and preparing steerable pyramid representations of the input views, the system ensures that subsequent view synthesis operations have the necessary information available, improving image quality in the generated views.
Solution Approach 2:
The patent replaces traditional mechanical view synthesis methods with computational filtering operations. Instead of relying on geometric warping and depth-based rendering that suffer from artifacts, the system uses steerable pyramid decomposition and filtering to synthesize views, substituting mechanical/geometric operations with signal processing techniques that preserve image quality.
3Reliability
If special filtering is applied to remove interperspective aliasing, then viewing comfort is improved, but processing complexity increases
Solution Approach 1:
The patent merges the filtering operation with the view synthesis process. The same steerable pyramid decomposition and filtering operations used to generate views are also applied to remove interperspective aliasing. By combining these functions, the system achieves both view generation and aliasing removal without requiring separate processing pipelines, reducing overall complexity.
Solution Approach 2:
The patent implements universal filtering operations that serve multiple purposes: generating synthetic views, removing aliasing artifacts, and maintaining viewing comfort. The steerable pyramid framework provides a unified approach that handles multiple tasks through the same mathematical operations, making the processing system more efficient and less complex.
4Manufacturing precision
If dense light field is used for antialiasing, then filtering quality is improved, but data requirements and processing load increase
Solution Approach 1:
The patent applies partial filtering operations through the steerable pyramid framework. Instead of requiring complete dense light field sampling for antialiasing, the system applies filtering at appropriate frequency bands and orientations to achieve sufficient aliasing removal. This partial action approach provides adequate filtering quality without the excessive data requirements of complete dense light field processing.
Data Source
AI summary
Multi-view autostereoscopic displays provide an immersive, glasses-free 3D viewing experience, but they preferably use correctly filtered content from multiple viewpoints. The filtered content, however, may not be easily obtained with current stereoscopic production pipelines. The proposed method and system takes a stereoscopic video as an input and converts it to multi-view and filtered video streams that may be used to drive multi-view autostereoscopic displays. The method combines a phase-based video magnification and an interperspective antialiasing into a single filtering process. The whole algorithm is simple and may be efficiently implemented on current GPUs to yield real-time performance. Furthermore, the ability to retarget disparity is naturally supported. The method is robust and works transparent materials, and specularities. The method provides superior results when compared to the state-of-the-art depth-based rendering methods. The method is showcased in the context of a real-time 3D videoconferencing system.