Stereo-to-Multi-View Conversion via Wavelet Re-projection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The adoption of stereoscopic 3D technology in home settings is limited due to the lack of multi-view content for automultiscopic displays, despite the availability of stereoscopic content, and existing methods struggle with real-time conversion and hardware implementation.
Innovation Solution
An Eulerian-Lagrangian stereo-to-multi-view conversion method and system that decomposes stereoscopic input signals using steerable pyramid decomposition and wavelet re-projection, enabling real-time expansion of stereoscopic content to high-quality multi-view format, suitable for hardware implementation on GPUs and FPGAs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If standard depth image-based rendering methods are used, then the conversion process is simple, but it is limited to small disparities and cannot handle large disparities effectively
Solution Approach 1:
The patent segments the stereoscopic input signal using steerable pyramid decomposition, breaking down the image into multiple frequency bands and directional components. This segmentation allows the system to process different frequency components separately, enabling effective handling of large disparities while maintaining computational efficiency through hierarchical processing.
Solution Approach 2:
The patent transforms the problem from standard 2D image processing to a multi-dimensional frequency-domain representation using wavelet decomposition. By operating in the frequency domain and using 3D wavelet transforms, the system can handle large disparities more effectively while maintaining real-time performance through efficient frequency-domain operations.
2Productivity
If real-time conversion is implemented, then the processing speed is fast, but the quality of multi-view content synthesis may be compromised
Solution Approach 1:
The patent performs preliminary decomposition of the stereoscopic input into frequency bands using steerable pyramid decomposition before view synthesis. This pre-processing step organizes the input data in a way that facilitates efficient real-time processing while preserving quality, as the decomposition separates frequency components that can be processed independently and recombined.
Solution Approach 2:
The patent replaces traditional spatial-domain processing with frequency-domain operations using 3D wavelet transforms. This substitution enables real-time processing by exploiting the efficiency of frequency-domain computations, while the wavelet re-projection technique maintains high synthesis quality through accurate frequency-component manipulation.
3Adaptability or versatility
If automultiscopic displays are used, then glasses-free 3D experience for multiple viewers is provided, but there is practically no multi-view content available
Solution Approach 1:
The patent enables automultiscopic displays to generate their own multi-view content from standard stereoscopic inputs using real-time conversion. The system performs self-service by converting readily available stereoscopic content into multi-view format, eliminating the need for separate multi-view content production while fully utilizing the display's capabilities.
Solution Approach 2:
The patent changes the viewing parameters by synthesizing multiple views from a single stereoscopic input. By adjusting disparity values and generating intermediate views through wavelet re-projection, the system transforms two-view stereoscopic content into multi-view content suitable for automultiscopic displays, effectively expanding content availability through parameter manipulation.
4Adaptability or versatility
If wavelet re-projection is used to handle large disparities, then disparity handling capability is improved, but computational complexity increases
Solution Approach 1:
The patent segments the wavelet transform into multiple frequency bands and directional components, allowing disparity handling to be performed on individual frequency components rather than the entire image at once. This segmentation reduces computational complexity by breaking down the complex operation into manageable, parallelizable frequency-band processing steps.
Solution Approach 2:
The patent moves wavelet re-projection operations to the frequency domain, transforming complex spatial-domain disparity handling into more efficient frequency-domain operations. This dimensional change enables better handling of large disparities while reducing computational complexity through the mathematical properties of frequency-domain transforms.
Data Source
AI summary
A method and system of converting stereo video content to multi-view video content combines an Eulerian approach with a Lagrangian approach. The method comprises generating a disparity map for each of the left and right views of a received stereoscopic frame. For each corresponding pair of left and right scanlines of the received stereoscopic frame, the method further comprises decomposing the left and right scanlines into a left sum of wavelets or other basis functions, and a right sum wavelets or other basis functions. The method further comprises establishing an initial disparity correspondence between left wavelets and right wavelets based on the generated disparity maps, and refining the initial disparity between the left wavelet and the right wavelet using a phase difference between the corresponding wavelets. The method further comprises reconstructing at least one novel view based on the left and right wavelets.