2D to 3D Image Conversion via Depth Map Correction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing 2D images and animations cannot be easily converted into 3D formats, limiting their enjoyment in a three-dimensional manner.
Innovation Solution
A method that estimates depths in 2D images and animations to generate depth maps, corrects these estimates, creates 3D models by connecting pixels with small depth differences, and uses virtual cameras to generate 3D images and animations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If depth estimation is performed on 2D images to generate 3D models, then 3D viewing experience is enabled, but depth accuracy deteriorates due to inherent ambiguity in monocular depth estimation
Solution Approach 1:
The patent combines multiple depth maps from different virtual camera viewpoints and merges them to generate a final depth map. This merging process leverages the complementary information from different angles to improve overall depth estimation accuracy while maintaining the ability to generate 3D views from 2D images.
Solution Approach 2:
The patent introduces virtual camera depth maps as intermediary representations. These intermediate depth maps from synthesized viewpoints serve as mediators that help resolve the ambiguity in direct monocular depth estimation by providing additional geometric constraints from multiple perspectives.
2Adaptability or versatility
If multiple virtual camera viewpoints are synthesized to improve 3D viewing experience, then viewing flexibility is enhanced, but computational complexity increases
Solution Approach 1:
The patent performs preliminary depth estimation and 3D model construction before synthesizing multiple viewpoint images. By pre-computing the depth structure and 3D geometry, the system reduces the computational burden during runtime when generating different virtual camera viewpoints, as the heavy lifting of depth estimation is already done.
Solution Approach 2:
The patent creates copies of the 3D model from different virtual camera perspectives rather than processing original multi-view images. These synthesized viewpoint copies allow flexible viewing without requiring actual multiple camera hardware, reducing physical and computational complexity while maintaining viewing flexibility.
3Manufacturing precision
If depth correction is performed to improve depth accuracy, then 3D model quality is enhanced, but processing time increases
Solution Approach 1:
The patent applies depth correction selectively to regions where it is most needed, such as boundaries between objects or areas with high depth variation. Rather than uniformly processing the entire image, the correction is applied partially to critical regions, improving 3D model quality while reducing overall processing time compared to full-image correction.
Data Source
AI summary
For transforming a 2D image into a 3D image, or for transforming a 2D animation into a 3D animation, depths of the 2D image or the 2D animation are required and corrected by enhancing edges. Moreover, a 3D model is generated according to a corrected 2D depth map, and the 3D model is then filmed by virtual cameras to generate the 3D image or the 3D animation. While generating the 3D animation, key frames are selected for reducing processing time, and depths of unselected frames between the key frames are calculated by using interpolation.