2D to 3D Image Conversion Using Depth Map Infilling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current 2D to 3D image conversion technologies face challenges in creating visually compelling stereoscopic images when focal axes of stereo cameras are not parallel and in handling occluded image portions, leading to inconsistencies and noise between frames.
Innovation Solution
The method involves projecting a 2D image onto computer-generated geometry, rendering a complementary 2D image from a different camera location, and using depth values to identify and fill gaps in image information, employing techniques like infilling and blurring to create a consistent stereoscopic image pair.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If computer-generated geometry and depth values are used to project 2D images onto 3D space, then the flexibility and speed of 2D to 3D conversion is enhanced, but inconsistencies and noise between frames occur due to occluded image portions and non-parallel camera axes
Solution Approach 1:
The system performs preliminary actions by pre-calculating depth values for all pixels before rendering the stereoscopic view. Depth maps are generated in advance, identifying which pixels will be occluded in the target view. This allows the system to proactively handle occlusions by preparing infill data from alternative sources, thereby maintaining image consistency while enabling fast conversion without frame-by-frame processing delays
Solution Approach 2:
Depth maps serve as an intermediary data structure between the source 2D image and the target stereoscopic image. The depth information acts as a mediator that predicts occlusions and guides the selection of infill strategies. By introducing this intermediate representation, the system can efficiently determine which pixels need infilling and from where, resolving the contradiction between fast conversion and consistent output
2Manufacturing precision
If infilling techniques are applied to fill gaps from occluded regions, then visual quality is improved, but processing complexity increases
Solution Approach 1:
The system applies local quality by treating different regions of the image differently based on their occlusion characteristics. Only pixels identified as occluded in the depth map undergo infilling operations, while unoccluded pixels are directly transferred. This localized approach maintains high image quality where needed while avoiding unnecessary processing complexity in regions that don't require infilling
Solution Approach 2:
The image processing is segmented into distinct stages: depth map generation, occlusion identification, infill source selection, and actual infilling. This segmentation allows each module to be optimized independently and enables parallel processing of different image regions, reducing overall processing complexity while maintaining comprehensive image quality
3Adaptability or versatility
If stereoscopic image pair is created with non-parallel camera axes, then flexibility in camera positioning is improved, but noise and jitter between frames increase
Solution Approach 1:
The system performs preliminary geometric transformations and coordinate system alignments before rendering images from non-parallel camera positions. By pre-calculating transformation matrices and establishing consistent reference frames in advance, the system maintains frame stability even when accommodating flexible camera positioning, preventing noise and jitter in the final stereoscopic output
Data Source
AI summary
A method of creating a complementary stereoscopic image pair is described. The method includes receiving a first 2D image comprising image data, where the first 2D image is captured from a first camera location. The method also includes projecting at least a portion of the first 2D image onto computer-generated geometry. The image data has depth values associated with the computer-generated geometry. The system includes rendering, using the computer-generated geometry and a second camera location that differs from the first camera location, a second 2-D image that is stereoscopically complementary to the first 2-D image, and infilling image data that is absent from the second 2-D image.