Depth Map Generation for 2D to 3D Video Conversion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional video coding techniques struggle with effectively converting two-dimensional (2D) video to three-dimensional (3D) video, particularly in scenarios where depth maps are required for view synthesis, as they often rely on intensive computations and may not accurately capture depth cues from monoscopic sources.
Innovation Solution
The proposed techniques involve a depth map initialization process where initial depth values are assigned to video units based on motion and adjusted based on color, with the generation of an initialized depth map that can be refined further, depending on scene changes and motion levels, to produce a final depth map for 2D to 3D video conversion.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional video coding techniques are used for 2D to 3D conversion, then the conversion process can be performed, but the computational complexity becomes excessive and depth accuracy is insufficient
Solution Approach 1:
The depth map generation process is segmented into distinct stages: initialization phase (assigning initial depth values based on motion), refinement phase (adjusting depth values based on color), and finalization phase (applying scene change detection and motion level analysis). This segmentation allows each stage to focus on specific aspects of depth estimation, improving overall accuracy while managing computational load efficiently.
Solution Approach 2:
The patent applies preliminary action by performing motion-based depth initialization before color-based refinement. By pre-establishing depth values using motion information from previous frames, the system creates a preliminary depth map that captures temporal depth cues. This preliminary depth estimation is then refined using color information, reducing the computational burden of exhaustive color analysis while maintaining depth accuracy.
2Measurement precision
If intensive computations are applied to generate accurate depth maps, then depth accuracy improves, but processing time and computational resources increase significantly
Solution Approach 1:
The patent applies partial action by selectively refining depth values based on color information only for pixels that require refinement, rather than processing all pixels uniformly. The system identifies pixels where motion-based depth estimation is insufficient and applies color-based refinement only to those specific regions. This selective approach maintains depth accuracy where needed while significantly reducing overall processing time and computational resource consumption.
3Productivity
If motion-based depth initialization is used, then processing efficiency improves, but depth accuracy deteriorates for static regions
Solution Approach 1:
The patent applies local quality by differentiating depth estimation strategies based on local pixel characteristics. For pixels in motion regions, motion-based depth initialization is used to maintain processing efficiency. For pixels in static regions where motion cues are insufficient, color-based refinement is applied to improve depth accuracy. This localized adaptation of estimation strategies ensures high processing efficiency overall while maintaining depth accuracy in static regions where it matters most.
Solution Approach 2:
The system implements feedback by using color information to verify and adjust motion-based depth estimates. After initial depth values are assigned based on motion, the refinement process provides feedback by analyzing color gradients and consistency with neighboring pixels. This feedback mechanism identifies and corrects depth estimation errors in static regions, improving overall depth accuracy while maintaining the efficiency benefits of motion-based initialization.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
This disclosure describes techniques for generating depth maps for video units, such as video frames or slices video frames. The techniques may be performed by a video encoder in order to convert two-dimensional (2D) video to three-dimensional (3D) video. The techniques may alternatively be performed by a video decoder in order to convert received 2D video to 3D video. The techniques may use a combination of motion and color considerations in the depth map generation process.