Binary Depth Map Generation via Motion Vector Projection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for generating 3D content from 2D video are computationally expensive and prone to artefacts, especially in non-static scenes with moving or deformable objects, and fail in large smooth areas and around object boundaries.
Innovation Solution
A method that estimates a binary depth map by calculating a global motion vector and block motion vectors, projecting these vectors to obtain correlation parameters, determining whether the background is static or dynamic, and computing a binary depth map to classify image parts as background or foreground using simple vector projection operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If SFM techniques are used to generate depth maps from 2D video, then depth estimation capability is improved, but computational complexity and cost increase significantly
Solution Approach 1:
The patent segments the video frame into multiple blocks and processes each block independently using motion vector analysis. This divides the complex global depth estimation problem into simpler local block-level problems, reducing overall computational complexity while maintaining depth estimation capability.
Solution Approach 2:
The patent extracts and uses only the essential motion vector information from the video stream to generate depth maps, rather than performing complete structure-from-motion analysis. This extraction approach obtains sufficient depth data with significantly reduced computational effort.
2Reliability
If SFM techniques are used to handle non-static scenes with moving objects, then depth estimation robustness is improved, but computational effort increases
Solution Approach 1:
The patent dynamically adapts the depth estimation process by analyzing motion vectors to detect moving objects and adjusting processing accordingly. The system identifies static background regions versus dynamic foreground regions and applies appropriate depth estimation strategies to each, improving robustness for non-static scenes while optimizing computational effort.
Solution Approach 2:
The patent changes processing parameters based on scene characteristics by using motion vector magnitude and direction to adjust depth map generation parameters. This allows the system to handle varying scene dynamics efficiently, maintaining robustness while adapting computational resources to actual needs.
3Manufacturing precision
If aggressive structure estimation is used to obtain continuous depth map, then depth map completeness is improved, but computational complexity increases
Solution Approach 1:
The patent applies partial action by generating depth information for only the necessary regions of the image based on motion analysis. Rather than aggressively estimating depth for every pixel throughout the entire frame, the method focuses computational effort on regions where depth information is most needed and can be most accurately determined from motion data.
Data Source
Figure 1
Figure 2~3B
Figure 4
AI summary
There is disclosed a method and apparatus for generating a depth map and 3D video. The method of generating a depth map operates on two frames in a 2D video stream. The method comprises estimating a global motion vector and block motion vectors for the first frame and projecting the block motion vectors onto the global motion vector to obtain correlation parameters. A metric is calculated for the number of static blocks in the first frame. Then the background is determined to be static or dynamic in accordance with the motion vectors, the number of static blocks and the correlation parameters. Finally, a binary depth map is computed in accordance with the motion vectors, the number of static blocks, the correlation parameters and the determination of whether the background is static or dynamic.