Spatiotemporal Depth Extraction Using Forward and Backward Prediction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional depth map extraction techniques for stereoscopic images result in temporal jittering artifacts in 3D playback due to instability of depth maps over time, especially in frames with significant texture changes.
Innovation Solution
A spatiotemporal depth extraction method incorporating forward and backward depth prediction, where the depth map of a current frame is generated based on both a previous and subsequent frame, using a combination of pixel matching, smoothness, and temporal cost functions to improve the smoothness and accuracy of depth maps.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a static depth extraction algorithm is applied to each frame pair, then the depth map extraction process is simple and fast, but temporal jittering artifacts occur due to instability of depth maps over time
Solution Approach 1:
The patent applies preliminary action by using forward depth prediction from the previous frame and backward depth prediction from the subsequent frame before finalizing the current frame's depth map. This pre-computation of predictive depth maps from temporal neighbors stabilizes the extraction process and reduces jittering artifacts while maintaining efficiency.
Solution Approach 2:
The patent implements feedback by incorporating backward depth prediction that uses the subsequent frame's depth map to refine the current frame's depth extraction. This feedback mechanism from future temporal information corrects instability and temporal jittering, improving depth map reliability without significantly increasing processing complexity.
2Reliability
If forward and backward depth prediction are used, then temporal jittering artifacts are reduced and depth map stability is improved, but the processing complexity increases
Solution Approach 1:
The patent merges forward depth prediction and backward depth prediction into a unified spatiotemporal extraction process. By combining these two predictive approaches with the current frame's static extraction, the system achieves improved stability through integrated temporal information while managing processing complexity through a cohesive algorithmic framework.
Solution Approach 2:
The patent transitions from static spatial depth extraction to spatiotemporal extraction by adding the time dimension. Forward and backward predictions leverage temporal dimensions (previous and subsequent frames) to stabilize current frame extraction, reducing jittering artifacts while the integrated approach manages the increased computational complexity.
Data Source
AI summary
A system and method for spatiotemporal depth extraction of images with forward and backward depth prediction are provided. The system and method of the present disclosure provide for acquiring a plurality of frames, generating a first depth map of a current frame in the plurality of frames based on a depth map of a previous frame in the plurality of frames, generating a second depth map of the current frame in the plurality of frames based on a depth map of a subsequent frame in the plurality of frames, and processing the first depth map and the second depth map to produce a third depth map for the current frame.