Dis-occlusion Reconstruction via Temporal Warping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In tele-immersive conferencing, reconstructing virtual views from a moving virtual viewpoint is challenging due to dis-occluded areas that are not visible to fixed cameras, and existing techniques fail to address the temporal aspect of this reconstruction effectively.
Innovation Solution
The temporal information integration dis-occlusion system and method use historical data from a single frontal depth color camera to reconstruct occluded areas by warping past data onto current data using a hierarchical transformation, matching points, and merging information to provide a complete view from a virtual viewpoint.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Area of stationary object
If multiple cameras are used to capture the scene from different positions, then the coverage of visible surfaces is improved, but the device complexity increases
Solution Approach 1:
The system performs preliminary action by capturing temporal history data from a single camera at multiple time points, preparing the data in advance so that dis-occluded areas can be reconstructed when needed without requiring multiple physical cameras
Solution Approach 2:
The system creates a virtual copy of the scene by warping and merging historical camera data to synthesize views from perspectives not currently captured, effectively replicating the function of multiple cameras through temporal data manipulation
2Measurement precision
If temporal information from the total history is warped onto captured information using hierarchical transformation, then the reconstruction accuracy of dis-occluded areas is improved, but the computational complexity increases
Solution Approach 1:
The warping process is segmented into hierarchical levels, dividing the complex transformation into manageable stages that process different regions or scales of the image, improving accuracy while controlling computational load
Solution Approach 2:
The system applies warping selectively to only those regions identified as dis-occluded areas rather than processing the entire image, performing partial action that reduces computational complexity while maintaining reconstruction accuracy for critical areas
Data Source
AI summary
A temporal information integration dis-occlusion system and method for using historical data to reconstruct a virtual view containing an occluded area. Embodiments of the system and method use temporal information of the scene captured previously to obtain a total history. This total history is warped onto information captured by a camera at a current time in order to help reconstruct the dis-occluded areas. The historical data (or frames) from the total history match only a portion of the frames contained in the captured information. This warping yields warped history information. Warping is performed by using one of two embodiments to match points in an estimation of the current information to points in the captured information. Next, regions of current information are split using a classifier. The warped history information and the captured information then are merged to obtain an estimate for the current information and the reconstructed virtual view.