Disocclusion Filling in Virtual View Synthesis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current depth-image-based rendering (DIBR) methods for autostereoscopic displays face challenges in filling disocclusions (exposed areas) effectively, especially with larger baselines, leading to visual artifacts and reduced depth impression, as they primarily support small baselines and ignore temporal correlations in image synthesis.
Innovation Solution
A temporally consistent occlusion filling scheme using texture synthesis, which involves identifying background and foreground regions, updating background sprites, and refining disoccluded areas through patch-based texture synthesis, ensuring visually plausible results across larger baselines.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If depth-image-based rendering (DIBR) is used to synthesize virtual views, then novel realistic images at different view perspectives can be generated, but disocclusions (exposed areas) appear in the synthesized view that require filling
Solution Approach 1:
The patent applies preliminary action by pre-processing the depth map through morphological operations (dilation and erosion) before view synthesis. This preprocessing fills potential disocclusion areas in advance, preventing exposed regions from appearing in the synthesized virtual views. The depth map is modified proactively rather than reactively filling disocclusions after they appear.
Solution Approach 2:
The patent uses copying by transferring background information from reference views to fill disocclusion areas in synthesized views. Background pixels are identified in reference views and copied into the exposed regions of virtual views, replacing the need for complex real-time synthesis in those areas.
2Reliability
If existing disocclusion filling methods are used, then small baseline rendering is supported, but the methods fail to provide satisfactory results for larger baselines
Solution Approach 1:
The patent applies parameter changes by adapting the disocclusion filling strategy based on the baseline magnitude. For small baselines, traditional interpolation methods are used, while for large baselines, the system switches to background copying and morphological operations on the depth map. This dynamic adjustment of processing parameters enables reliable filling across the full range of baseline values.
3Ease of manufacture
If frame-by-frame rendering is used, then processing is simpler, but temporal correlation of the filled area is ignored leading to reduced quality
Solution Approach 1:
The patent merges temporal information from multiple frames by accumulating background pixels across time. Instead of processing each frame independently, the system combines background information from historical frames with current frame data, creating a temporally consistent background model that improves filling quality while maintaining processing efficiency.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
In accordance with a first aspect, filling disocclusions in a virtual view video comprises identifying a background portion of a first virtual view image of the virtual view video based on a depth/disparity map associated with a first virtual view image, updating a background sprite image based on the background portion of a first virtual view image, and filling a disoccluded area of a second virtual view image of the virtual view video based on a corresponding portion of the background sprite image. According to a second aspect, filling a disoccluded area in a virtual view image is performed by patch-based texture synthesis being responsive to a depth/disparity map associated with a virtual view image.