3D Video Hole-Filling Using Depth-Aware Bilateral Filtering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current 3D video systems face challenges in producing high-quality images due to the presence of hole-regions in synthesized intermediate virtual viewpoint images, which are difficult to fill effectively, leading to deteriorated image quality.
Innovation Solution
A method is introduced to fill hole-regions in virtual viewpoint images using depth values and color values of adjacent pixels, where the smallest depth value is selected from adjacent pixels and color values are calculated using a bilateral filter with weights based on depth differences and distances, to reconstruct natural and high-quality 3D images.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If intermediate virtual viewpoint image is synthesized using obtained viewpoint images, then a realistic 3D image can be displayed from any viewpoint, but hole-regions are created that deteriorate image quality
Solution Approach 1:
The patent uses depth maps as intermediary information to guide the hole-filling process. By introducing depth data as a mediator between the synthesized image and the filling operation, the system can accurately identify hole regions and fill them with appropriate pixel values from reference images, thereby maintaining both viewpoint flexibility and image quality
Solution Approach 2:
The patent performs preliminary actions by first synthesizing the intermediate virtual viewpoint image to identify hole-regions before displaying the final 3D image. This preliminary identification and filling of holes ensures that the displayed image maintains high quality while preserving the adaptability of displaying from any viewpoint
2Ease of manufacture
If traditional image inpainting or boundary noise removal is used to fill hole-regions, then processing can be performed, but image quality deteriorates due to unnatural filling
Solution Approach 1:
The patent changes the parameters used for hole-filling by incorporating depth information and using bilateral filtering with depth-aware weights. Instead of using traditional inpainting methods that only consider color and texture, this approach modifies the filtering parameters to include depth-based weighting, which enables more accurate and natural hole-region filling while maintaining processing feasibility
Solution Approach 2:
The patent applies local quality by using bilateral filtering that adapts to local depth characteristics. The filtering process considers local depth variations and applies different weights to different regions, ensuring that hole-filling maintains local image quality and natural appearance rather than applying a uniform processing approach
Data Source
AI summary
Method for filling a hole-region and a three-dimensional (3D) video system using the same. In order to synthesize an image, a plurality of images each having a different viewpoint may be obtained. By using depth maps of the obtained plurality of images, an intermediate virtual viewpoint image may be synthesized. After synthesizing, a hole-region in the intermediate virtual viewpoint image may be filled using a color image of the intermediate virtual viewpoint image, a mask image indicating the hole-region in the intermediate virtual viewpoint image, and a depth image of the intermediate virtual viewpoint image.