Layered View Synthesis With Depth-Guided Inpainting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for generating synthesized perspective view images from a single 2D image often result in visual artefacts such as striping and are not robust to multi-level occlusion, which affects the quality of 3D image rendering.
Innovation Solution
A method involving depth estimation, dilated depth map generation, inpainting mask creation, and blending map utilization to render synthesized view images, mitigating artefacts and enhancing robustness against occlusion.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If previously reported methods are used to generate synthesized perspective view images, then the process is simpler, but visual artefacts such as striping and dilation artefacts occur
Solution Approach 1:
The patent segments the image synthesis process into distinct stages: depth map generation, foreground extraction using inpainting masks, background generation, and blending. This segmentation allows each stage to be optimized independently, reducing artefacts while maintaining process manageability
Solution Approach 2:
The patent performs preliminary actions by generating depth maps and inpainting masks before the actual view synthesis. These preliminary structures (depth maps, masks) are created to identify occluded regions and guide the synthesis process, preventing artefacts before they occur
2Device complexity
If previously reported methods are used for view synthesis, then the method is less complex, but robustness to multi-level occlusion is poor
Solution Approach 1:
The patent introduces depth as an additional dimension by generating depth maps associated with each pixel. This depth information provides a third dimension beyond the 2D image plane, enabling the system to understand and handle multi-level occlusions by processing pixels in depth order
Solution Approach 2:
The patent uses inpainting masks as intermediary structures that mediate between the input image and the synthesized view. These masks identify occluded regions and guide the blending process, ensuring that multi-level occlusions are handled correctly without requiring complex direct processing
3Quantity of substance
If a single 2D image is used as input, then the data requirement is minimal, but the ability to generate accurate 3D perspective views is limited
Solution Approach 1:
The system performs self-service by automatically generating depth maps from the single 2D input image using neural networks or algorithms. Rather than requiring external depth sensing hardware, the system uses the image data itself to infer depth information, maintaining minimal data requirements while improving accuracy
Data Source
AI summary
A method of computer-implemented synthesized view image generation and a synthesized view image generation system provide layered view synthesis. The method includes receiving an input image having a plurality of pixels having color values; generating a dilated depth map by dilating a depth map associated with the input image, the depth map with depth values respectively associated with each pixel in the input image; determining an inpainting mask using the dilated depth map; performing an inpainting operation based on the inpainting mask and the input image to generate a background image; and rendering a synthesized view image using the background image, the input image, and the dilated depth map.