Multi-View Image Hole Rendering via Disparity Priority
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Generating and transmitting multi-view images using a limited number of input views is costly and inefficient, as it requires high-capacity storage, efficient transmission lines, and high bandwidth, and often results in holes in the final image due to occlusion regions hidden by foreground objects.
Innovation Solution
An image processing method and apparatus that generates an output view and disparity information using at least one input view, assigns priorities to hole pixels based on disparity information, and applies hole rendering in descending order of priority to fill these holes, utilizing an output view generating unit, priority generating unit, and hole rendering unit.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If multi-view images are generated using a limited number of input views, then transmission cost and storage capacity requirements are reduced, but hole regions appear in the output image due to occlusion
Solution Approach 1:
The patent introduces disparity information as an intermediary element that mediates between the limited input views and the output view generation. By using disparity maps to guide the view synthesis process, the system can accurately determine which regions are occluded and should be filled with hole rendering, thus maintaining image completeness while using fewer input views
Solution Approach 2:
The patent applies preliminary action by performing hole rendering before final image composition. The system identifies occlusion regions in advance using disparity information and pre-fills these regions with appropriate background content before generating the final multi-view image, preventing holes from appearing in the output
2Reliability
If hole rendering is applied to all hole pixels, then image quality is improved, but processing time and computational complexity increase
Solution Approach 1:
The patent applies local quality by differentiating between different types of hole pixels and applying hole rendering selectively. The system classifies hole pixels into foreground and background categories, and applies different rendering strategies to each type. Background hole pixels are filled using background modeling, while foreground hole pixels are handled differently, thus improving image quality without uniformly processing all pixels
Solution Approach 2:
The patent changes parameters by introducing priority values for different hole pixels based on their position and type. The system assigns different priorities to background hole pixels versus foreground hole pixels, and processes them in order of priority. This parameter-based differentiation allows the system to focus computational resources on the most important regions, reducing overall processing time while maintaining quality
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method and apparatus for processing a multi-view image are provided. A priority may be assigned to each hole pixel in a hole region generated when an output view is generated. The priority of each hole pixel may be generated by combining a structure priority, a confidence priority, and a disparity priority. Hole rendering may be applied to a target patch including a hole pixel having a highest priority. The hole pixel may be restored by searching for a source patch most similar to a background of the target patch, and copying a pixel in the found source patch into a hole pixel of the target patch.