Virtual Viewpoint Image Generation Using Blur-Aware Visibility
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for generating virtual viewpoint images suffer from image quality deterioration due to the combination of images captured with cameras having different depths of field, leading to increased blurring.
Innovation Solution
A system and method that sets a visibility determination region based on image blurring characteristics to determine visibility, using camera parameters and depth images to selectively use clear regions for image generation, reducing blurring in virtual viewpoint images.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If captured images from cameras with different depths of field are combined to generate virtual viewpoint images, then more cameras can be used for image generation, but blurring increases and image quality deteriorates
Solution Approach 1:
The patent applies local quality by evaluating and selecting pixels based on their individual blurring characteristics. Instead of uniformly accepting or rejecting entire images from different cameras, the system assesses each pixel's clarity and selectively combines only the clear pixels from multiple cameras to generate the virtual viewpoint image, thereby maintaining high image quality while utilizing multiple camera sources
Solution Approach 2:
The patent changes the parameter selection criteria from simple visibility based on geometric occlusion to a quality-based selection that considers pixel clarity and blurring levels. By introducing quality metrics as selection parameters, the system can adaptively choose which camera pixels to use, resolving the contradiction between utilizing multiple cameras and maintaining image quality
2Productivity
If visibility is determined without considering blurring, then processing is simpler and faster, but image quality deteriorates due to inclusion of blurred pixels
Solution Approach 1:
The patent applies preliminary action by pre-evaluating and marking pixels as clear or blurred before the actual image generation process. This preliminary classification allows the main processing to quickly reference pre-computed quality data rather than performing complex blurring analysis during real-time rendering, thus maintaining high processing speed while ensuring quality
Solution Approach 2:
The patent introduces an intermediary quality evaluation mechanism that acts as a bridge between simple visibility determination and complex image quality assessment. This intermediary layer provides pre-computed clarity information that guides the pixel selection process, enabling both speed and quality without requiring the main processing to perform exhaustive quality analysis
Data Source
Figure 1
Figure 2A~2B
Figure 3~4
AI summary
An information processing apparatus performs first determination of determining whether a blurring state of a region in a captured image corresponding to an element of three-dimensional shape data of an object satisfies a condition and performs second determination of, in a case where the blurring state is determined to satisfy the condition, determining whether the region is visible or invisible from an image capture apparatus having captured the captured image and, in a case where the blurring state is determined not to satisfy the condition, determining that the region is invisible. The information processing apparatus generates information used in generating a virtual viewpoint image, based on a determination result by the second determination.