Virtual Viewpoint Image Generation via Occlusion-Aware Pixel Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing techniques for generating virtual viewpoint images do not adequately account for occlusion, leading to reduced quality due to the selection of cameras based on positional relationships, which can result in incorrect pixel values for the virtual viewpoint image.
Innovation Solution
An image processing apparatus that selects an image where a specific position in the target object is not occluded by another object from multiple images captured by cameras, determines the corresponding pixel value, and generates the virtual viewpoint image based on this selection to prevent occlusion-related quality reduction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a camera is selected based on positional relationship between virtual viewpoint and camera, then the image generation process is simplified, but occlusion may occur leading to reduced image quality
Solution Approach 1:
The system performs preliminary occlusion determination before selecting the camera for pixel value determination. By checking whether the target object is occluded from each camera's viewpoint in advance, the system可以避免 using images with occlusion, thereby maintaining image quality while still using a relatively simple selection process.
Solution Approach 2:
The invention introduces an intermediary occlusion determination step between the camera selection based on positional relationship and the actual pixel value determination. This intermediary check acts as a filter to eliminate cameras that would produce occluded images, thus resolving the contradiction between process simplicity and image quality.
2Loss of information
If pixel values are determined from all captured images, then complete color information is obtained, but occlusion causes incorrect pixel values to be used
Solution Approach 1:
Before determining pixel values from captured images, the system performs preliminary occlusion determination to identify which cameras have unoccluded views of the target object. This preliminary check ensures that only reliable, non-occluded images are used for pixel value determination, maintaining both information completeness and accuracy.
Solution Approach 2:
The system uses feedback from the occlusion determination result to guide the pixel value determination process. Based on whether occlusion is detected, the system selectively chooses which camera images to use, creating a feedback loop that ensures accurate pixel values are obtained while avoiding occluded regions.
3Productivity
If camera selection is based solely on positional relationship, then processing speed is improved, but image quality is reduced due to occlusion
Solution Approach 1:
The system performs a quick preliminary occlusion determination that does not significantly slow down processing. By checking occlusion status before final pixel value determination, the system maintains high processing speed while ensuring image quality is not compromised by occluded views.
Solution Approach 2:
The invention performs a partial check (occlusion determination) rather than a complete analysis of all possible factors. This partial action approach maintains processing speed by only checking the necessary occlusion condition, while still achieving the goal of avoiding occluded images for better quality.
Data Source
AI summary
A reduction in quality of a virtual viewpoint image is suppressed. An image processing apparatus specifies an image in which a specific position in a target object is not occluded by another object from among a plurality of images obtained by the plurality of imaging apparatuses based on a pixel value of the plurality of images; determines, based on the specified image, a value of a pixel corresponding to the specific position in the virtual viewpoint image to be generated based on the plurality of images; and generates the virtual viewpoint image including the target object based on the determined value of the pixel.