Virtual Viewpoint Image Rendering Using Selective Multi-Camera Resolution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing techniques for generating virtual viewpoint images fail to adequately utilize high-resolution images, leading to degraded image quality due to the combination of images with high line-of-sight matching degrees and low effective resolutions.
Innovation Solution
An image processing apparatus that specifies the relationship between a virtual viewpoint and image capturing directions, selects image capturing apparatuses based on resolution and effectiveness, and determines pixel values using selected images to prioritize high-resolution and effective images for rendering.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If multi-viewpoint images are combined based on matching degree in line-of-sight direction, then the virtual viewpoint image can be generated, but the image quality is degraded due to insufficient utilization of high-resolution images
Solution Approach 1:
The patent applies local quality by differentiating the selection criteria for different types of images. High-resolution images are selected based on effective resolution, while images with high matching degrees are selected based on line-of-sight alignment. This allows each image type to contribute its strengths to specific regions of the virtual viewpoint image, preventing degradation from mixing incompatible image qualities.
Solution Approach 2:
The patent segments the image selection process into two distinct stages: first selecting images based on effective resolution, then selecting additional images based on matching degree. This segmentation ensures that high-resolution images are prioritized and not diluted by lower-resolution images, while still incorporating the benefits of multiple viewpoints where appropriate.
2Adaptability or versatility
If captured images with high matching degree and high effective resolution are combined without distinction, then more viewpoint information is utilized, but the high-resolution image is not used sufficiently and image quality is degraded
Solution Approach 1:
The patent performs preliminary selection of images based on effective resolution before combining images based on matching degree. This preliminary action ensures that high-resolution images are identified and prioritized first, preventing them from being diluted by lower-resolution images in the combination process, while still allowing comprehensive viewpoint information to be utilized.
3Manufacturing precision
If images with high effective resolution are preferentially used, then image quality is improved, but the matching degree in line-of-sight direction is not considered
Solution Approach 1:
The patent employs a dynamic, two-stage selection process that adapts to different image characteristics. First, images are selected based on effective resolution to ensure high quality. Then, additional images are dynamically selected based on matching degree to ensure proper geometric alignment. This dynamic approach maintains both image quality and reliability by considering both criteria in sequence rather than using a single static criterion.
Data Source
AI summary
The image processing apparatus that generates a virtual viewpoint image based on images obtained by capturing an image capturing area from directions different from one another by a plurality of image capturing apparatuses includes: a specification unit configured to specify a relationship between a sight direction from a virtual viewpoint corresponding to the virtual viewpoint image and an image capturing direction of an image capturing apparatus; a selection unit configured to select one or more image capturing apparatuses based on the specified relationship and a resolution relating to image capturing by the image capturing apparatus; and a pixel value determination unit configured to determine a pixel value of the virtual viewpoint image by using a pixel value or pixel values of one or more images obtained based on image capturing by the selected one or more image capturing apparatuses.