Virtual Viewpoint Image Processing With Selective Super-Resolution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for generating virtual viewpoint images face a trade-off between image quality and the number of imaging apparatuses, leading to increased processing load as the number of virtual viewpoints increases.
Innovation Solution
An image processing apparatus that enhances image quality by increasing the number of pixels using super-resolution techniques, while reducing processing load by selectively processing images close to the virtual camera position and optimizing virtual camera orientation and position settings.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If super-resolution technique is applied to all captured images to enhance image quality, then image quality is improved, but processing load increases significantly
Solution Approach 1:
The patent applies super-resolution processing selectively only to images that are spatially close to the virtual camera position, rather than processing all captured images uniformly. This local quality approach enhances image quality where it matters most (near the viewpoint) while avoiding unnecessary processing of distant images, thereby resolving the contradiction between image quality and processing load.
Solution Approach 2:
The patent performs super-resolution processing on only a subset of captured images (those within a certain distance threshold from the virtual camera), rather than processing the complete set. This partial action approach maintains sufficient image quality for the virtual viewpoint while significantly reducing the processing burden compared to exhaustive processing of all images.
2Manufacturing precision
If the number of imaging apparatuses is increased to improve virtual viewpoint image quality, then image quality is improved, but system complexity and cost increase
Solution Approach 1:
The patent changes the parameter of image resolution through super-resolution processing applied to existing images, rather than changing the physical configuration by adding more imaging apparatuses. This parameter-based approach (changing resolution via processing) achieves improved virtual viewpoint image quality while avoiding the complexity and cost of expanding the imaging apparatus array.
3Manufacturing precision
If super-resolution processing is applied to increase the number of pixels, then image quality is improved, but processing time increases
Solution Approach 1:
The patent applies super-resolution processing only to images located in the local region near the virtual camera position, rather than processing all images globally. This local quality strategy increases the number of pixels and improves image quality where it is most needed for the virtual viewpoint, while minimizing processing time by excluding distant images from the computationally intensive super-resolution operation.
Data Source
AI summary
An imaging processing apparatus includes one or more memories storing instructions, and one or more processors executing the instructions to acquire a first image captured by an imaging apparatus, generate a second image by performing processing of increasing the number of pixels with respect to the first image, and set information indicating a position and an orientation of a virtual camera corresponding to the second image.