Virtual Viewpoint Image Compositing With Selective Foreground Output
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image processing systems face challenges in generating high-quality virtual viewpoint images due to increased data volume and processing load when using multiple imaging devices with high resolution, which can result in reduced image quality and resolution, especially when the virtual camera is positioned close to the object.
Innovation Solution
A system utilizing a combination of wide-angle and zoom cameras to capture images, where the wide-angle cameras capture images from a distance and zoom cameras capture close-up images, with a server processing these images to generate virtual viewpoint images and combined images, allowing for selective output based on the virtual camera's position relative to the object.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If the number of imaging devices and resolution are increased to generate higher quality virtual viewpoint images, then image quality is improved, but data amount and processing load increase
Solution Approach 1:
The patent divides the captured image into foreground and background portions, processing and transmitting only the foreground image data to the server. This segmentation reduces the total data amount while maintaining image quality for the important foreground objects.
Solution Approach 2:
The patent applies different processing strategies to different parts of the image: foreground objects are extracted and processed with high detail, while the background is handled differently or with reduced detail, optimizing quality where it matters most while reducing overall data load.
2Manufacturing precision
If the number of imaging devices and resolution are increased to generate higher quality virtual viewpoint images, then image quality is improved, but processing load increases
Solution Approach 1:
The patent extracts only the foreground objects from the captured images before transmitting to the server. This extraction reduces the processing load on the server by eliminating the need to process large amounts of background image data, while still maintaining high image quality for the foreground subjects.
3Quantity of substance
If foreground extraction is performed to reduce data amount, then data transmission load is reduced, but image quality may be insufficient for certain viewing scenarios
Solution Approach 1:
The patent dynamically selects between different image generation approaches based on the virtual camera position. When the virtual camera is close to the object, it uses the foreground extraction method; when the virtual camera is far away, it uses the full image method, thus adapting to different quality requirements.
Solution Approach 2:
The patent changes the processing parameters (foreground extraction vs. full image processing) based on the viewing distance and scenario requirements, optimizing both data efficiency and image quality for different conditions.
Data Source
AI summary
An object is to make it possible to provide an appropriate image. A background combined image that takes, as a foreground, an image corresponding to a virtual viewpoint that is set in an image capturing space and a background combined image that takes, as a foreground, part of a captured image captured from a specific viewpoint among a plurality of captured images are generated. Then, the configuration is designed so that it is possible to selectively output one of the background combined images.