Virtual Viewpoint Imaging With Foreground-Background Switching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing techniques for generating virtual viewpoint images face challenges in providing images of appropriate quality for different viewing scenarios and positions, often requiring increased data processing loads and resources without ensuring optimal image resolution.
Innovation Solution
A system utilizing a combination of wide-angle and zoom cameras to capture images, with a server processing these images to generate high-quality virtual viewpoint images by selectively using foreground and background models, and switching between different image outputs based on viewer position and desired image quality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If the number of imaging devices is increased to capture multiple objects and viewpoints, then the image quality and coverage are improved, but the amount of data and processing load increase significantly
Solution Approach 1:
The patent divides the captured image into a foreground region (containing the main object) and a background region. By processing and transmitting only the foreground extraction results along with the corresponding background image, the system reduces the amount of data that needs to be processed and transmitted, thereby lowering the processing load while maintaining the ability to generate high-quality virtual viewpoint images.
Solution Approach 2:
The patent extracts the foreground object from the captured image and processes it separately to generate three-dimensional shape data. This extraction allows the system to focus computational resources only on the relevant portions of the image, reducing the overall processing load while maintaining the ability to generate high-quality virtual viewpoint images from multiple perspectives.
2Quantity of substance
If foreground extraction is performed to reduce data amount, then data transmission is reduced, but image quality for close-up virtual viewpoints deteriorates
Solution Approach 1:
The patent dynamically adjusts the processing method based on the virtual camera's position relative to the object. When the virtual camera is at a close distance, the system uses the extracted foreground object with high-resolution texture mapping to maintain image quality. When the virtual camera is at a distance, the system uses the background image directly. This dynamic switching allows the system to maintain appropriate image quality across different viewing distances while reducing data transmission.
Solution Approach 2:
The patent changes the processing parameters based on the virtual camera's position. For close-up views, the system applies high-resolution texture mapping to the extracted three-dimensional object. For distant views, the system uses the background image directly. This parameter adjustment ensures that image quality is optimized for each viewing condition while minimizing data transmission requirements.
3Manufacturing precision
If high-resolution captured images are used to generate virtual viewpoint images, then image quality is improved, but processing time and resource requirements increase
Solution Approach 1:
The patent extracts only the necessary foreground information from the captured image and processes it to generate three-dimensional shape data. By focusing processing only on the foreground region rather than the entire image, the system reduces processing time and resource requirements while maintaining high image quality for the virtual viewpoint generation.
Solution Approach 2:
The patent performs preliminary processing by extracting the foreground object and generating three-dimensional shape data in advance. This preliminary action allows the system to prepare the necessary data structures before actual rendering is required, reducing the processing time needed during virtual viewpoint generation while maintaining high image quality.
Data Source
Figure 1
Figure 2A~2B
Figure 3
AI summary
An object is to make it possible to provide an appropriate image. A background combined image that takes, as a foreground, an image corresponding to a virtual viewpoint that is set in an image capturing space and a background combined image that takes, as a foreground, part of a captured image captured from a specific viewpoint among a plurality of captured images are generated (202, 206). Then, the configuration is designed so that it is possible to selectively output one of the background combined images (210).