Virtual Viewpoint Image Generation with Surface Reflection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing techniques for generating virtual viewpoint images do not effectively incorporate the reflection of a subject on background surfaces, such as floors or walls, which is crucial for creating a realistic visual experience.
Innovation Solution
A generation apparatus that acquires multiple images from various viewpoints, identifies a reflection surface within the three-dimensional space, and generates a virtual viewpoint image that simulates the reflection of objects on that surface, using a combination of image processing and three-dimensional shape data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a virtual viewpoint image is generated using only three-dimensional shape data and background images, then the generation process is simple, but the background lacks realism and does not show reflections of subjects
Solution Approach 1:
The image generation process is divided into multiple stages: first generating a virtual viewpoint image from 3D shape data, then generating a reflection viewpoint image separately, and finally combining them. This segmentation allows each stage to focus on specific aspects (realism vs. complexity management) without overwhelming the system.
Solution Approach 2:
A reflection viewpoint image is introduced as an intermediary element that bridges the gap between the simple 3D-generated image and the desired realistic reflection effect. This intermediary contains the reflected subject information that is then overlaid onto the background, achieving realism without directly complicating the core generation process.
2Reliability
If multiple images are captured and processed to show reflections, then the visual realism is improved, but the processing time and computational load increase
Solution Approach 1:
The reflection viewpoint image is generated in advance as a separate preprocessing step before being combined with the virtual viewpoint image. This preliminary action separates the computationally intensive reflection calculation from the final composition, allowing optimization of each stage independently and reducing overall processing time.
Solution Approach 2:
Instead of processing multiple original captured images to extract reflection information, the system creates a synthesized reflection viewpoint image that copies and transforms the necessary reflection data. This copying approach reduces computational load compared to processing multiple high-resolution source images.
3Reliability
If reflection information is added to the virtual viewpoint image, then the immersion in virtual environment is enhanced, but the device complexity increases
Solution Approach 1:
The virtual viewpoint image and reflection viewpoint image are merged through overlay combination to create the final output. This merging approach integrates reflection information into the existing virtual environment framework without requiring a complete redesign of the system architecture, thus enhancing immersion while controlling complexity growth.
Data Source
AI summary
A generation apparatus includes following units. A first acquisition unit acquires a first virtual viewpoint image generated based on a plurality of images captured by a plurality of image capturing apparatuses, and a first virtual viewpoint in a three-dimensional space including an image capturing space captured by the plurality of image capturing apparatuses. A second acquisition unit acquires a second virtual viewpoint image generated based on the plurality of captured images, and a second virtual viewpoint arranged with respect to a predetermined surface in the three-dimensional space and the first virtual viewpoint. A generation unit generates a third virtual viewpoint image corresponding to the first virtual viewpoint and including an image indicating a state where an object in the image capturing space is reflected on the predetermined surface, based on the first virtual viewpoint image, and the second virtual viewpoint image.