Virtual Viewpoint Image Nesting in 3D Space
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies do not allow users to view a virtual viewpoint image corresponding to another virtual viewpoint independently while experiencing a three-dimensional virtual space, limiting the ability to observe multiple virtual viewpoints simultaneously.
Innovation Solution
An image processing apparatus and method that generates a first virtual viewpoint image based on inputs from a device, allowing a second virtual viewpoint image to be displayed within the first virtual viewpoint image, enabling the user to view multiple virtual viewpoints in a three-dimensional virtual space.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a virtual viewpoint image is displayed on HMD displays to create a three-dimensional virtual space, then the user can experience immersion in the virtual space, but the user cannot view virtual viewpoint images from other virtual viewpoints simultaneously
Solution Approach 1:
The patent embeds a second virtual viewpoint image within the first virtual viewpoint image displayed on HMD displays. The nested virtual display is positioned at specific coordinates (x, y, z) within the three-dimensional virtual space, allowing users to view multiple viewpoints simultaneously without adding external display devices, thus resolving the contradiction between enhanced viewing capability and system complexity
Solution Approach 2:
The patent introduces a virtual display positioned in three-dimensional space within the virtual viewpoint image, rather than limiting displays to the two-dimensional HMD screen surface. This dimensional extension allows the nested virtual display to be placed at arbitrary positions and orientations within the 3D space, enabling multiple viewpoint visualization while maintaining the existing HMD display system
2Loss of information
If multiple virtual viewpoint images are displayed simultaneously, then users can observe multiple viewpoints, but the system requires processing multiple images which increases computational load
Solution Approach 1:
The patent pre-generates multiple virtual viewpoint images from the captured image data before displaying them. By performing the viewpoint generation processing in advance and storing the pre-generated images, the system reduces real-time computational requirements during actual viewing, allowing multiple viewpoints to be available while managing power consumption through optimized processing timing
Data Source
AI summary
An image processing apparatus obtains a plurality of images based on image capturing performed by a plurality of image capturing apparatuses, receives an input corresponding to operation of a first device, receives an input corresponding to operation of a second device different from the first device; and generates, based on the plurality of images obtained, a first virtual viewpoint image corresponding to a first virtual viewpoint determined based on the input corresponding to the operation of the first device, wherein a second virtual viewpoint image corresponding to a second virtual viewpoint determined based on the input received by the second receiving unit is displayed within the first virtual viewpoint image.