Virtual Viewpoint Determination Using Subject and Interest Position
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems for generating virtual viewpoint images from multiple camera captures lack efficient methods for determining the optimal virtual viewpoint and line-of-sight direction based on subject and interest position information.
Innovation Solution
An information processing system that acquires subject and interest position information, and determines the position of a virtual viewpoint and its line-of-sight direction by considering the subject's position and the interest position in a virtual space corresponding to a real space.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If the virtual viewpoint position and line-of-sight direction are determined based only on subject position information, then the system can easily track the subject, but the interest position may not be properly included in the virtual viewpoint image
Solution Approach 1:
The system changes the parameters used for viewpoint determination from only subject position to a combination of subject position and interest position. By introducing interest position information as an additional parameter, the system maintains subject tracking capability while ensuring the interest position is properly included in the virtual viewpoint image.
Solution Approach 2:
The system adds another dimension to the viewpoint determination process by incorporating interest position information alongside subject position information. This multi-dimensional approach allows simultaneous consideration of both subject tracking and interest position inclusion, resolving the contradiction between ease of subject tracking and reliability of interest position inclusion.
2Reliability
If the virtual viewpoint is determined to include both subject and interest position, then the image quality improves, but the computational complexity increases
Solution Approach 1:
The system performs preliminary determination of the virtual viewpoint position and line-of-sight direction based on both subject and interest position information before generating the virtual viewpoint image. This preliminary calculation of viewpoint parameters simplifies the subsequent image generation process, as the optimal viewpoint is pre-determined rather than computed during rendering.
Solution Approach 2:
The system uses virtual viewpoint information (position and line-of-sight direction) as a template or copy that guides the generation of the virtual viewpoint image. By pre-calculating and storing the viewpoint parameters as reusable data, the system avoids repeated complex computations during image generation, reducing overall computational complexity while maintaining high image quality.
Data Source
AI summary
An information processing system includes one or more memories storing instructions, and one or more processors executing the instructions to: acquire subject information indicating a position of a subject, images of which are captured by a plurality of imaging apparatuses, in a virtual space that corresponds to a position in a real space, region information indicating a region in the virtual space, and interest position information indicating an interest position associated with the region, and in a case where the subject exists within the region, determine a position of a virtual viewpoint corresponding to a virtual viewpoint image and a line-of-sight direction from the virtual viewpoint based on the subject information and the interest position information.