Virtual Viewpoint Image Generation Using Selective 3D Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies for generating virtual viewpoint images face quality degradation if the object is not included in the image capture area, as the number of cameras capturing the object is less than the total camera group.
Innovation Solution
An information processing apparatus acquires viewpoint information to generate virtual viewpoint images using three-dimensional geometric data if the object is within the captured area, and generates images without three-dimensional data if the object is outside the captured area by a subset of cameras.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a fixed camera group is used for virtual viewpoint generation, then the system complexity is reduced and ease of operation is improved, but image quality degrades when objects move outside the predefined capture area
Solution Approach 1:
The patent applies dynamics by making the camera group configuration changeable in real-time based on object position. Instead of a fixed camera group, the system dynamically selects and configures camera groups according to where the object is located in the three-dimensional space, ensuring high image quality regardless of object position.
Solution Approach 2:
The system changes parameters (camera selection, capture area, viewpoint positions) based on object position detection. When an object moves to a different region, the system adjusts the camera group configuration and viewpoint parameters to maintain optimal image quality for the new position.
2Adaptability or versatility
If cameras are directed toward a specific gaze point, then the number of cameras required is reduced and device complexity is lowered, but virtual viewpoint images cannot be generated for objects outside the central capture area
Solution Approach 1:
The patent segments the overall capture area into multiple regional capture areas, each with its own optimized camera group. This allows different camera configurations to serve different spatial regions, expanding the total coverage area without requiring a single large complex system.
Solution Approach 2:
The system achieves multi-functionality by enabling a single camera to serve multiple purposes across different regional capture areas. The same camera may be included in different camera groups for different regions, and viewpoint information is shared across regions, reducing the total number of cameras needed.
3Reliability
If three-dimensional geometric data is always used for virtual viewpoint generation, then image quality is maintained, but processing time increases and productivity decreases when objects are outside the capture area
Solution Approach 1:
The patent applies local quality by using different processing methods in different regions. For objects within the capture area, full three-dimensional geometric data processing is used to maintain high quality. For objects outside the capture area, the system switches to alternative processing methods that are faster, accepting localized quality trade-offs for overall system efficiency.
Data Source
AI summary
A back-end server acquires viewpoint information representing a position of a virtual viewpoint and a line-of-sight direction from the virtual viewpoint used to generate a virtual viewpoint image based on a plurality of images captured by a plurality of image capturing apparatuses, outputs a first virtual viewpoint image generated using three-dimensional geometric data representing the three-dimensional shape of an object for which the virtual viewpoint image is to be generated if the object is included in an area that is captured by the plurality of image capturing apparatuses, and outputs a second virtual viewpoint image generated without using the three-dimensional geometric data if the object is included in an area that is not captured by a subset of the plurality of image capturing apparatuses.