Virtual Viewpoint Video Generation with Real Camera Movement Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing techniques for generating virtual viewpoint videos fail to accurately replicate camera movements, such as camera shake, when switching between real camera videos and virtual viewpoint videos, leading to a noticeable difference and incongruity for the user.
Innovation Solution
An information processing apparatus that obtains image capturing information, including the position, orientation, and movement of image capturing devices, and generates viewpoint information to create a virtual viewpoint video that reflects the movement of real cameras, ensuring seamless switching by incorporating camera movement information into the virtual camera path.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If the virtual viewpoint video is generated without incorporating real camera movement information, then the generation process is simpler, but the user perceives a noticeable difference and incongruity when switching between real camera video and virtual viewpoint video
Solution Approach 1:
The system performs preliminary action by obtaining movement information from the real camera before generating the virtual viewpoint video. This movement information is then incorporated into the virtual camera parameters during the generation process, ensuring that the virtual video already contains the necessary movement characteristics before switching occurs, thereby achieving seamless transitions.
2Reliability
If the movement information of the real camera is incorporated into the virtual viewpoint video, then the sense of incongruity during switching is reduced, but the complexity of the video generation process increases
Solution Approach 1:
The system uses copying by obtaining movement information from the real camera and applying it to the virtual camera parameters. Instead of complex real-time synchronization, the real camera's movement data is copied and integrated into the virtual viewpoint generation process, achieving seamless switching while maintaining manageable system complexity.
3Manufacturing precision
If camera movement information is obtained and integrated into the virtual viewpoint generation, then the camera shake and movement are accurately reproduced, but the processing requirements and system complexity increase
Solution Approach 1:
The system introduces an intermediary approach by using movement information obtained from the real camera as a bridge. This movement information serves as a mediator that transfers the real camera's characteristics to the virtual viewpoint generation process, enabling accurate reproduction of camera shake and movement without requiring direct complex integration of the real camera system itself.
Data Source
AI summary
An information processing apparatus according to one embodiment of the present disclosure obtains image capturing information including information representing a position and an orientation of an image capturing device and movement information representing specific movement of the image capturing device; and generates viewpoint information representing a position of a virtual viewpoint and a view direction from the virtual viewpoint for generating a virtual viewpoint video based on the obtained image capturing information and the obtained movement information in a case where a captured image obtained by the image capturing device and the virtual viewpoint video generated based on a plurality of images are switched.