Virtual Viewpoint Image Shaking Reduction via Periodic Update
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing techniques for generating virtual viewpoint images often result in unnatural movement of objects due to mismatched frame update intervals, leading to shaking or unsmooth motion, particularly in slow-motion or low-frame-rate scenarios.
Innovation Solution
An information processing apparatus that acquires update information and frame information from multiple image capturing devices to generate viewpoint information for virtual viewpoint images, adjusting the update interval of the virtual viewpoint to match or exceed the frame update interval, thereby reducing object shaking by using fewer frames of multi-viewpoint images than the virtual viewpoint images forming a particular reproduction period.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If multiple frames of virtual viewpoint images are generated from each frame of captured image to achieve variable-speed reproduction, then the frame rate of the virtual viewpoint image is increased, but the position of the object changes every frame while the virtual viewpoint changes every frame, causing shaking or unsmooth movement
Solution Approach 1:
The patent applies periodic action by updating the virtual viewpoint position at specific intervals rather than continuously. The control unit determines whether to update the virtual viewpoint based on the frame update interval, creating a periodic update pattern that synchronizes with the object's movement characteristics. This periodic updating reduces unnecessary viewpoint changes that cause shaking while maintaining smooth visual reproduction at the desired frame rate.
2Speed
If the virtual viewpoint is updated at every frame to maintain high frame rate reproduction, then the frame update interval is shortened, but the object position changes cause shaking or unsmooth movement in the virtual viewpoint image
Solution Approach 1:
The patent implements dynamics by making the virtual viewpoint update interval adaptive rather than fixed. The control unit dynamically adjusts the update timing based on the relationship between the frame update interval and object movement characteristics. This dynamic adjustment allows the system to maintain high frame rates while adapting the viewpoint update frequency to match the actual movement patterns, ensuring natural and smooth object movement representation.
Data Source
AI summary
An apparatus acquires update information indicating a predetermined frame update interval on a moving image at a virtual viewpoint reproduced by updating a virtual viewpoint image at the interval, the virtual viewpoint image being generated using multi-viewpoint images and viewpoint information; acquires frame information of the multi-viewpoint images for use in generating the moving image, the frame information including information of a plurality of frames corresponding to different imaging times; and generates the viewpoint information based on the update information and the frame information, such that the viewpoint information indicating a position of a virtual viewpoint and a direction from the virtual viewpoint updated at an interval longer than the predetermined frame update interval is generated as the viewpoint information for the moving image generated using a smaller number of frames of multi-viewpoint images than frames of virtual viewpoint images forming the moving image.