Multi-camera virtual viewpoint synthesis with simultaneous capture timing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing motion picture capturing systems that generate virtual viewpoint images using multiple cameras face challenges in maintaining accurate estimation of camera positions and orientations, especially when handheld or with moving subjects, due to errors in corresponding point detection and estimation accuracy.
Innovation Solution
An image-capturing apparatus with multiple units, including a timing control unit for simultaneous image capture, a motion estimation unit to calculate camera motion, and a virtual viewpoint synthesis unit to improve frame rate performance while maintaining accurate camera position and orientation estimation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If image-capturing is performed with timing being shifted by multiple cameras, then frame rate performance is improved, but estimation accuracy of camera position and orientation deteriorates due to errors in corresponding point detection and moving subjects
Solution Approach 1:
The system performs preliminary calibration to establish the positional and orientational relationships between multiple cameras before actual image capturing. This pre-established relationship data is then used during virtual viewpoint image generation, eliminating the need for real-time correspondence point detection and its associated errors with moving subjects.
Solution Approach 2:
The system introduces an intermediary calibration process that creates a reference framework for camera relationships. This intermediary step separates the accurate determination of camera positions and orientations from the actual image capturing process, allowing shifted timing capture without compromising estimation accuracy.
2Adaptability or versatility
If handheld cameras with varied positions and orientations are used, then versatility and adaptability are improved, but estimation accuracy deteriorates due to difficulty in maintaining precise camera relationship data
Solution Approach 1:
The system performs preliminary calibration to establish the positional and orientational relationships between multiple cameras before actual image capturing. This pre-established relationship data is then used during virtual viewpoint image generation, eliminating the need for real-time correspondence point detection and its associated errors with moving subjects.
Solution Approach 2:
The system allows cameras to be handheld and move freely during operation, but uses the pre-calibrated positional and orientational relationships to maintain accurate virtual viewpoint image generation despite camera movement, effectively separating the flexibility of camera handling from the precision of spatial relationship data.
Data Source
AI summary
In a motion picture capturing system for generating a virtual viewpoint image by a plurality of cameras, if image-capturing is performed with timing being shifted to improve frame rate performance, estimation accuracy is deteriorated due to an error in corresponding point detection between images or the presence of a moving subject. The image-capturing timing is controlled such that two or more of a plurality of image-capturing units perform simultaneous image-capturing, and motion of the whole of an image-capturing apparatus is estimated from a plurality of images captured by the simultaneous image-capturing. The virtual viewpoint image is synthesized from the images captured by the simultaneous image-capturing by the plurality of image-capturing units and the estimated motion.