Multi-Camera Presence Capture for Motion Parallax Rendering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing camera systems fail to provide an immersive experience by accurately capturing and presenting media that closely resemble the perception of the real world, as they do not adequately account for motion parallax and discrepancies between recorded media and real-world perception.
Innovation Solution
A device comprising an array of cameras captures multiple media streams from different perspectives, processes these streams based on a viewer's position and orientation, and generates output streams that reflect these parameters, incorporating motion parallax to create a realistic and immersive experience.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a single camera captures video data for display on a screen, then the system is simple and easy to operate, but it cannot provide an immersive experience or accurately represent motion parallax and depth perception
Solution Approach 1:
The patent divides the camera system into multiple cameras arranged in an array, where each camera captures video data from a different perspective. This segmentation allows the system to capture motion parallax and depth information that a single camera cannot obtain, thereby providing an immersive experience while managing complexity through modular camera units
Solution Approach 2:
The patent transitions from a single-camera 2D capture system to a multi-camera 3D capture system. By adding the spatial dimension with multiple cameras positioned at different locations, the system captures depth and motion parallax information, enabling immersive 3D video content that accurately represents real-world perception
2Measurement precision
If stereoscopic camera systems provide media streams for both eyes to enable depth perception, then depth estimation is improved, but there is still a discrepancy between recorded media and perception of the real world due to lack of motion parallax
Solution Approach 1:
The patent segments the capture system into multiple cameras positioned to capture different perspectives simultaneously. This allows the system to record not only stereoscopic images for depth perception but also motion parallax information as the viewer moves, eliminating the information loss present in traditional stereoscopic systems
Solution Approach 2:
The patent implements a dynamic video generation system that adapts the output video content based on the viewer's position and orientation. By dynamically selecting and processing video streams from different camera perspectives according to real-time viewer data, the system preserves motion parallax information and accurately reproduces real-world perception as the viewer moves
3Adaptability or versatility
If 360-degree camera systems allow viewers to look in various directions, then viewing flexibility is improved, but the selected subset of pixels must be warped which can lose motion parallax information
Solution Approach 1:
The patent performs preliminary action by capturing video data from multiple perspectives simultaneously using an array of cameras before the viewing process. This pre-capture of multiple viewpoints allows the system to later select and combine appropriate video streams based on viewer position without needing to warp a single 360-degree image, thereby preserving motion parallax information
Solution Approach 2:
The patent implements dynamic selection of video streams from the camera array based on real-time viewer position and orientation data. Instead of using a static warped 360-degree image, the system dynamically combines video frames from different camera perspectives to generate output video that maintains motion parallax while providing flexible viewing directions
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A device is disclosed which comprises a media interface configured to provide a plurality of media streams, each media stream being captured using a camera of at least one array of cameras, an input interface configured to receive a device position associated with an output device, and a processor configured to generate from the plurality of media streams an output stream adjusted to the device position.