Multi-Sensor Omni-Stereo Video Capture System
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for capturing omni-stereo videos face challenges such as low resolution, insufficient light capture, and computational complexity, particularly with single sensor systems and bulky multi-camera rigs that struggle to produce high-quality videos in real-time, especially in indoor scenes with limited ambient light.
Innovation Solution
A multi-sensor system comprising at least three left eye and three right eye cameras arranged tangentially around a viewing circle, which calibrate and compute panoramas without stitching, allowing for real-time capture of omni-stereo images with a 360-degree horizontal and 180-degree vertical field of view, using wide-angle or fish-eye lenses for enhanced coverage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If single sensor based optical system is used, then device complexity is reduced, but manufacturing precision and image quality deteriorate due to inability to produce high resolution omni-stereo videos and capture enough light with small aperture
Solution Approach 1:
The system divides the imaging task into multiple segments by using multiple separate camera sensors (at least three left eye cameras and three right eye cameras) arranged around a viewing circle. Each camera captures a specific angular segment of the scene, and these segments are computationally combined to form the complete omni-stereo video, thereby achieving high resolution without requiring a single complex sensor
Solution Approach 2:
The system transitions from a single-point sensor approach to a distributed spatial arrangement of multiple sensors around a viewing circle. By adding the spatial dimension of multiple camera positions, the system achieves comprehensive angular coverage and high resolution omnidirectional imaging that cannot be obtained with a single sensor
2Illumination intensity
If multi camera rigs are used, then image quality and light capture are improved, but device complexity and weight increase making the system heavy and bulky
Solution Approach 1:
The camera rig is segmented into multiple independent camera modules arranged around a viewing circle, with each camera having its own aperture optimized for light capture. This segmentation allows each sensor to capture sufficient light independently while the collective arrangement provides comprehensive coverage, avoiding the need for a single large complex aperture system
Solution Approach 2:
Instead of increasing aperture size in a single camera, the system distributes multiple cameras with optimized apertures around a viewing circle. This spatial distribution in another dimension achieves superior light capture across all viewing angles without requiring any single camera to have an excessively large aperture
3Adaptability or versatility
If existing multi camera rigs are used, then comprehensive scene coverage is achieved, but productivity decreases due to high computation requirements for stitching and inability to capture in real time
Solution Approach 1:
The cameras are pre-calibrated with known intrinsic and extrinsic parameters before operation. This preliminary calibration establishes the geometric relationships between all cameras and the viewing circle, enabling real-time processing without requiring complex post-capture stitching computations. The calibration data is used directly to map captured images to the final omni-stereo output
Solution Approach 2:
The system replaces the mechanical stitching process with a computational approach using pre-calibrated parameters. Instead of physically aligning and manually stitching images, the patent uses mathematical models based on the known camera positions and orientations to directly compute the omni-stereo video in real-time, significantly reducing processing requirements
Data Source
AI summary
A method of calibrating cameras used to collect images to form an omni-stereo image is disclosed. The method may comprise determining intrinsic and extrinsic camera parameters for each of a plurality of left eye cameras and right eye cameras arranged along a viewing circle or ellipse and angled tangentially with respect to the viewing circle or ellipse; categorizing left-right pairs of the plurality of left eye cameras and the plurality of right eye cameras into at least a first category, a second category or a third category; aligning the left-right pairs of cameras that fall into the first category; aligning the left-right pairs of cameras that fall into the second category; and aligning the left-right pairs of cameras that fall into the third category by using extrinsic parameters of the left-right pairs that fall into the first category, and of the left-right pairs that fall into the second category.