Rolling Shutter Camera Arrays for Accurate 3D Volumetric Capture
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Rolling shutter cameras capture images of moving objects with artifacts due to different portions being captured at different times, leading to synchronization issues and inaccurate 3D models when multiple cameras are misaligned.
Innovation Solution
Positioning cameras at the same orientation, with each image captured by a rolling shutter and associated with its position and orientation, performing feature extraction, matching features, and generating a 3D pointcloud to merge into a volumetric model, while ensuring cameras capture images colinearly to reduce artifacts.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If rolling shutter cameras are used to capture images, then the cost is reduced and the device is more economically feasible, but synchronization issues arise and artifacts appear in 3D models when multiple cameras are misaligned
Solution Approach 1:
The patent segments the image capture process into discrete line scans with associated timing information. Each line of the rolling shutter capture is treated as a separate temporal snapshot, allowing the system to process and synchronize data from multiple cameras by dividing the continuous capture into manageable discrete units that can be aligned in time.
Solution Approach 2:
The patent implements feedback through timestamp synchronization and post-capture alignment algorithms. By recording timing information for each line scan and using this feedback to adjust and align data from multiple cameras during processing, the system compensates for the inherent lack of hardware synchronization in rolling shutter cameras, resolving the contradiction between cost and precision.
2Ease of manufacture
If rolling shutter captures images by scanning lines across the image, then the hardware is more economically feasible, but 10-33 milliseconds pass between beginning and end of capture causing position changes in moving objects
Solution Approach 1:
The patent applies preliminary action by recording precise timing metadata for each line scan before the actual 3D reconstruction process. This timing information is captured and stored in advance, allowing the system to later compensate for the temporal displacement of moving objects by reversing the rolling shutter scan pattern using the pre-recorded timing data.
Solution Approach 2:
The patent creates a temporal copy of the scene by reconstructing the moving object's position at different time points during the rolling shutter capture window. By copying and repositioning features according to their detected motion and the known scan timing, the system effectively creates multiple time-synchronized versions of the scene from a single capture sequence.
3Adaptability or versatility
If multiple cameras capture images at different times with rolling shutter, then volumetric capture can be achieved, but artifacts appear in 3D models due to misalignment
Solution Approach 1:
The patent merges data from multiple rolling shutter cameras by aligning their line scan sequences in time and space. The system combines the segmented line scan data from multiple cameras, using timing synchronization and feature matching to integrate them into a coherent 3D volumetric model, thereby achieving both versatility and precision.
Solution Approach 2:
The patent changes parameters by transforming the rolling shutter's temporal dimension into a spatial alignment parameter. By treating the scan line timing as a configurable parameter that can be adjusted and synchronized across cameras, the system optimizes the alignment of multi-camera data to minimize artifacts while maintaining volumetric capture capability.
Data Source
AI summary
An apparatus includes one or more processors and logic encoded in one or more non-transitory media for execution by the one or more processors and when executed operable to receive images from a set of cameras that are positioned in a plurality of levels around one or more objects, wherein the set of cameras are positioned at a same orientation and each image is captured by a rolling shutter and associated with a position and orientation of a corresponding camera from the set of cameras. The logic is further operable to, for each level of cameras generate a three-dimensional pointcloud based on the position and orientation of corresponding cameras and matched extracted features. The logic is further operable to merge the three-dimensional pointcloud for each level of the cameras to form a three-dimensional volumetric model of one or more objects.