Multi-Resolution Camera Set for Depth Map Generation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing camera array systems face challenges in generating high-resolution depth maps efficiently, leading to computational intensity, high hardware costs, and image artifacts due to occlusions and low-resolution depth maps, which are exacerbated by the need for expensive laser or Time of Flight systems and inflexible industrial designs.
Innovation Solution
A method using a multi-resolution camera set with a central high-resolution camera surrounded by multiple lower-resolution cameras, where images are captured, down-scaled, and up-scaled to generate a high-resolution depth map, optimizing depth map generation and occlusion determination, and implemented in various configurations including individual camera modules, wafer-based solutions, and integration with mobile devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a large number of high-resolution cameras are used in camera arrays, then depth map accuracy and image quality are improved, but hardware cost and device complexity increase significantly
Solution Approach 1:
The patent applies local quality by using different resolution cameras in different spatial positions within the array. Specifically, certain strategic positions are occupied by high-resolution cameras to capture critical depth information, while other positions use lower-resolution cameras. This non-uniform distribution optimizes depth map accuracy in key areas while reducing overall system complexity and cost.
Solution Approach 2:
The camera array is segmented into multiple resolution tiers rather than using uniform high-resolution cameras throughout. The system divides the camera array into different functional groups with varying resolutions, allowing independent optimization of each segment's contribution to depth mapping while controlling overall system complexity.
2Measurement precision
If traditional depth map generation methods are used with camera arrays, then depth information can be obtained, but computational complexity and processing time increase
Solution Approach 1:
The patent applies preliminary action by performing depth map generation at multiple resolution levels in a hierarchical sequence. First, a coarse depth map is generated using lower-resolution camera data, establishing initial depth estimates. Then, this preliminary depth map is refined using high-resolution camera data only in regions where improved accuracy is needed, rather than processing all data at full resolution from the start.
Solution Approach 2:
The depth map generation process is made dynamic through multi-resolution processing. The system adaptively selects which resolution level to use for processing different regions of the scene based on the preliminary depth estimates and confidence metrics, optimizing computational resources in real-time during the depth mapping process.
3Productivity
If low-resolution depth maps are used to reduce computational load, then processing complexity decreases, but image quality and depth accuracy deteriorate
Solution Approach 1:
The patent transitions from a single-resolution approach to a multi-resolution dimensional hierarchy. Instead of choosing between one resolution level, the system operates across multiple resolution dimensions, using low-resolution data for initial processing and high-resolution data for refinement, effectively adding a resolution dimension to the processing pipeline.
Solution Approach 2:
The system performs preliminary depth estimation at low resolution to identify regions requiring high-resolution processing. This preliminary action guides subsequent high-resolution processing only where necessary, maintaining depth accuracy while preserving computational efficiency by avoiding unnecessary high-resolution processing in all areas.
4Measurement precision
If expensive laser or Time of Flight systems are used, then depth measurement accuracy is improved, but hardware cost and device complexity increase
Solution Approach 1:
The patent creates multiple virtual views of the scene by computationally synthesizing images from different perspectives using the camera array geometry and depth information. These synthesized views serve as copies that can be used for depth verification and refinement without requiring additional physical sensors or expensive specialized depth measurement hardware.
Solution Approach 2:
The camera array system is designed to perform multiple functions using only standard cameras: it can capture color images, generate depth maps, create synthetic views, and perform refocusing. This multi-functionality eliminates the need for separate specialized depth measurement systems like laser scanners or Time of Flight cameras, reducing hardware cost and complexity while maintaining measurement accuracy.
Data Source
AI summary
Techniques for generating 3D images using multi-resolution camera set are described. In one example embodiment, the method includes, disposing a set of multi-resolution cameras including a central camera, having a first resolution, and one or more multiple camera groups, having one or more resolutions that are different from the first resolution, that are positioned substantially surrounding the central camera. Images are then captured using the multi-resolution camera set. A low-resolution depth map is then generated by down scaling the captured higher resolution image to lower resolution image. Captured lower resolution images are then up-scaled. A high-resolution depth map is then generated using the captured image of the central camera, the up-scaled captured images of the one or more multiple camera groups, and the generated low-resolution depth map. The 3D image of the captured image is then generated using the generated high-resolution depth map and the captured images.