Array Camera Autofocus Using Depth Maps from Conventional Video
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Array cameras face high computational load and power consumption issues when generating real-time video due to the need for synthesizing images from multiple viewpoints, which is particularly problematic in mobile devices where processing time and power expenditure are significant.
Innovation Solution
Incorporating a separate camera with a fixed baseline distance from the array camera module, which captures images from a different viewpoint, and using a processor to determine depth estimates and generate depth maps to refine image synthesis, thereby reducing the computational load and power requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If array camera synthesizes video from multiple individual camera images, then depth estimation accuracy is improved, but computational load and power consumption increase significantly
Solution Approach 1:
The system segments the imaging function into two parts: the array camera module captures images from multiple viewpoints for depth estimation, while the separate conventional camera captures video frames. This division allows depth information to be extracted from the array camera without requiring full video synthesis, reducing computational load while maintaining depth accuracy.
Solution Approach 2:
The separate conventional camera acts as an intermediary that captures video frames which are then combined with depth maps generated from array camera images. This intermediary approach allows the system to leverage depth information from the array camera without performing computationally intensive video synthesis from all array camera images.
2Manufacturing precision
If array camera synthesizes high resolution video from multiple individual camera images, then image resolution is improved, but processing time increases
Solution Approach 1:
The system performs preliminary depth estimation using images from the array camera module before video capture. The depth maps generated in advance are then applied to video frames from the separate conventional camera, eliminating the need for time-consuming real-time synthesis from multiple array camera images during video recording.
Solution Approach 2:
The separate conventional camera captures video frames that serve as copies of the scene, which are then enhanced with depth information from the array camera module. This copying approach allows the system to obtain high-resolution video without performing computationally intensive image synthesis from multiple array camera sources.
3Use of energy by moving object
If array camera captures video in desired resolution directly, then power consumption is reduced, but depth estimation capability is lost
Solution Approach 1:
The system merges the depth estimation capability of the array camera module with the video capture function of the separate conventional camera. The array camera module captures images specifically for depth map generation, while the conventional camera captures video frames, and these two functions are combined to produce video with enhanced depth information without requiring the array camera to capture full-resolution video.
4Adaptability or versatility
If array camera performs cropping and super-resolution processing, then video format compatibility is improved, but computational load increases
Solution Approach 1:
The system extracts depth information from array camera images and applies it to video frames from the conventional camera. This extraction approach allows the system to obtain depth maps for format conversion and super-resolution processing without performing these computationally intensive operations on all array camera images, thereby reducing overall computational load while maintaining format compatibility.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach allows for efficient video capture and still image processing, reducing power consumption and improving depth estimation accuracy, enabling the array camera to operate with lower computational load and power usage compared to conventional cameras.
Implementation Method 1
An array camera includes a plurality of individual cameras (i.e., cameras) that can capture images of a scene where the image obtained by each camera is from a slightly different viewpoint
Data Source
AI summary
Systems with an array camera augmented with a conventional camera in accordance with embodiments of the invention are disclosed. In some embodiments, the array camera is used to capture a first set of image data of a scene and a conventional camera is used to capture a second set of image data for the scene. An object of interest is identified in the first set of image data. A first depth measurement for the object of interest is determined and compared to a predetermined threshold. If the first depth measurement is above the threshold, a second set of image data captured using the conventional camera is obtained. The object of interest is identified in the second set of image data and a second depth measurement for the object of interest is determined using at least a portion of the first set of image data and at least a portion of the second set of image data.