Heterogeneous Camera Array for Low-Light Parallax Imaging
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Image sensors in mobile devices suffer from low light sensitivity, reduced signal-to-noise ratio, and limited dynamic range due to smaller pixel sizes and constraints in light gathering ability, leading to poor performance in low light conditions.
Innovation Solution
A camera array with a plurality of imagers, each with distinct imaging characteristics, including different filters, optics, and operating parameters, captures images that are combined using super-resolution processes to enhance resolution and quality, and incorporates near-IR imagers to reduce noise and increase sensitivity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Volume of moving object
If pixel size is reduced for mobile devices, then device miniaturization is achieved, but light gathering ability and signal-to-noise ratio deteriorate
Solution Approach 1:
The patent divides the image sensor into multiple heterogeneous imagers, each with specialized characteristics (e.g., some optimized for low light, others for color accuracy). This segmentation allows each imager to be smaller while collectively providing the light-gathering capability of a larger sensor, resolving the contradiction between device miniaturization and light gathering ability.
Solution Approach 2:
The patent employs a composite architecture combining different types of imagers (color imagers, monochrome imagers, near-IR imagers) in a single array. This composite approach leverages the complementary strengths of each imager type to achieve both compact size and superior low-light performance, as each imager contributes differently to the overall image quality.
2Loss of information
If Bayer filter is used to capture color information, then color imaging is achieved, but low light sensitivity deteriorates
Solution Approach 1:
The patent segments the imager array into specialized groups: color imagers with Bayer filters for accurate color reproduction and monochrome imagers without filters for maximum light sensitivity. This segmentation allows the system to capture both color information and low-light performance by combining the strengths of different imager types.
Solution Approach 2:
The heterogeneous imager array provides multi-functionality where different imagers serve different purposes within the same system. Color imagers handle chromatic information while monochrome and near-IR imagers handle luminance and low-light information, making the overall system universally capable of handling various lighting conditions and imaging requirements.
3Ease of manufacture
If conventional image sensor is used, then manufacturing simplicity is maintained, but dynamic range and noise performance deteriorate
Solution Approach 1:
The patent segments the sensor into multiple heterogeneous imagers with different characteristics (different filter configurations, optical elements, and operating parameters). This segmentation enables each imager to be optimized for specific dynamic range requirements while maintaining compatibility with existing manufacturing processes, thus improving overall dynamic range without completely complicating manufacturing.
Solution Approach 2:
The patent changes key parameters of the imagers such as filter type, optical element configuration, and operating parameters to create heterogeneous imagers with different dynamic ranges and noise characteristics. By varying these parameters across the imager array, the system achieves extended dynamic range while keeping individual imager manufacturing relatively simple.
4Device complexity
If single imager is used, then device complexity is minimized, but image quality and resolution are limited
Solution Approach 1:
The patent segments the imaging function across multiple heterogeneous imagers, each capturing different aspects of the scene (color, luminance, near-IR). This segmentation improves image quality and resolution by combining complementary information from different imagers, while the modular nature of the segmentation keeps the added complexity manageable through standardized interfaces and processing protocols.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The solution provides high dynamic range, panoramic, and hyper-spectral images with improved low light sensitivity and reduced noise, achieving higher quality images than conventional sensors by leveraging the synergistic capabilities of the heterogeneous imagers.
Implementation Method 1
Each of the plurality of imagers may include a different optical filter. In one embodiment, each of the plurality of imagers may have different optical filters. In another embodiment, a first imager includes a first optical filter and a second imager includes a second optical filter.
Implementation Method 2
The image sensor consists of pixels that generate signals upon receiving light via the optical element
Implementation Method 3
In a typical imaging device, light enters through an opening (aperture) at one end of the imaging device and is directed to an image sensor by an optical element such as a lens
Implementation Method 4
The images from the first imager and the second imager are combined using a super-resolution process to obtain images of higher resolution
Data Source
Figure 1~2A
Figure 2B~3B
Figure 3C~4
AI summary
A method for estimating distance to an object in a scene using an imaging system (400) comprising a camera array (410) with multiple imagers (540), an image processing pipeline module (420) and a controller (440). The image processing pipeline module (420) comprises an upstream pipeline processing module (510), an image pixel correlation module (514), and a parallax confirmation and measurement module (518). The method comprises: providing images (412), including a current image, captured by the multiple imagers (540) to the upstream pipeline processing module (510) for processing; aligning, by the image pixel correlation module (514), portions of images, including the current image, captured by different imagers of the multiple imagers to compensate for parallax; processing, by the parallax confirmation and measurement module (518), the current image to detect and measure the parallax by keeping track of various pair-wise measurements and calculating a parallax difference that is a best fit to a sample data; and determining distance to an object in the scene using the measured parallax.