Heterogeneous Camera Array Imaging for Parallax Depth and Low-Light SNR
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional image sensors in mobile devices face performance constraints such as low light sensitivity, limited dynamic range, and reduced signal-to-noise ratio (SNR) due to small pixel size and limited light gathering capability.
Innovation Solution
A distributed approach using a camera array with multiple heterogeneous imagers, each with custom filters, optics, and operating parameters, captures images that are then processed to generate an enhanced image with improved resolution and performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Volume of moving object
If pixel size is reduced to fit mobile device constraints, then device compactness is improved, but light gathering capability and SNR deteriorate
Solution Approach 1:
The patent divides the image sensor into multiple independent photodetectors (e.g., 2x2 array = 4 photodetectors) within a compact footprint. Each photodetector can be independently controlled with different integration times, allowing some to accumulate more light for better SNR while others capture faster-changing information, resolving the contradiction between small size and signal quality.
Solution Approach 2:
The patent implements dynamic control of integration time for each photodetector independently. During a single exposure period, different photodetectors can have different integration durations (e.g., 10ms, 20ms, 30ms), enabling adaptive light gathering optimization without increasing device size or pixel count.
2Illumination intensity
If integration time is increased to improve light sensitivity, then low light performance is improved, but dynamic range and temporal resolution deteriorate
Solution Approach 1:
The patent dynamically varies integration time across multiple photodetectors within a single exposure period. Some photodetectors use long integration times (e.g., 30ms) for low-light sensitivity, while others use short integration times (e.g., 10ms) for temporal resolution, with all readings combined to produce a single image that benefits from both approaches.
Solution Approach 2:
The patent adds a temporal dimension to the imaging process by allowing different integration times within a single exposure period. This transforms the traditional single integration-time model into a multi-temporal-dimensional model, enabling simultaneous optimization of both low-light performance and temporal resolution through computational combination of readings.
3Loss of information
If color filters are added to achieve color imaging, then color information is improved, but light gathering capability and SNR deteriorate
Solution Approach 1:
The patent segments the color filtering function across multiple photodetectors rather than placing filters on every pixel. Each photodetector can be assigned a specific color filter (e.g., red, green, blue, or panchromatic), and the system uses computational methods to reconstruct full-color images from the segmented readings, improving light gathering while maintaining color information.
Solution Approach 2:
The patent makes photodetectors multi-functional by allowing them to serve different color channels at different times or in different spatial positions. Panchromatic photodetectors can capture all wavelengths for luminance information, while color-filtered photodetectors provide chroma information, with the system adaptively using each detector's strengths to overcome the SNR penalty of color filtering.
4Measurement precision
If multiple camera arrays are used to improve image quality, then dynamic range and resolution are improved, but device complexity and manufacturing cost deteriorate
Solution Approach 1:
The patent merges multiple photodetector readings taken during a single exposure period into one composite image. By combining information from photodetectors with different integration times and spectral responses captured simultaneously, the system achieves high dynamic range and resolution without requiring multiple separate camera modules, thereby reducing device complexity and manufacturing cost.
Solution Approach 2:
The patent performs preliminary capture of multiple temporal and spectral samples within a single exposure period, then processes these pre-captured readings computationally to generate the final high-quality image. This preliminary multi-sample capture approach eliminates the need for multiple physical camera arrays, simplifying the system while maintaining image quality.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The camera array achieves enhanced image quality with increased dynamic range, improved low light sensitivity, and higher SNR, effectively addressing the performance limitations of conventional image sensors in mobile devices.
Implementation Method 1
The image sensor consists of pixels that generate signals upon receiving light via the optical element
Data Source
Figure 1~2A
Figure 2B~3B
Figure 3C~4
AI summary
A method for estimating distance to an object in a scene using an imaging system (400) comprising a camera array (410) with multiple imagers (540), an image processing pipeline module (420) and a controller (440). The image processing pipeline module (420) comprises an upstream pipeline processing module (510), an image pixel correlation module (514), and a parallax confirmation and measurement module (518). Images (412), including a current image, captured by the multiple imagers (540) are provided to the upstream pipeline processing module (510) for processing. The image pixel correlation module (514) aligns portions of images, including the current image, captured by different imagers of the multiple imagers to compensate for parallax. The parallax confirmation and measurement module (518) processes the current image to detect and measure the parallax. Distance to an object in the scene is determined using the measured parallax.