Camera Array Super-Resolution for Mobile Sensor Noise
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional image sensors in mobile devices face performance constraints such as limited dynamic range, low signal-to-noise ratio, and reduced sensitivity to low light levels due to small pixel size and the use of Bayer filters, which hinder image quality, especially in mobile camera systems.
Innovation Solution
A camera array comprising multiple imagers with varying imaging characteristics, including different pixel sizes, filters, and optical configurations, which capture images that are combined using a super-resolution process to achieve higher resolution and quality images, and incorporating near-IR imagers to enhance low-light sensitivity and noise reduction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Weight of moving object
If small pixel size is used in mobile devices, then device portability and integration are improved, but dynamic range and signal-to-noise ratio deteriorate
Solution Approach 1:
The patent divides the imaging function into multiple independent imagers, each with its own lens and sensor elements. This segmentation allows each imager to have optimized characteristics for specific tasks (e.g., low-light sensitivity, color capture) while the combined array provides overall high performance, resolving the contradiction between small pixel size and signal quality.
Solution Approach 2:
Multiple imagers are merged into a single camera array that operates as an integrated system. By combining the output of multiple imagers with different characteristics (some optimized for low light, others for color), the system achieves superior signal-to-noise ratio and dynamic range while maintaining compact form factor suitable for mobile devices.
2Loss of information
If Bayer filters are used, then color information is captured, but dynamic range and low-light sensitivity deteriorate
Solution Approach 1:
The patent segments the color filtering function by providing separate imagers for different spectral regions rather than using a single Bayer-filtered imager. This allows each imager to capture specific wavelength ranges with optimized sensitivity, improving dynamic range while maintaining color information through the combined array output.
Solution Approach 2:
Different imagers in the array are assigned different spectral sensitivities and filtering characteristics tailored to their specific functions. Some imagers are optimized for low-light sensitivity while others are optimized for color capture, with each imager having locally optimized quality characteristics that contribute to overall system performance.
3Measurement precision
If multiple imagers with different characteristics are combined, then image quality and dynamic range are improved, but device complexity increases
Solution Approach 1:
Multiple imagers are merged into a single integrated camera array that functions as one unified imaging device. The imagers share common structural elements such as the substrate, packaging, and signal processing circuits, which reduces the overall complexity compared to having separate imaging devices while maintaining the benefits of multiple imagers with different characteristics.
Solution Approach 2:
The camera array is designed as a universal imaging platform where different imagers can be used for various functions (low-light imaging, color imaging, spectral analysis) depending on the application requirements. This multi-functionality allows a single device to replace multiple specialized devices, managing complexity through versatility.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The solution enables the capture of high dynamic range, panoramic, and hyper-spectral images with improved low-light sensitivity and image quality by combining images from imagers with different characteristics, effectively addressing the limitations of conventional image sensors in mobile devices.
Implementation Method 1
lens stack arrays that can be utilized in camera arrays in accordance with embodiments of the invention
Implementation Method 2
light enters through an opening (aperture) at one end of the imaging device and is directed to an image sensor by an optical element such as a lens
Implementation Method 3
Filters are often employed in the image sensor to selectively transmit lights of certain wavelengths onto pixels
Implementation Method 4
A Bayer filter mosaic is often formed on the image sensor. The Bayer filter is a color filter array that arranges one of the RGB color filters on each of the color pixels
Implementation Method 5
The image sensor consists of pixels that generate signals upon receiving light via the optical element
Implementation Method 6
Commonly used image sensors include CCD (charge-coupled device) image sensors and CMOS (complementary metal-oxide-semiconductor) sensors
Implementation Method 7
images that are combined using a super-resolution process to achieve higher resolution and quality images
Data Source
AI summary
Systems and methods for implementing array cameras configured to perform super-resolution processing to generate higher resolution super-resolved images using a plurality of captured images and lens stack arrays that can be utilized in array cameras are disclosed. An imaging device in accordance with one embodiment of the invention includes at least one imager array, and each imager in the array comprises a plurality of light sensing elements and a lens stack including at least one lens surface, where the lens stack is configured to form an image on the light sensing elements, control circuitry configured to capture images formed on the light sensing elements of each of the imagers, and a super-resolution processing module configured to generate at least one higher resolution super-resolved image using a plurality of the captured images.