Gaze-directed denoising multi-camera imaging
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing imaging systems, particularly in extended-reality devices, face challenges in capturing high-resolution, blur-free images in low-light environments due to limitations in exposure time, camera shake, and dynamic content, leading to noise and brightness inconsistencies across the field of view.
Innovation Solution
A multi-camera system that simultaneously captures images with varying illumination parameters, using a first camera to detect regions with low brightness and high noise and a second camera to detect regions with high brightness and blurriness, applying denoising techniques to combine image segments for improved image quality and uniformity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Illumination intensity
If exposure time is increased to improve brightness, then image brightness is improved, but image sharpness deteriorates due to camera shake and dynamic content
Solution Approach 1:
The patent segments the image processing task across multiple cameras with different exposure times. One camera captures with longer exposure for brightness, another with shorter exposure for sharpness. The segmentation allows each camera to optimize for its specific function without compromising the other.
Solution Approach 2:
The patent merges images from multiple cameras with different exposure characteristics. By combining the long-exposure bright image with the short-exposure sharp image through image fusion techniques, the system achieves both brightness and sharpness simultaneously in the final output.
2Illumination intensity
If ISO sensitivity is increased to improve brightness, then image brightness is improved, but image noise increases
Solution Approach 1:
The patent introduces multiple cameras as intermediaries to capture the scene under different exposure conditions. Instead of relying on a single camera with high ISO (which introduces noise), the system uses multiple cameras with lower ISO settings but different exposure times, allowing noise-free image fusion.
Solution Approach 2:
The patent changes the exposure time parameter across multiple cameras while keeping ISO sensitivity relatively low. This parameter variation allows each camera to capture different aspects of the scene (brightness vs. sharpness) without the noise penalty of high ISO amplification.
3Illumination intensity
If aperture size is increased to improve brightness, then image brightness is improved, but depth of field decreases and optical quality deteriorates
Solution Approach 1:
The patent dynamically assigns different aperture settings to different cameras based on their specific roles. One camera operates with larger aperture for brightness, another with smaller aperture for depth and sharpness. This dynamic configuration allows the system to optimize optical quality while maintaining brightness through image fusion.
4Measurement precision
If high PPD is used to improve resolution, then image resolution is improved, but light receiving capability per pixel decreases leading to noise
Solution Approach 1:
The patent adds a temporal dimension to the imaging system by using multiple cameras capturing at different exposure times. This allows the system to achieve high effective resolution through fusion of images from multiple temporal snapshots, rather than relying solely on spatial pixel density which would reduce per-pixel light gathering.
Data Source
AI summary
When a first camera has a first value of an illumination parameter in first region(s) of a first field of view (FOV) of the first camera, while a second camera has, in corresponding region(s) of a second FOV of the second camera, a second value of the illumination parameter that is greater than the first value, a denoising technique is applied on first image segment(s) of the first image that represents the first region(s), based on corresponding image segment(s) of the second image that represents the corresponding region(s). The illumination parameter is any one of: (i) a ratio of a per-pixel area to pixels per degree (PPD), (ii) a ratio of a multiplication product of the per-pixel area and a relative illumination to the PPD.