Noise Model for Image Sensor Pixel Merging
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current low dynamic range (LDR) imaging technologies struggle to capture scenes with wide brightness ranges, often resulting in loss of detail in both bright and dark regions due to limitations in exposure time adjustments.
Innovation Solution
A noise model for image sensors is developed to compare pixel characteristics across multiple images captured at different exposure times, allowing for the selection and merging of pixels based on noise deviation, enabling the creation of high dynamic range (HDR) images that retain details in both bright and dark areas.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If LDR imaging is used to capture scenes with wide brightness ranges, then the image can be displayed on current video output devices, but details in both bright and dark regions are lost
Solution Approach 1:
The image capture process is segmented into multiple exposures with different exposure times. The imaging device captures several images of the same scene at different exposure levels, allowing separate optimization for bright and dark regions. These segmented exposures are then merged to preserve details across the full brightness range.
Solution Approach 2:
The exposure time is made dynamic rather than fixed. The system adjusts exposure times across multiple captures to adapt to different brightness regions in the scene. By varying exposure parameters dynamically and merging results, the system overcomes the static limitation of LDR imaging while maintaining compatibility with standard display devices.
2Reliability
If multiple images are merged to create HDR images, then details in bright and dark areas are preserved, but noise deviation increases
Solution Approach 1:
A noise model provides feedback about the expected noise characteristics of the imaging device under different operating conditions. This noise model is used to guide the merging process by comparing corresponding pixels from multiple images and determining whether differences are due to noise or actual scene variations. Pixels are merged only when differences fall within expected noise bounds, preventing noise amplification.
Solution Approach 2:
The system changes parameters (exposure time, gain) across multiple image captures to optimize the signal-to-noise ratio for different brightness regions. By varying these parameters and selectively merging images based on noise characteristics, the system preserves details while managing noise levels in the final HDR image.
3Reliability
If exposure time is increased to capture dark regions, then details in dark areas are improved, but bright regions become overexposed
Solution Approach 1:
The brightness range is segmented across multiple exposures. One exposure captures dark regions with adequate detail, while another exposure captures bright regions without overexposure. By segmenting the capture task into multiple images with different exposure settings, the system achieves complete coverage of the brightness range.
Solution Approach 2:
Multiple images captured with different exposure times are merged into a single HDR image. The merging process combines the strengths of each exposure: dark region details from longer exposures and bright region details from shorter exposures, achieving a final image that preserves details across the entire brightness range without overexposure or underexposure.
Data Source
AI summary
A plurality of images of a scene may be obtained. These images may have been captured by an image sensor, and may include a first image and a second image. A particular gain may have been applied to the first image. An effective color temperature and a brightness of a first pixel in the first image may be determined, and a mapping between pixel characteristics and noise deviation of the image sensor may be selected. The pixel characteristics may include pixel brightness. The selected mapping may be used to map at least the brightness of the first pixel to a particular noise deviation. The brightness of the first pixel and the particular noise deviation may be compared to a brightness of a second pixel of the second image. The comparison may be used to determine whether to merge the first pixel and the second pixel.