Image Sensor Multi-Exposure Merging for HDR Scenes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Image sensors have a limited dynamic range compared to human vision, leading to captured images that differ from actual scenes, particularly in high dynamic range environments like backlighting, where backgrounds are overly bright and subjects appear too dark.
Innovation Solution
An image sensor and processing system that generates multiple images with different luminances through varying exposure times, merges these images using pre-processing to enhance dynamic range, and performs high dynamic range (HDR) processing to create an image with increased dynamic range and signal-to-noise ratio.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multiple images with different exposure times are captured and merged, then the dynamic range of the output image is extended, but the processing complexity and data volume increase
Solution Approach 1:
The image sensor performs preliminary actions by capturing multiple images with different exposure times (long, medium, short) before the main processing stage. This pre-capture of multiple exposures allows the processing system to later merge these images to extend dynamic range, resolving the contradiction by performing the complex data collection in advance rather than during final processing.
Solution Approach 2:
The imaging process is segmented into multiple exposures with different exposure times, where each exposure captures a specific luminance range. The long exposure captures dark regions, medium exposure captures mid-tones, and short exposure captures bright regions. This segmentation allows the system to handle the full dynamic range by combining segmented portions from different exposures.
2Measurement precision
If multiple images are captured and processed to extend dynamic range, then the signal-to-noise ratio is improved, but the processing load on the image processor increases
Solution Approach 1:
The image sensor acts as an intermediary that performs preliminary merging of multiple exposures and generates optimized image data before transmitting to the image processor. This intermediary function reduces the processing load on the image processor by pre-computing the merged result, while still achieving improved signal-to-noise ratio through the multi-exposure technique.
3Device complexity
If the image sensor accommodates a narrower luminance range, then the device complexity is reduced, but the captured image differs from the actual scene appearance
Solution Approach 1:
The system merges multiple images captured with different exposure times to create a final image that represents the full dynamic range of the scene. By combining the long exposure (for dark regions), medium exposure (for mid-tones), and short exposure (for bright regions), the system achieves accurate representation of high dynamic range scenes without requiring each individual sensor to accommodate the full range simultaneously.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The system produces images with improved dynamic range and signal-to-noise ratio by pre-processing multiple exposures, allowing for better representation of high dynamic range scenes and reducing the processing load on the image processor.
Implementation Method 1
a pixel array that converts optical signals incident thereon into electrical signals based on different exposure times
Data Source
AI summary
An image sensor includes a pixel array and a readout circuit that generates a first image of an object based on a first exposure time, a second image of the object based on a second exposure time, and a third image of the object based on a third exposure time, the first exposure time, the second exposure time, and the third exposure time being different from each other and the second exposure time and the third exposure time are shorter than the first exposure time, a pre-processor that performs linearization processing on the second image and the third image to generate a merged image with an increased number of bits of a pixel value, and an interface circuit that outputs first image data based on the first image and second image data based on the merged image to an external processor.