Multi-Exposure Image Sensor Merging for Expanded Dynamic Range
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Image sensors have a limited dynamic range compared to human vision, leading to captured images that differ from actual scenes, especially in high dynamic range scenarios like backlighting, resulting in underexposed or overexposed areas.
Innovation Solution
An image sensor and processing system that generates multiple images with different exposures, merges them through pre-processing to enhance dynamic range, and performs high dynamic range (HDR) processing to create an image with increased luminance range.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multiple images with different exposure times are captured and merged, then the dynamic range of the output image is extended, but the device complexity increases
Solution Approach 1:
The image sensor performs preliminary actions by capturing multiple images with different exposure times (long, medium, short) before the main processing stage. This pre-capture of multiple exposure images allows the processing system to later merge them into a high dynamic range image, resolving the contradiction by performing the complex capture operation in advance rather than requiring complex real-time processing
Solution Approach 2:
The imaging process is segmented into multiple discrete exposure operations (long exposure image, medium exposure image, short exposure image) rather than attempting to capture the entire dynamic range in a single operation. This segmentation allows each individual capture operation to remain simple while the combination of segmented results achieves the desired high dynamic range
2Measurement precision
If multiple images with different exposure times are captured, then the dynamic range is extended, but the processing load on the image processor increases
Solution Approach 1:
The image sensor extracts and performs the complex multi-exposure capture functionality independently, separating this task from the image processor. By taking out the multi-image capture operation to the sensor level, the image processor's workload is reduced to primarily merging and processing the extracted images rather than performing the complex capture operations itself
Solution Approach 2:
The image sensor acts as an intermediary that handles the complex multi-exposure capture operations and provides processed image data to the image processor. This intermediary role allows the image processor to focus on higher-level processing tasks rather than being burdened by the computational load of capturing and merging multiple exposure images
3Measurement precision
If pre-processing is performed to merge images, then the dynamic range is increased and processing load is reduced, but the device complexity increases
Solution Approach 1:
The image sensor performs preliminary merging operations on the multiple exposure images within the sensor itself before outputting to the image processor. This pre-processing action reduces the complexity burden on the image processor while achieving the desired dynamic range extension, as the complex merging operation is performed in advance at the sensor level
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The system produces images with an expanded dynamic range and improved signal-to-noise ratio, enhancing image quality and reducing processing load on the image processor, enabling real-time preview and video capabilities.
Implementation Method 1
a pixel array configured to convert optical signals incident thereon into electrical signals based on different exposure times
Data Source
AI summary
An image sensor includes a pixel array and a readout circuit that generates a first image of an object based on a first exposure time, a second image of the object based on a second exposure time, and a third image of the object based on a third exposure time, the first exposure time, the second exposure time, and the third exposure time being different from each other and the second exposure time and the third exposure time are shorter than the first exposure time, a pre-processor that performs linearization processing on the second image and the third image to generate a merged image with an increased number of bits of a pixel value, and an interface circuit that outputs first image data based on the first image and second image data based on the merged image to an external processor.