Highlight Recovery in Image Signal Processors
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional image processing techniques fail to adequately address image distortions and errors introduced by imaging device components, such as defective pixels and lens imperfections, and are inefficient, often causing information loss and inaccuracies in color reproduction.
Innovation Solution
The proposed solution involves an image processing method that includes determining saturated pixel channels, computing highlight recovery values based on alternative channels, and applying these values to clipped channels, while also employing refined image processing blocks for demosaicing, tone mapping, and noise reduction to improve image quality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If conventional image processing techniques are applied, then image processing can be performed, but highlight information is lost due to pixel clipping
Solution Approach 1:
The patent uses an intermediary approach by computing highlight recovery values from alternative color channels that are not saturated, and then blending these recovered values with the original clipped channel values. This intermediary computation allows recovery of highlight information that would otherwise be lost, while maintaining color accuracy through controlled blending.
Solution Approach 2:
The patent changes the parameter state by detecting saturation conditions in color channels and selectively applying different processing paths. When saturation is detected, the system transitions from standard processing to highlight recovery processing, adjusting the processing parameters dynamically based on the saturation state of each pixel channel.
2Productivity
If conventional image processing operations are performed, then processing can be completed, but image information is lost in shadows or obscured by highlights
Solution Approach 1:
The patent applies local quality by performing highlight recovery and tone mapping operations at the pixel level rather than globally. Each pixel is processed individually based on its own saturation characteristics, allowing preservation of local highlight and shadow information without affecting the entire image uniformly. This localized approach maintains processing efficiency while preventing information loss.
3Ease of operation
If lookup tables are repeatedly loaded from memory, then color correction can be applied, but processing efficiency decreases
Solution Approach 1:
The patent implements preliminary action by pre-computing and storing highlight recovery values and tone mapping parameters in lookup tables during a preliminary processing stage. These pre-computed values are then rapidly retrieved during main processing without requiring repeated memory loading, significantly improving processing speed while maintaining color correction capability.
4Illumination intensity
If global contrast operations are performed, then overall image contrast is enhanced, but local highlight and shadow details are lost
Solution Approach 1:
The patent applies segmentation by dividing the image processing into separate stages: global contrast enhancement is performed first, then local highlight recovery and tone mapping are applied to specific regions. This segmentation allows global contrast improvement while preserving local detail information through subsequent localized processing of highlights and shadows.
Data Source
AI summary
Image sensors have finite ranges of illuminance that may be captured. When the sensors for particular pixels receive an amount of light exceeding these finite ranges, the pixel values clip to the maximum pixel value. Systems and methods for estimating pixel values that are clipped or near clipping are provided. In one example, a method for processing image data includes determining that a first channel of the image data is saturated or near saturation. The method further includes computing a highlight recovery value for the first channel based upon alternative channels in the image data that are not saturated or near saturation. The highlight recovery value is applied to the first channel.