Adaptive RGB-IR Sensor Correction for Artifact-Free Occupant Monitoring
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional image signal processors (ISPs) struggle with accurately processing image data containing substantial infrared (IR) components, leading to image artifacts and signal-to-noise ratio degradation due to inconsistent IR subtraction across varying scenes, particularly in RGB-IR image streams.
Innovation Solution
Implementing a locally adaptive IR correction function in the ISP pipeline that adjusts IR subtraction based on tonal and IR-to-color ratio metrics within a defined local support region for each pixel, using scaling factors to retain residual color information and prevent artifacts.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional ISP pipelines apply uniform IR subtraction across the image, then processing is simple and fast, but image artifacts and signal-to-noise ratio degradation occur due to inconsistent IR components across varying scenes
Solution Approach 1:
The patent implements locally adaptive IR correction by dividing the image into local support regions and computing separate correction factors for each region based on local tonal and color metrics. This allows different parts of the image to be corrected according to their specific characteristics, eliminating the need for uniform correction and thereby preventing image artifacts while maintaining processing efficiency.
2Measurement precision
If aggressive IR subtraction is applied to remove IR components, then IR contamination is reduced, but critical color information is lost leading to degraded color accuracy
Solution Approach 1:
The patent dynamically adjusts the IR correction strength by computing local correction factors based on tonal metrics and color channel ratios. Instead of applying fixed aggressive subtraction, the system adapts the correction parameter (subtraction amount) according to local image characteristics, preserving color information in regions where it exists while effectively removing IR contamination in regions where it dominates.
3Loss of information
If minimal IR subtraction is applied to preserve color information, then color fidelity is maintained, but IR artifacts persist in regions with high IR contamination
Solution Approach 1:
The system applies different correction strategies to different local regions based on their characteristics. In regions with high IR contamination, stronger correction is applied to remove artifacts, while in regions with dominant color information, milder correction preserves fidelity. This local differentiation resolves the contradiction by allowing both color preservation and artifact removal in appropriate contexts.
4Measurement precision
If pixel-by-pixel IR correction is applied, then correction precision is maximized, but computational complexity and processing time increase significantly
Solution Approach 1:
The patent segments the image processing into local support regions rather than treating each pixel independently or the entire image uniformly. This segmentation allows correction factors to be computed for regions rather than individual pixels, reducing computational complexity while maintaining the precision benefits of localized correction. The local support region approach balances precision and efficiency by operating at an intermediate scale.
Data Source
AI summary
In various examples, infrared correction for visible wavelength color data channel processing systems and applications are provided. An ISP pipeline may define a local support region for a pixel that is being processed for IR correction. A locally adaptive IR correction function computes color channel and IR channel value estimates for a target pixel based on spatial filtering of pixels in the local support region. Localized corrections may be applied based on local color channel metrics derived from the local support region. The IR correction function may apply a first scaling factor to the IR value estimate prior to subtraction from the initial color value estimates for the RGB color channels to retain residual color information that otherwise might be lost. Other scaling factors may be applied to restore a saturated RGB color channel to a saturated value due to high RGB color levels.