RGB-NIR Image Calibration for NIR Decontamination and Edge Alignment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
RGB-NIR sensors face challenges in accurately processing images due to NIR contamination of R, G, and B pixels, leading to distorted colors, and existing methods fail to effectively decontaminate and align edges in RGB-NIR images, resulting in artifacts like overshoots and undershoots.
Innovation Solution
A comprehensive image processing pipeline for RGB-NIR sensors that includes NIR interpolation, decontamination, and Bayer image reconstruction, utilizing photometric distances and demosaicing techniques to preserve edges and correct NIR contamination, with optional temporal subtraction for enhanced accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If NIR pixels are added to capture near-infrared spectrum, then the sensor can capture both visible and NIR bands simultaneously, but the number of visible light pixels is reduced
Solution Approach 1:
The patent combines RGB and NIR sensing capabilities into a single sensor array, merging two separate spectral detection functions into one integrated system. This allows simultaneous capture of both visible and near-infrared bands without requiring two separate sensors, thereby maintaining versatility while optimizing pixel utilization.
Solution Approach 2:
The sensor array is designed to perform multiple functions: capturing visible light (RGB) and near-infrared spectrum simultaneously. Each pixel location can contribute to different spectral bands depending on the filtering applied, making the sensor system universal in its capability to detect multiple wavelengths.
2Adaptability or versatility
If IR-Cut filters are removed to allow NIR spectrum to reach pixels, then NIR capture is enabled, but R, G, and B pixels become contaminated with NIR signal
Solution Approach 1:
The patent segments the pixel array into different functional groups: dedicated NIR pixels and RGB pixels. By spatially separating the detection functions and applying appropriate filtering to specific pixel regions, the system allows NIR signal to reach certain pixels uncontaminated while preventing NIR contamination in RGB pixels, thus maintaining both NIR sensitivity and color accuracy.
Solution Approach 2:
Different regions of the sensor array have different optical properties. The patent applies IR-Cut filters locally to specific pixel locations (RGB pixels) while leaving other pixels (NIR pixels) without filters. This local differentiation allows each region to optimize for its intended function: RGB pixels maintain color accuracy by blocking NIR, while NIR pixels capture near-infrared signal without contamination.
3Device complexity
If simple NIR subtraction is applied to decontaminate RGB pixels, then processing complexity is reduced, but edge artifacts like overshoots and undershoots appear
Solution Approach 1:
The patent performs preliminary alignment and calibration of the NIR channel relative to the RGB channels before decontamination processing. By pre-aligning the NIR image to match the RGB image geometry and applying calibration factors derived from reference measurements, the system prepares the data in advance to prevent edge artifacts during the subsequent subtraction process, thereby maintaining edge accuracy without excessive complexity.
4Manufacturing precision
If photometric distance-based weighting is used for NIR interpolation, then edge preservation is improved, but computational complexity increases
Solution Approach 1:
The patent changes the parameter used for interpolation weighting from simple spatial distance to photometric distance, which incorporates color information. By using photometric distance in the weighting function, the interpolation process adapts to local color variations and preserves edges more effectively. The computational complexity increase is managed by implementing efficient algorithms that leverage the statistical properties of natural images.
Data Source
AI summary
A method for processing images acquired by a multi-spectral RGB-NIR (red/green/blue/near infra-red) sensor includes receiving a RGB-NIR digital image from a multi-spectral RGB-NIR sensor, interpolating an NIR contribution to each R, G and B pixel value, wherein an NIR image is obtained, subtracting the NIR contribution from each R, G and B pixel value in the RGB-NIR digital image wherein a decontaminated RGB-NIR image is obtained, constructing a red, green and blue (RGB) Bayer image from the decontaminated RGB-NIR image, and processing the Bayer image wherein a full color image is obtained. The RGB-NIR digital image includes red (R) pixels, green (G) pixels, blue (B) pixels, and NIR pixels, and every other row in the RGB-NIR digital image includes NIR pixels that alternate with green pixels, and every other row in the RGB-NIR digital image includes green pixels that alternate with red and blue pixels.