Multispectral Decorrelation Model for RGB-IR Signal Separation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Multispectral imaging systems face challenges in accurately decorrelating RGB and IR signals due to high correlation, especially in scenarios with highly IR reflective materials, active IR illumination, and tungsten lamps, leading to unsatisfactory color reproduction and metamerism issues.
Innovation Solution
The use of estimated or measured quantum efficiency curves and dual band-pass filters, combined with large datasets of synthetic reflected light spectra, to predict pixel responses and create a multispectral decorrelation model using Gaussian process regression or other mathematical methods, generating a lookup table for real-time decorrelation of RGB outputs from multispectral inputs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a dual band-pass filter is used to reduce RGB-IR correlation, then the decorrelation problem becomes mathematically solvable, but color reproduction accuracy deteriorates in certain scenarios (highly IR reflective materials, active IR illumination, tungsten lamps)
Solution Approach 1:
The patent changes the parameters of the optical filter system by using a multi-bandpass filter with specific transmission characteristics that allow different wavelength bands (visible and infrared) to pass through at different rates. This enables the system to capture both visible and infrared information while maintaining distinguishable spectral signatures for accurate color reproduction across various lighting conditions.
Solution Approach 2:
The patent introduces an intermediary computational model that uses quantum efficiency curves and synthetic reflected light spectra to predict pixel responses. This intermediary layer bridges the gap between the filtered optical signals and the desired accurate color reproduction, allowing the system to compensate for filter-induced color shifts through mathematical modeling and lookup table-based correction.
2Ease of manufacture
If standard RGB colour filters are used in multispectral sensors, then manufacturing complexity is reduced, but signal decorrelation becomes difficult due to high correlation between visible and multispectral signals
Solution Approach 1:
The patent extracts the decorrelation problem from the optical hardware design and relocates it to the computational processing domain. By using standard RGB filters and adding a computational decorrelation step based on quantum efficiency modeling, the system separates the manufacturing simplicity (standard filters) from the signal processing complexity (computational decorrelation), allowing each to be optimized independently.
Solution Approach 2:
The patent replaces complex mechanical/optical filter design with a computational approach. Instead of designing custom filters with specific transmission characteristics, the system uses standard filters and substitutes the filtering function with a computational model that predicts and corrects spectral responses based on quantum efficiency curves and synthetic spectra.
3Measurement precision
If quantum efficiency curves and synthetic spectra are used to create a decorrelation model, then color correction accuracy is improved, but computational requirements and processing time increase
Solution Approach 1:
The patent performs preliminary computational work by pre-calculating quantum efficiency curves, generating synthetic reflected light spectra, and creating lookup tables based on these models. This preliminary action is done offline during system setup or calibration, so that during actual image capture and processing, the system can quickly retrieve pre-computed correction values from the lookup table, significantly reducing real-time processing time while maintaining high color correction accuracy.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach enables accurate visible RGB pixel responses in various illumination conditions and scene types, improving color correction and reducing noise and artifacts in multispectral imaging systems.
Implementation Method 1
using estimated or measured quantum efficiency curves of the multispectral sensor
Implementation Method 2
use of a dual band-pass filter (also termed a 'notch' filter) in front of the imaging sensor. This transmits wavelengths in the visible spectrum from 400-650 nm, and in the near-infrared spectrum
Implementation Method 3
Multispectral imaging sensors capture information in one or more bands outside the visible spectrum (e.g. near-infrared (NIR) or ultraviolet (UV)), as well as RGB visible spectrum image information
Data Source
AI summary
A method of creating a multispectral decorrelation model for use in determining a visible image from a multispectral image captured using a multispectral image sensor, the method comprising the steps of: generating, using a plurality of quantum efficiency curves for the multispectral image sensor and a plurality of synthetic light spectrum vectors, a grid of synthetic multispectral pixel values and a corresponding grid of synthetic visible pixel values, wherein each synthetic visible pixel value is substantially decorrelated from a non-visible component of a corresponding synthetic multispectral pixel value; and determining a multispectral decorrelation model using the grid of synthetic multispectral pixel values and the corresponding grid of synthetic visible pixel values, wherein the multispectral decorrelation model in use maps a multispectral pixel value of the multispectral image to a visible pixel value of the visible image.