Neural Image Processing for Multispectral Illuminant Estimation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image processing technologies using deep learning-based neural networks face challenges in accurately determining illuminant information due to the limited number of color channels captured by general cameras, which affects the precision of image transformations like white balancing.
Innovation Solution
The method involves converting input images with fewer color channels into multispectral images with more channels, generating an illumination map and a confidence score map using neural network models, and fusing these maps to determine illuminant information, leveraging spatial and spectral features through attention mechanisms.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If general cameras with limited color channels are used for image processing, then device complexity is reduced, but measurement precision of illuminant information deteriorates
Solution Approach 1:
The patent transforms the input image from a 2D color image with limited channels into a 3D multispectral image by adding spectral dimension. This is achieved by converting the input image into a multispectral image with additional spectral channels, enabling the neural network to capture both spatial and spectral information simultaneously, thereby improving illuminant estimation accuracy without requiring physically complex hardware
Solution Approach 2:
The patent introduces a neural network-based conversion module as an intermediary that transforms the limited color channel input into enriched multispectral representation. This intermediary processing layer synthesizes additional spectral information from the input image, acting as a bridge between simple camera hardware and the requirement for multispectral data, thus avoiding direct hardware complexity while achieving improved measurement precision
2Measurement precision
If multispectral images with more color channels are used, then measurement precision of illuminant information is improved, but device complexity increases
Solution Approach 1:
The patent replaces the mechanical approach of using physical multispectral cameras with multiple color channels with a computational approach. Instead of requiring complex hardware to capture multispectral data, the system uses a neural network to synthesize multispectral representations from standard color images, substituting mechanical complexity with algorithmic processing
3Measurement precision
If spatial feature extraction is performed on input image, then processing speed is improved, but measurement precision of spectral information deteriorates
Solution Approach 1:
The patent segments the image processing task into separate modules: spatial feature extraction module and spectral feature extraction module. Each module processes specific aspects independently - spatial module handles spatial information from input images while spectral module extracts spectral features from multispectral images. This segmentation allows each module to optimize for its specific function, maintaining processing speed while improving spectral precision
Solution Approach 2:
The patent adds spectral dimension to the processing pipeline by converting input images to multispectral images. This dimensional expansion enables simultaneous extraction of both spatial and spectral features without compromising processing speed, as the neural network processes both dimensions in parallel through the converted representation
Data Source
AI summary
A processor-implemented method including converting an input image based on first sub-images of first color channels into a multispectral image based on second sub-images of second color channels, generating an illumination map representing an illumination configuration of the input image, based on the input image, generating a confidence score map of the illumination map, based on the multispectral image, and determining illuminant information of the input image by fusing the illumination map with the confidence score map, a second number of channels of the second color channels being greater than a first number of channels of the first color channels.