Multispectral Image Sensing for Deep-Learning White Balance
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image sensors struggle with accurately decomposing illumination and reflectance components due to overlapping wavelength information in RGB spaces, leading to improper color correction and white balance, especially when assumptions like the Gray World Assumption are not satisfied.
Innovation Solution
Utilizing a multispectral image sensor and deep learning network to estimate illumination information, allowing for accurate decomposition of illumination components into a diagonal matrix, and performing color conversion using a pre-trained deep learning network to enhance white balance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If RGB image sensors are used for color capture, then the device complexity is low and ease of manufacture is high, but the measurement precision of illumination decomposition deteriorates due to overlapping wavelength information
Solution Approach 1:
The patent transitions from traditional RGB three-channel wavelength division to multispectral imaging that captures information across multiple spectral dimensions. By adding spectral dimensionality beyond the basic RGB bands, the system achieves more precise illumination decomposition while managing complexity through efficient data processing architectures.
Solution Approach 2:
The patent segments the spectrum into multiple discrete wavelength bands beyond the traditional RGB ranges. This segmentation allows independent measurement of different spectral components, enabling accurate decomposition of illumination sources while using modular sensor designs that balance precision requirements with manufacturing feasibility.
2Measurement precision
If deep learning networks are used for illumination estimation, then the measurement precision of white balance improves, but the loss of time for processing increases
Solution Approach 1:
The patent implements pre-training of deep learning networks using large-scale multispectral datasets before deployment. This preliminary action prepares the model in advance, allowing it to perform rapid illumination estimation during actual operation. The heavy computational lifting is done beforehand, enabling real-time or near-real-time white balance correction.
Solution Approach 2:
The patent replaces traditional iterative optimization methods for illumination estimation with pre-trained deep learning inference. This substitution transitions from computationally intensive mechanical optimization processes to efficient neural network forward propagation, significantly reducing processing time while maintaining or improving precision.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enables more accurate expression of object colors by reducing wavelength overlap, improving white balance and color correction, particularly in challenging lighting conditions.
Implementation Method 1
Image sensors receive light incident from an object and photoelectrically convert the received light into an electrical signal
Data Source
Figure 1A~1B
Figure 2
Figure 3A
AI summary
An image acquisition apparatus includes: a multispectral image sensor configured to acquire images in at least four channels based on a second wavelength band of about 10 nm to about 1,000 nm; and a processor configured to estimate illumination information of the images by inputting the images of at least four channels to a deep learning network trained in advance, and convert colors of the acquired images using the estimated illumination information.