Illuminant Estimation Using Facial Color Features
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Automatic white balance in electronic media devices often fails to accurately estimate illuminant for skin tones due to variations across individuals and similarities with white point locus, leading to errors in professional and consumer cameras, especially in Front-Side camera setups where user's face is a significant portion of the image.
Innovation Solution
The method involves acquiring Facial Color Features Set from the user, which includes skin tone, eyeball white, and teeth color profiles, to conduct automatic white balance illuminant estimation, gain adjustment, and color enhancement, using machine learning and user input to create a personalized color rendition.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If conventional illuminant estimation algorithms (Grey World, Perfect Reflector, Color by Correlation Models) are used, then general white balance processing can be performed, but skin tone accuracy deteriorates due to significant variations across individuals and races and similarity with white point locus
Solution Approach 1:
The system performs preliminary action by acquiring and storing facial color feature sets from users in advance through a facial color features acquisition mode. This pre-acquired data is then used during automatic white balance processing to improve illuminant estimation accuracy for skin tones, avoiding the need to process skin tone variations in real-time during image capture.
Solution Approach 2:
The patent introduces an intermediary approach by using a reference surface method where facial color features serve as a mediator between the unknown illuminant and the image capture. By detecting skin tone regions and using them as reference surfaces with known color properties, the system can estimate illuminant characteristics more accurately without being affected by the diversity of skin tones across different individuals.
2Measurement precision
If face area is removed as noise from illuminant estimation (as proposed in US2008037975), then illuminant estimation errors are reduced, but skin tone rendering quality deteriorates
Solution Approach 1:
The system applies local quality by making different parts of the image undergo different processing. Skin tone regions are identified and processed with specialized algorithms that preserve their color characteristics, while non-skin regions use conventional illuminant estimation methods. This allows the face area to be used as a reference for illuminant estimation rather than being excluded, improving both accuracy and skin tone rendering.
Solution Approach 2:
The patent performs preliminary action by pre-acquiring facial color feature sets through a dedicated acquisition mode. These pre-stored features serve as reference data that enables the system to accurately estimate illuminant characteristics from skin tone regions without causing rendering errors, thus resolving the contradiction between using face area for estimation and maintaining skin tone quality.
3Measurement precision
If a reference color surface (grey chart) is used for illuminant estimation, then white balance accuracy is improved, but device complexity and ease of operation deteriorate due to cumbersome setup requirements
Solution Approach 1:
The system implements self-service by automatically using the user's own facial features as the reference surface for white balance processing. The camera automatically detects skin tone regions, retrieves corresponding facial color features from stored data, and uses these for illuminant estimation without requiring the user to manually position or select any reference objects. This eliminates the complexity of setting up external reference surfaces while maintaining high white balance accuracy.
Solution Approach 2:
The patent uses copying by replacing the need for physical reference surfaces (like grey charts) with digital copies of facial color features stored in memory. Instead of requiring a physical reference object to be present in the scene, the system copies and uses pre-acquired facial color data as the reference, significantly simplifying the operation while maintaining measurement precision.
4Productivity
If conventional white balance processing is applied to Front-Side camera setups, then general image processing is efficient, but skin tone accuracy deteriorates because user's face occupies a significant portion of the image
Solution Approach 1:
The system applies segmentation by dividing the image processing into distinct pathways: skin tone region detection and processing, and non-skin region processing. By segmenting the image to identify facial areas, the system can apply specialized illuminant estimation algorithms specifically to skin tone regions while maintaining efficient conventional processing for other areas, thus improving skin tone accuracy without significantly impacting overall processing efficiency.
Solution Approach 2:
The patent performs preliminary action by pre-acquiring and storing facial color feature sets for each user. This pre-processing allows the system to quickly retrieve appropriate reference data during image capture without adding significant processing time, maintaining productivity while enabling accurate skin tone rendering through personalized reference data.
Data Source
AI summary
Disclosed is a method and apparatus of illuminant estimation referencing characterized facial color features in a digital image. Example embodiments of automatic white balance of the digital image leveraging the illuminant estimation are illustrated.