Highlight Removal in Face Recognition via Shadow Vector Estimation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The generation of an illumination model from a registration image that includes a specular reflection component or shadow deteriorates recognition accuracy in face image recognition systems, as these components can interfere with the matching process between the registration image and the recognition target image.
Innovation Solution
An image processing device comprising a shadow base calculating unit, a perfect diffuse image estimating unit, and a highlight removing unit, which calculates shadow base vectors, estimates shadows, and removes specular reflections from the registration image to generate illumination base vectors that are free from the influence of specular reflections, thereby improving recognition accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If an illumination model is generated from a registration image that includes a specular reflection component or shadow, then the illumination model can be created using the original image data, but the recognition accuracy deteriorates due to interference from these components
Solution Approach 1:
The patent segments the registration image into multiple components: a perfect diffuse image component (representing the base reflectance) and a specular reflection component. By separating these components, the system can eliminate the harmful specular reflection while preserving the useful diffuse reflection information for accurate recognition.
Solution Approach 2:
The patent extracts the specular reflection component from the registration image and removes it to create a cleaned image for illumination model generation. This extraction process isolates the harmful element (specular reflection) and separates it from the useful information, thereby improving recognition accuracy.
2Measurement precision
If a complex skin reflectance model with many parameters is used to achieve high accuracy, then the model can capture detailed skin characteristics, but the computation amount and number of required images increase
Solution Approach 1:
The patent changes the parameter representation by assuming uniform albedo (reflectance) across the skin surface, thereby reducing the number of parameters from a complex spatially-varying reflectance model to a single uniform value. This simplification dramatically reduces computation while still achieving sufficient accuracy for recognition purposes.
Solution Approach 2:
The patent applies different quality assumptions to different aspects of the model: while the overall albedo is assumed uniform (simple), the illumination model captures local variations in lighting conditions through the shadow base vector. This selective application of complexity where needed reduces overall computational burden.
3Ease of manufacture
If a perfect diffuse model is used instead of a complex skin reflection model, then the computation amount is reduced and the model is easier to use, but the ability to reproduce fine shadows and texture is lost
Solution Approach 1:
The patent performs preliminary action by calculating shadow base vectors from the registration image and three-dimensional shape data before generating the illumination model. These pre-computed shadow base vectors capture the essential shadow patterns, which are then used to construct the illumination model, ensuring accurate shadow reproduction without requiring complex real-time calculations.
Solution Approach 2:
The patent introduces shadow base vectors as an intermediary representation that bridges the simple perfect diffuse model and the need for accurate shadow reproduction. These vectors serve as a compact intermediate form that encodes shadow information, allowing the simple model to achieve accurate shadow effects through linear combination of the base vectors.
Data Source
AI summary
An image processing device calculates, from a registration image representing a photographed object and three-dimensional shape data in which respective points of a three-dimensional shape of the object are correlated with pixels of the registration image, by assuming uniform albedo, a shadow base vector group having components from which an image under an arbitrary illumination condition can be generated through linear combination. A shadow in the registration image is estimated with using the vector group. A perfect diffuse component image including the shadow is generated, and based on the image a highlight removal image is generated in which a specular reflection component is removed from the registration image. Thus, an image recognition system generates illumination base vectors from the highlight removal image and thereby can obtain the illumination base vectors based on which an accurate image recognition process can be carried out without influence of a specular reflection.