Cross-modality Face Matching via Vector Distance Distributions
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing face verification systems struggle with matching faces captured by different sensor modalities and poses, particularly due to challenges in facial landmark localization and alignment, especially when using thermal or depth images which have lower resolution and are less dependent on illumination.
Innovation Solution
A method and system for matching faces across different sensor modalities without using facial landmarks or geometric transformation, by extracting features, determining vector distance distributions, and calculating scores based on these distributions for accurate matching.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If facial landmark-based alignment methods are used for face verification, then matching accuracy for same-modality images is improved, but performance deteriorates when matching across different sensor modalities (thermal, depth) due to lower resolution and different feature characteristics
Solution Approach 1:
The patent changes the fundamental parameters of face representation from landmark-based geometric features to distribution-based spectral features. By representing faces as vectors of distance distributions between codewords in a learned codebook, the system adapts to different sensor modalities without relying on modality-specific alignment cues that fail in thermal and depth images.
Solution Approach 2:
The patent introduces an intermediary codebook learning mechanism that bridges different sensor modalities. The codebook serves as a modality-agnostic feature space where both visible and thermal/depth faces can be represented and compared, eliminating the need for direct alignment between dissimilar modalities.
2Measurement precision
If geometric transformation and alignment are applied to match faces from different poses and viewpoints, then matching performance is improved for frontal faces, but accuracy decreases for faces in extreme poses or occluded views
Solution Approach 1:
The patent segments the face into multiple local patches and learns a codebook for each patch type (eyes, nose, mouth, etc.). This segmentation allows the system to capture pose-invariant local patterns that remain recognizable even when the overall face geometry changes due to extreme poses or partial occlusions.
Solution Approach 2:
The patent applies different processing and codebook learning to different facial regions, making each patch representation optimized for its specific characteristics. This local quality approach ensures that matching relies on robust local features rather than global geometric relationships that fail under pose variation.
3Manufacturing precision
If visible light cameras are used for face verification, then image quality and detail are improved under good illumination, but the system fails completely in dark or low-illumination environments
Solution Approach 1:
The patent creates a universal face verification system that works across multiple sensor modalities (visible, thermal, depth). The codebook-based representation serves all modalities equally, making the system multi-functional and eliminating dependence on any single modality's illumination requirements.
Solution Approach 2:
The codebook acts as an intermediary that translates different sensor modalities into a common representation space. This allows thermal and depth images, which are insensitive to illumination, to be matched with visible images without requiring the visible images to have good lighting conditions.
4Object-affected harmful factors
If thermal infrared or depth cameras are used for face verification in dark environments, then illumination independence is achieved, but matching accuracy decreases due to lower image resolution
Solution Approach 1:
The patent transitions from spatial-domain matching (pixel-based) to feature-space matching (codebook distance-based). By representing faces as vectors in a high-dimensional codebook space rather than comparing raw pixel values, the system compensates for lower resolution in thermal and depth images through learned semantic features.
Solution Approach 2:
The patent changes the matching parameters from direct pixel comparison to distance distribution comparison in codebook space. This parameter transformation allows the system to extract meaningful matching information even from low-resolution thermal and depth images by comparing the statistical distributions of codebook distances rather than individual pixel intensities.
Data Source
AI summary
A computer-implementable method for determining a score for matching images includes extracting and determining. Features are extracted from each of a query image acquired in a first modality and a database image acquired in a second modality. A vector distance distribution is determined based on a codebook for the first modality for each of the query image and the database image represented by the extracted features. The method determines a set of distances between the vector distance distribution determined for the query image and the vector distance distribution determined for the database image. The score for matching the query image and the database image is determined based on the determined set of distances.