Multimodal Decorrelated Embedding Model for Biometric Identification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional biometric identification systems using multi-modalities suffer from correlation between output embeddings, leading to decreased accuracy and increased false positive identifications due to common features and traditional training techniques that reinforce correlation.
Innovation Solution
A multimodal decorrelated embedding model is employed, utilizing a machine learning network with an intersection branch and an XOR branch to learn features that are unique to each modality, reducing correlation by explicitly utilizing intersection and XOR features, and minimizing common features across modalities.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional multi-modal embedding models are used, then the system can process multiple modalities, but correlation between output embeddings increases leading to decreased accuracy and increased false positives
Solution Approach 1:
The embedding model is segmented into multiple independent embedding generators, each specialized for processing a specific modality. This segmentation allows each generator to learn modality-specific features independently, reducing correlation between different modalities while maintaining the ability to process multiple modalities simultaneously.
Solution Approach 2:
Each embedding generator is designed with local quality optimization, where the architecture and parameters are tailored specifically for its designated modality. This ensures that each generator produces high-quality embeddings for its specific input type while maintaining independence from other modalities, thereby reducing overall correlation.
2Ease of manufacture
If traditional training techniques are used, then the training process is simple, but correlation between features is reinforced leading to decreased identification accuracy
Solution Approach 1:
The training process incorporates feedback mechanisms where the loss function explicitly penalizes correlation between embeddings from different modalities. This feedback guides the optimization process to reduce inter-modal correlation while maintaining intra-modal consistency, improving identification accuracy without significantly complicating the training workflow.
3Adaptability or versatility
If multiple embedding models are used for different modalities, then each modality can be processed independently, but false positive identifications increase due to correlated outputs
Solution Approach 1:
The system segments the embedding generation process into independent modality-specific generators that produce uncorrelated embeddings. This segmentation maintains adaptability to multiple modalities while reducing false positives through architectural independence.
Solution Approach 2:
The system uses a composite architecture combining multiple specialized embedding generators, each optimized for its specific modality. This composite structure leverages the strengths of each individual generator while the diversity of modalities and their independent processing reduces overall correlation and false positive rates.
Data Source
AI summary
A biometric identification system uses inputs acquired using different modalities. A model having an intersection branch and an XOR branch is trained to determine an embedding using features present in all modalities (an intersection of modalities), and features that are distinctive to each modality (an XOR of that modality relative to the other modality(s)). During training, a first loss function is used to determine a first loss value with respect to the branches. Probability distributions are determined for the output from the branches, corresponding to the intersection and XORs of each modality. A second loss function uses these probability distributions to determine a second loss value. A total loss function for training the model may be a sum of the first loss and the second loss. Once trained, the model may process query inputs to determine embedding data for comparison with embedding data of a previously enrolled user.