Modular Siamese Network for Interpretable Face Verification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional face verification systems using deep learning methods are susceptible to adversarial noise and fail to provide interpretable decision-making insights, especially when processing partially occluded images, leading to security concerns and lack of understanding in decision-making processes.
Innovation Solution
A modular Siamese network is implemented for face verification, which includes pre-trained feature extractors, disentangling encoders, and Siamese networks to generate feature-specific disentangling embeddings and compute distance vectors, providing feature-wise Euclidean distances and heatmaps for intuitive and interpretable decision-making, while being robust against adversarial attacks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional deep learning methods are used for face verification, then the system can process images efficiently, but the system becomes susceptible to adversarial noise and lacks interpretable decision-making insights
Solution Approach 1:
The patent segments the face verification process into multiple independent Siamese networks, each responsible for verifying specific facial features (eyes, nose, mouth, etc.). This segmentation allows the system to provide interpretable decision-making insights by showing which specific features contribute to the verification decision, while maintaining robustness through the collective verification of multiple features against adversarial attacks.
Solution Approach 2:
The patent introduces disentangling encoders as intermediary components that separate and independently encode different facial features from the input images. These encoders act as mediators between the raw image input and the verification process, enabling the system to provide feature-level explanations while protecting against adversarial attacks that target the entire image.
2Ease of operation
If conventional deep learning methods are used for face verification, then the system can provide verification output, but the system fails to provide feature-level explanations for decision making
Solution Approach 1:
The verification system is segmented into multiple specialized Siamese networks, each handling specific facial features. This segmentation improves interpretability by providing feature-level explanations, while the modular structure allows each component to remain relatively simple and focused on a specific task.
Solution Approach 2:
Each Siamese network in the modular system serves multiple functions: it verifies a specific facial feature, provides interpretable output for that feature, and contributes to the overall verification decision. This multi-functionality reduces the need for separate complex components for each function.
3Reliability
If conventional deep learning methods are used for face verification, then the system can process images quickly, but the system is susceptible to malicious events leading to security issues
Solution Approach 1:
The verification process is divided into parallel Siamese networks that process different facial features simultaneously. This segmentation enhances security by requiring multiple features to be verified, making it harder for adversarial attacks to succeed, while the parallel processing maintains efficiency.
Solution Approach 2:
The system performs verification on multiple facial features (more than the minimum required), providing a safety margin against adversarial attacks. This excessive verification of features improves security while the parallel processing ensures that the additional checks do not significantly impact processing speed.
Data Source
AI summary
Performance enhancement of face verification systems is credited due to advancement in deep learning methods. However, these systems fail to provide interpretations for decision makings despite their ability to attain high accuracy. Various post-hoc methods have been proposed due to increased demand of deep learning models for better interpretations. However, face verification systems are still prone to adversarial attacks. Present disclosure provides a face verification system and method which addresses the issue of interpretability by employing modular neural network(s), wherein representations for each individual facial feature such as nose, mouth, eyes, etc., are learned separately and verification of input face images is performed. Through experiments, present disclosure demonstrates that the method described herein is resistant to adversarial attacks, thereby addressing another crucial weakness concerning deep learning models.