Biometric Feature Disentanglement for Activity-Based Person Identification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing person identification methods, particularly facial recognition and whole-body identification, struggle with scenarios where faces are not visible or obscured, and current video-based approaches fail to accurately identify individuals performing diverse daily activities due to appearance biases and lack of robustness across varying conditions.
Innovation Solution
A novel approach using a bias-less silhouette teacher neural network and a student biometric feature-extraction network that disentangles biometric features from non-biometric features through iterative training and geometric distortion, combined with a multimodal pipeline employing a query-transformer module and vision-language model to extract stable biometric embeddings.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If facial recognition techniques are used for person identification, then identification accuracy is improved in visible-face scenarios, but identification fails when faces are not visible or obscured
Solution Approach 1:
The patent segments the identification task into multiple feature types: biometric features (gait patterns, body shape) and non-biometric features (clothing, background). By separating these feature extraction processes and using dedicated neural network branches, the system can identify individuals even when facial features are unavailable, resolving the contradiction between accuracy in visible scenarios and adaptability to obscured scenarios
Solution Approach 2:
The system creates a universal identification framework that works across multiple scenarios (visible face, obscured face, different activities) by incorporating multiple feature extraction pathways. The biometric feature extraction network and non-biometric feature extraction network operate in parallel, allowing the system to adapt to different identification scenarios without requiring separate systems, thus achieving both high accuracy and broad adaptability
2Measurement precision
If whole-body identification methods focusing on gait recognition are used, then identification is effective in controlled environments, but accuracy decreases in diverse daily activity scenarios
Solution Approach 1:
The patent implements dynamic activity recognition that adapts to different daily activities (walking, sitting, bending, etc.) by using an activity recognition neural network that processes video sequences and identifies current activities. This dynamic adaptation allows the system to maintain accurate biometric feature extraction across varying activities, resolving the contradiction between controlled environment performance and diverse scenario adaptability
Solution Approach 2:
The system changes operational parameters by adjusting feature extraction weights and processing methods based on detected activity type. When different activities are detected, the system modifies how biometric and non-biometric features are weighted and processed, enabling accurate identification across diverse activities while maintaining the precision achieved in controlled gait recognition scenarios
3Loss of information
If video-based methods using LSTM or 3D CNNs are used, then spatio-temporal features are captured, but appearance biases from clothing and background affect identification accuracy
Solution Approach 1:
The patent extracts and separates biometric features from non-biometric features using dedicated neural network branches. The biometric feature extraction network focuses specifically on identity-related features (gait, body shape) while the non-biometric feature extraction network handles appearance-related features (clothing, background). This extraction and separation eliminates the harmful effect of appearance biases on identification accuracy while preserving spatio-temporal information
Solution Approach 2:
The patent introduces an intermediary mechanism in the form of separate neural network branches that mediate between the input video data and the final identification decision. The biometric feature extraction network acts as an intermediary that filters out appearance biases by focusing only on biometrically relevant features, while the non-biometric feature extraction network handles the appearance-related information separately, preventing it from interfering with identity recognition
Data Source
AI summary
A system and method for person identification from video data by disentangling biometric identity features from non-biometric appearance and activity features are disclosed. The system processes RGB video sequences depicting individuals performing various activities to extract spatio-temporal features. These features are separated into distinct biometric identity representations and non-biometric features related to appearance and performed activities. To achieve this separation and minimize appearance bias, the system utilizes an auxiliary supervisory model. At least two implementations of this supervisory model are disclosed: one using semantic supervision via structured embeddings processed through a vision-language model, and another employing silhouette-based feature distillation from a silhouette-trained neural network. Joint training for biometric identification and activity classification ensures accurate identification of individuals independently of facial visibility, clothing differences, or activity variations.