Multimodal Fusion Decision Module for Identity Verification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional multi-modal verification systems lack accuracy in identifying individuals, especially in unconstrained environments, and fail to effectively associate non-biometric modality inputs, leading to performance degradation under varying conditions such as lighting and image resolution variations.
Innovation Solution
A novel multimodal fusion decision module architecture that computes performance parameters like F-measure and scoring accuracy to weight raw scores, enabling association between different modality inputs and generating an overall score for improved decision-making, capable of handling both biometric and non-biometric data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional multi-modal verification systems use maximum probable value for each input to make decisions, then the decision-making process is simple, but the accuracy of individual identification deteriorates due to ignorance of modality inputs with less probable values
Solution Approach 1:
The patent segments the decision-making process into distinct functional modules: a score generation module that produces raw scores for each modality, a weight generation module that computes confidence weights based on performance parameters, and a score fusion module that combines weighted scores. This segmentation allows the system to consider multiple modality inputs beyond just the maximum probable value while maintaining organized and manageable system complexity.
Solution Approach 2:
The patent implements preliminary computation of performance parameters (such as F-measure and scoring accuracy) and confidence weights before the final decision-making process. By pre-calculating these weights based on historical performance data, the system prepares the necessary weighting factors in advance, enabling more accurate fusion of multiple modality scores without adding significant real-time computational burden.
2Measurement precision
If the system processes multiple modality inputs with different performance characteristics, then the accuracy can be improved, but the difficulty of detecting and measuring performance parameters increases
Solution Approach 1:
The patent replaces complex manual or manual-like computation of performance parameters with automated statistical computations. The system automatically calculates performance parameters (F-measure, scoring accuracy) and confidence weights using standardized mathematical formulas applied to the score data, eliminating the need for manual analysis and reducing the difficulty of measuring and processing multiple modality performance characteristics.
3Reliability
If existing solutions are used in unconstrained environments with varying conditions, then the implementation is straightforward, but the performance degrades due to lighting variations, distance variations, and image resolution issues
Solution Approach 1:
The patent dynamically adjusts the confidence weights assigned to different modality inputs based on their computed performance parameters. When environmental conditions affect certain modalities (e.g., lighting variations affecting image-based recognition), the system automatically reduces the weight of those degraded inputs and increases reliance on more reliable modalities. This adaptive parameter adjustment maintains system reliability across varying environmental conditions.
Solution Approach 2:
The patent creates a universal decision fusion architecture that can handle multiple types of modality inputs (biometric and non-biometric) with varying performance characteristics. The confidence weight mechanism provides a unified approach to combining diverse input types, making the system adaptable to different environmental conditions and input quality levels without requiring separate processing paths for each modality type.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A computer-based system and method to improve the multimodal fusion output at decision level is disclosed. The method proposes computation of a confidence weighted measure for the individual score values obtained for each modality and fuse these new updated scores to get the final decision. These confidence weights are the performance parameters (measured in terms of F-measure) during the offline training step. The process significantly increases the accuracy of the multimodal system.