Multimodal Hand Scanner Embedding via Patch Aggregation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional biometric identification systems face issues with accuracy and speed due to reliance on single images, which may be poorly illuminated, out of focus, or insufficiently detailed, leading to susceptibility to fraud and operational limitations.
Innovation Solution
A system that acquires a set of images using multiple modalities over a short interval, processing these images with convolutional neural networks to generate aggregate data and subsequently determine a single embedding vector for accurate user recognition, incorporating multimodal image data and dynamic hand poses.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a single image is used for biometric identification, then the processing speed is fast, but the recognition accuracy is poor and the system is susceptible to fraud
Solution Approach 1:
The patent combines multiple images of the same biometric object into a single composite representation. The aggregation module merges corresponding patches from multiple input images to create aggregate data, which is then used to generate a unified embedding vector. This merging process improves recognition accuracy by integrating information from multiple sources while maintaining computational efficiency through the unified embedding approach.
Solution Approach 2:
The patent segments the biometric object into multiple patches or regions of interest. Each patch is processed individually from multiple images, and the results are aggregated. This segmentation allows the system to focus on specific discriminative features while robustly handling variations in illumination, pose, and quality across different images.
2Measurement precision
If multiple images are processed to improve recognition accuracy, then the embedding quality improves, but the processing time increases
Solution Approach 1:
The system performs preliminary processing of individual images by extracting patches and creating intermediate representations before aggregation. This preliminary action prepares the data in a standardized format that facilitates efficient combining, reducing the overall processing time compared to handling multiple complete images separately.
Solution Approach 2:
The patent extracts only the essential discriminative features from multiple images by selecting and aggregating corresponding patches rather than processing entire images. This extraction approach maintains embedding quality by focusing on relevant features while significantly reducing processing time and computational resources required.
3Reliability
If traditional single-image biometric systems are used, then the system is simple to operate, but it is susceptible to fraud and has operational limitations
Solution Approach 1:
The system proactively captures multiple images of the biometric object before processing, creating a buffer of redundant information that cushions against fraudulent attempts. By having multiple independent observations available for aggregation, the system can robustly distinguish genuine biometric data from spoofing attempts, thereby improving fraud resistance while maintaining operational simplicity through automated multi-image capture.
Data Source
AI summary
A scanner acquires a set of images of a hand of a user to facilitate identification. These images may vary, due to changes in relative position, pose, lighting, obscuring objects such as a sleeve, and so forth. A first neural network determines output data comprising a spatial mask and a feature map for individual images in the set. The output data for two or more images is combined to provide aggregate data that is representative of the two or more images. The aggregate data may then be processed using a second neural network, such as convolutional neural network, to determine an embedding vector. The embedding vector may be stored and associated with a user account. At a later time, images acquired from the scanner may be processed to produce an embedding vector that is compared to the stored embedding vector to identify a user at the scanner.