One-shot Multimodal Document Identification via Fingerprint Extraction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data-driven machine learning methods for automatic document classification require a large number of training samples for each class, making it time-consuming and expensive to build a high-quality model, especially when only one or few samples are available.
Innovation Solution
A multimodal model is trained using one-shot learning, where a fingerprint is generated for each template document image by detecting and filtering regions of interest, and then used to identify query documents by generating multimodal feature vectors and applying them to the trained model for prediction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If existing data-driven machine learning methods are used for document classification, then classification accuracy can be achieved, but a large number of training samples are required for each class, making it time-consuming and expensive
Solution Approach 1:
The patent applies preliminary action by pre-processing template document images to extract distinctive fingerprints before actual classification. These fingerprints capture essential document characteristics in advance, enabling the model to perform accurate one-shot classification without requiring numerous training samples. The fingerprint extraction process includes identifying document regions, extracting features, and creating compact representations that can be quickly compared during inference.
Solution Approach 2:
The patent uses copying by creating fingerprint representations that serve as simplified copies of the original document images. Instead of training on full document images, the system extracts and stores compact fingerprint features that capture the essential characteristics. These fingerprint copies enable efficient one-shot learning by allowing direct comparison between query document fingerprints and template document fingerprints without requiring the original full-resolution images.
2Reliability
If more training samples are collected and labeled to improve model quality, then document identification accuracy improves, but the time and cost for data collection and labeling increase
Solution Approach 1:
The patent applies extraction by isolating and removing the time-consuming data collection and labeling steps from the traditional machine learning pipeline. Instead of requiring numerous labeled training samples, the system extracts only the essential fingerprint features from a single template document. This extraction approach eliminates the need for extensive manual annotation while maintaining high classification reliability through the use of distinctive, carefully selected document features.
Solution Approach 2:
The patent changes parameters by transitioning from a data quantity-dependent approach to a feature quality-dependent approach. The system modifies the classification paradigm from requiring many samples with varying parameters to using a single sample with carefully extracted, high-quality fingerprint parameters. This parameter change enables the model to achieve high reliability with minimal training data by focusing on the most discriminative document characteristics.
3Productivity
If traditional machine learning approaches are used, then document classification can be performed, but the process requires extensive training data and computational resources
Solution Approach 1:
The patent applies segmentation by dividing the document classification task into distinct stages: fingerprint extraction from template documents, fingerprint generation from query documents, and fingerprint comparison for classification. This segmentation allows each component to be optimized independently, improving overall productivity while reducing training complexity. The fingerprint extraction stage processes templates offline, the generation stage handles queries efficiently, and the comparison stage provides rapid classification decisions.
Data Source
AI summary
In some embodiments, techniques are provided for document identification using a multimodal model that has been trained using one-shot learning. In one example, a first method of document image processing includes generating, for each template document image of a plurality of template document images, a corresponding fingerprint of a plurality of fingerprints; and based on the plurality of fingerprints, training a multimodal model. For each template document image of the plurality of template document images, generating the corresponding fingerprint may include detecting a plurality of regions within the template document image, wherein the plurality of regions comprises a plurality of text regions; and filtering the plurality of regions to obtain a plurality of regions of interest, wherein the fingerprint is based on the plurality of regions of interest.