Deep Learning Image Fingerprinting for Robust Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image matching systems are inadequate for identifying matches in images that have undergone transformations such as rotation, cropping, or color distortion, and require large datasets for classifier training, making them resource-intensive and inefficient.
Innovation Solution
A deep learning image fingerprinting model generates semantically rich feature vectors for images, allowing for robust matching and classification even with distortions, and enables classifier training with a minimal number of images using approximate nearest neighbors models and embedding converters.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional file hashing methods are used for image matching, then exact match speed and reliability are improved, but the ability to match transformed images (rotated, cropped, distorted) deteriorates
Solution Approach 1:
The patent transforms images into a different parameter space using deep learning models. Instead of comparing raw pixel values or simple hashes, the system converts images into feature embeddings that capture semantic content. This parameter transformation allows the system to maintain reliability for exact matches while gaining robustness to transformations, as the embedding space is designed to be invariant to common image modifications.
Solution Approach 2:
The patent replaces traditional mechanical hashing algorithms with deep learning-based embedding models. Rather than using deterministic hash functions that are sensitive to any pixel change, the system employs neural networks that learn robust feature representations. This substitution enables the system to achieve both exact match reliability and transformation robustness by leveraging the pattern recognition capabilities of deep learning models.
2Reliability
If deep learning models are retrained to improve performance, then model accuracy is improved, but embedding compatibility and system continuity deteriorate
Solution Approach 1:
The patent introduces an embedding conversion model as an intermediary between different deep learning model versions. When a model is retrained with improved accuracy, the conversion model translates embeddings from the new model into the format compatible with the original model. This intermediary layer maintains system continuity and embedding compatibility while allowing the underlying model to be updated for improved performance.
Solution Approach 2:
The patent performs preliminary embedding conversion when model updates occur. Instead of allowing compatibility issues to arise after model retraining, the system proactively converts existing embeddings to match the new model's output format. This preliminary action ensures that the updated high-accuracy model can be deployed without disrupting system continuity or breaking existing image matching functionality.
3Reliability
If large training datasets are used for classifier training, then classification accuracy is improved, but training time and processing resources increase
Solution Approach 1:
The patent uses pre-trained deep learning models as copies of knowledge already learned from large datasets. Instead of training classifiers from scratch on large datasets, the system leverages embeddings generated by pre-trained models that have already captured extensive visual patterns. This copying approach allows the system to achieve high classification accuracy without incurring the time and resource costs of training on large datasets.
Solution Approach 2:
The patent performs preliminary training of deep learning models on large datasets to create pre-trained embedding models. This preliminary action extracts and stores knowledge from large datasets in advance. When classification is needed, the system can quickly apply this pre-learned knowledge without retraining, thereby achieving high accuracy while minimizing training time and resource consumption for specific classification tasks.
Data Source
AI summary
Image fingerprints (embeddings) are generated by an image fingerprinting model and indexed with an approximate nearest neighbors (ANN) model trained to identify the most similar fingerprint based on a subject embedding. For image matching, a score is provided that indicates a similarity between the input embedding and the most similar identified embedding, which allows for matching even when an image has been distorted, rotated, cropped, or otherwise modified. For image classification, the embeddings in the index are clustered and the clusters are labeled. Users can provide just a few images to add to the index as a labeled cluster. The ANN model returns a score and label of the most similar identified embedding for labeling the subject image if the score exceeds a threshold. As improvements are made to the image fingerprinting model, a converter model is trained to convert the original embeddings to be compatible with the new embeddings.