Image Core-Area Matching for Fine-Grained Similarity Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image processing techniques struggle to detect fine-grained similarity among images within the same class, as they either fail to discern detailed differences or require continual retraining for new content, making them impractical.
Innovation Solution
A method involving generating an image salient map to identify a core area, using dense scale invariant feature transformers (D-SIFT) to create feature descriptors, and forming capsule vectors through dynamic routing, followed by comparing these vectors to determine a top-K matrix and calculating bundled similarity scores for fine-grained image matching.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If global features are extracted using techniques such as HOG or SIFT, then image similarity can be measured across different classes, but fine-grained similarities among images within the same class cannot be discerned
Solution Approach 1:
The patent segments the image into multiple local regions using a grid structure, extracting features from each region independently rather than treating the entire image as a single global feature. This segmentation enables detection of fine-grained similarities by comparing local region features while maintaining the ability to handle images within the same class.
2Reliability
If supervised machine learning is used for image similarity, then specific tasks perform well, but the model requires continual retraining when search images change
Solution Approach 1:
The patent employs unsupervised learning techniques where the system automatically learns image similarity metrics without requiring manual annotation or retraining. The algorithm self-adjusts to new search images by dynamically determining core areas and extracting features adaptively, eliminating the need for continual retraining while maintaining reliable performance across different tasks.
3Productivity
If global features are used to represent images as single vectors, then computation is efficient, but images within the same class with low information levels cannot be differentiated
Solution Approach 1:
The patent transitions from representing images as single global vectors to a multi-dimensional representation where each local region contributes separate feature vectors. These local feature vectors are then aggregated to form a comprehensive image representation, adding dimensional complexity that enables discrimination of fine-grained similarities while maintaining computational efficiency through vector operations.
Data Source
AI summary
Detecting fine-grained similarity in image includes determining a core area of a search image by generating an image salient map from a plurality of layers of the search image and determining a connected area based on the image salient map. Feature descriptors are generated from the core area of the search image. A plurality of capsule vectors are generated from different ones of a plurality of keypoints of the feature descriptors. Capsule vectors of the search image are compared with capsule vectors of each image of the dataset to generate a top-K matrix. Similarity scores for the top-K matrix are calculated. One or more image of the dataset having fine-grained similarity with the search image are selected based a bundled similarity score for each image of the dataset. The bundled similarity score is a summation of the similarity scores of the image.