Continuous Indexing for Image Matching Systems

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Traditional indexing methods for image features in image matching and visual systems face challenges such as high dimensionality, discontinuity, and sensitivity to changes in image appearance and lighting, leading to inefficiencies and inaccuracies in image retrieval.

Innovation Solution

A continuous indexing system is implemented, comprising ingestion and image retrieval servers that dynamically update a feature database with new images, allowing for real-time matching and retrieval of images with similar visual content, using models like SIFT, GLOH, HOG, and CLIP for feature extraction and matching.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If traditional indexing methods (bag-of-visual-words, local feature descriptors) are used, then image retrieval can be performed, but the system suffers from high dimensionality, discontinuity, and sensitivity to changes in image appearance and lighting

Engineering Contradiction:
Improveimage retrieval accuracyVSAvoidindexing system complexity
Core Design Contradiction:
ReliabilityVSDevice complexity

Solution Approach 1:

The patent transforms image features from traditional discrete descriptors (bag-of-visual-words, local feature descriptors) to continuous embedding vectors generated by deep neural networks. This parameter transformation enables the system to capture semantic information more effectively, improving retrieval accuracy while reducing sensitivity to appearance and lighting changes. The continuous nature of the embeddings allows for smoother feature spaces and better generalization.

Inventive Principle:
Principle #35Parameter changes

Solution Approach 2:

The patent replaces traditional mechanical indexing mechanisms with a neural network-based embedding system. Instead of using hand-crafted features and discrete visual words, the system uses learned continuous representations from deep neural networks. This substitution eliminates the discontinuity and high dimensionality problems of traditional methods while maintaining scalability.

Inventive Principle:
Principle #28Mechanics substitution (Replace mechanical system)

2Productivity

If static image feature index is used, then indexing can be performed, but the system is unable to match to new images without the index being completely rebuilt

Engineering Contradiction:
Improveimage retrieval efficiencyVSAvoidindex rebuilding time
Core Design Contradiction:
ProductivityVSLoss of time

Solution Approach 1:

The patent implements a dynamic indexing system where the feature index is continuously updated with new images rather than being static. The system maintains an active index that can accommodate new images without requiring complete rebuilding. This dynamic approach allows the system to adapt to changing image data while maintaining retrieval efficiency, as the index structure remains stable while individual entries are updated incrementally.

Inventive Principle:
Principle #15Dynamics

Solution Approach 2:

The patent pre-processes and indexes images in advance, creating a searchable feature index that can be queried efficiently. The system performs preliminary feature extraction and indexing operations so that when new images arrive, they can be matched against the existing index without requiring full reindexing. This preliminary action enables fast retrieval while the system continues to update with new data.

Inventive Principle:
Principle #10Preliminary action

3Measurement precision

If deep learning models (CNNs, feature embeddings) are used for image feature extraction, then performance in image classification, object detection, and semantic segmentation improves, but the problem of indexing and querying high-dimensional and continuous data remains challenging

Engineering Contradiction:
Improveimage analysis accuracyVSAvoidindexing and querying complexity
Core Design Contradiction:
Measurement precisionVSDevice complexity

Solution Approach 1:

The patent extracts the most relevant information from high-dimensional neural network embeddings by projecting them into a lower-dimensional searchable index. Instead of querying the full high-dimensional embedding space directly, the system extracts key features and creates a compressed index structure that maintains the essential semantic information while reducing computational complexity for retrieval operations.

Inventive Principle:
Principle #2Taking out (Extraction)

Solution Approach 2:

The patent introduces an intermediate indexing layer between the neural network embeddings and the retrieval query. This intermediate structure acts as a mediator that translates high-dimensional continuous data into a format suitable for efficient searching. The intermediary index structure enables fast approximate nearest neighbor search while preserving the semantic meaning of the deep learning features.

Inventive Principle:
Principle #24Intermediary (Mediator)

Data Source

PatentUS20240403352A1Continuous indexing strategy for image matching and visual search systems
Publication Date: 2024.12.05 SNAP INC
  • US20240403352A1 patent drawing
  • US20240403352A1 patent drawing
  • US20240403352A1 patent drawing

AI summary

A continuous indexing system configured to enable the continuous indexing of image features for use in image matching and visual systems. In certain embodiments, the system comprises a plurality of servers, wherein the plurality of servers include a first server for extracting and writing image features to a database, and a second server configured for querying the database and updating image retrieval artifacts.