Sketch-Based Image Retrieval Using Generative Domain Migration Hashing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current sketch-based image retrieval (SBIR) techniques face challenges in accurately retrieving relevant images due to the abstract nature of free-hand sketches, leading to cross-domain discrepancies and inefficient query times, especially with large datasets.
Innovation Solution
The implementation of a neural network architecture using generative adversarial networks (GANs) that migrates sketches into the image domain, generating synthetic images and hash codes for accurate retrieval, while preserving semantic consistency across domains, thereby improving retrieval accuracy and reducing computational and memory costs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If free-hand sketches are used for image retrieval, then user flexibility and ease of operation are improved, but cross-domain discrepancies between sketches and real-world images increase, reducing retrieval accuracy
Solution Approach 1:
The patent introduces a generative adversarial network (GAN) as an intermediary that translates sketches into the image domain, creating a bridge between the sketch and image domains. This mediator transforms abstract sketches into realistic images that match the database format, thereby resolving the cross-domain discrepancy while preserving user flexibility in sketch-based querying
Solution Approach 2:
The patent changes the domain parameters of sketches by transforming them from abstract line drawings into photorealistic images through the GAN. This parameter transformation includes changing texture, color, lighting, and structural details to match real-world image characteristics, thereby improving retrieval accuracy without sacrificing ease of operation
2Device complexity
If traditional SBIR methods are used, then implementation simplicity is maintained, but retrieval accuracy on large datasets with large variances deteriorates
Solution Approach 1:
The patent performs preliminary action by pre-translating sketches into the image domain using a trained GAN before the actual retrieval process. This preliminary transformation ensures that all queries are converted to a consistent image format that matches the database, improving accuracy across large datasets with diverse variances while maintaining relatively simple implementation through a standardized pipeline
3Measurement precision
If exact matching between sketches and images is pursued, then retrieval precision is improved, but query time increases exponentially with database size and representation dimensions
Solution Approach 1:
The patent creates copies of sketches in the image domain through the GAN translation process. These synthetic image copies preserve the semantic content of original sketches while matching the visual characteristics of database images, enabling efficient comparison and retrieval without requiring computationally expensive exact matching of high-dimensional representations
4Measurement precision
If high-dimensional sketch and image representations are used, then retrieval accuracy is improved, but computational complexity and memory costs increase
Solution Approach 1:
The patent creates equipotentiality between sketch and image domains by translating both to the same image domain representation. This unifies the feature space and reduces the need for complex high-dimensional transformations, thereby maintaining retrieval accuracy while reducing computational complexity and memory requirements through consistent dimensional representation
Data Source
AI summary
This disclosure relates to improved sketch-based image retrieval (SBIR) techniques. The SBIR techniques utilize a neural network architecture to train a domain migration function and a hashing function. The domain migration function is configured to transform sketches into synthetic images, and the hashing function is configured to generate hash codes from synthetic images and authentic images in a manner that preserves semantic consistency across the sketch and image domains. The hash codes generated from the synthetic images can be used for accurately identifying and retrieving authentic images corresponding to sketch queries, or vice versa.