Fine-Grained Sketch Image Retrieval via Feature Graph Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image retrieval methods using sketches are limited in accuracy, often returning general results within broad scene categories, failing to effectively retrieve specific target images due to lack of detailed object comparison in scene sketches.
Innovation Solution
The implementation of a fine-grained image segmentation method using convolution neural networks and feature graph generation to segment and compare detailed attributes of objects within scene sketches, enabling accurate retrieval of target images from large unannotated image collections.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If keyword semantic searching or scene category name matching is used, then the search process is simple and fast, but the retrieval accuracy is low and only general results are obtained
Solution Approach 1:
The patent segments the scene image into multiple object instances with fine-grained attributes (category, size, location, visual features). Each object instance is independently analyzed and represented as nodes in a feature graph, enabling detailed comparison rather than treating the entire image as a single unit. This segmentation approach directly improves retrieval accuracy by comparing specific object details rather than general scene categories.
Solution Approach 2:
The patent transforms the image retrieval problem from traditional pixel-space or feature-vector-space comparison into graph-space comparison. By representing objects as nodes and their relationships as edges in a feature graph, the system adds a structural dimension to the comparison process, enabling more precise matching of object instances and their attributes beyond simple categorical matching.
2Measurement precision
If fine-grained image segmentation and feature graph generation are used, then the retrieval accuracy is improved, but the computational complexity increases
Solution Approach 1:
The patent performs fine-grained segmentation and feature extraction as preliminary actions before the actual retrieval comparison. By pre-processing images into structured feature graphs with object instances, attributes, and relationships, the system prepares data in an optimized format that facilitates efficient comparison during retrieval operations, reducing computational burden during the actual search process.
Solution Approach 2:
The patent changes the parameter representation from traditional image features (pixels, color histograms) to graph-based parameters (object attributes, relationships, spatial configurations). This parameter transformation enables more meaningful comparisons with lower computational cost, as graph structures allow for selective comparison of relevant object attributes rather than processing entire image data.
3Measurement precision
If traditional sketch-based image retrieval is used, then the method is simple to operate, but it cannot effectively compare detailed object attributes and returns general results
Solution Approach 1:
The patent creates a structured feature graph copy of the scene image that preserves all essential object information (instances, attributes, relationships) in a simplified computational format. This graph representation serves as an efficient surrogate for the original image, enabling detailed object attribute comparison without requiring complex image processing operations during retrieval, thus maintaining ease of operation while improving precision.
Data Source
AI summary
A sketch-based image retrieval method, device and system, to improve accuracy of image searching from a scene sketch image. For example, the image retrieval method, device and system can be used to retrieve a target scene image from a collection of stored images in a storage (i.e., an image collection). The image retrieval method includes: segmenting the scene sketch image using an image segmentation module into semantic object-level instances, and fine-grained features are obtained for each object instance, generating an attribute graph which integrates the fine-grained features for each semantic object instance detected from the query scene sketch image, generating a feature graph by using a graph encoder module from the attribute graph, and computing a similarity or distance between the feature graphs of the query scene sketch image and the scene images in the image collection by a graph matching module and the most similar scene images are returned.