Attribute Similarity Search Using Convolutional Neural Networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional keyword-based search methods struggle to accurately refine search results based on visual attributes, particularly for subjective and difficult-to-quantify features like fashion details, leading to missed relevant content and inaccurate classifications.
Innovation Solution
The system employs convolutional neural networks and attribute similarity pipelines to analyze and refine search results based on visual attributes by localizing and extracting features from images, allowing users to specify attributes for similarity searches without relying on exact keywords, and uses n-dimensional attribute spaces for ranking and recommending visually similar items.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If keyword-based search methods are used, then search queries can be submitted and results returned, but the accuracy and relevance of search results for visual attributes deteriorate
Solution Approach 1:
The patent replaces keyword-based text processing with image-based visual attribute analysis using convolutional neural networks. Instead of relying on users to formulate precise keyword queries, the system processes images directly to extract visual attributes, substituting the mechanical keyword-matching system with an automated visual recognition system that understands subjective visual features.
Solution Approach 2:
The patent introduces visual attribute embeddings as an intermediary representation between images and search results. These embeddings capture visual characteristics in a structured format that enables accurate similarity comparison, serving as a mediator that translates visual information into searchable data without requiring explicit keyword formulation.
2Reliability
If conventional keyword-based approaches are used, then search queries can be processed, but the ability to locate relevant content based on subjective visual attributes deteriorates
Solution Approach 1:
The patent replaces the manual query formulation process with automated visual attribute extraction. Users simply upload or select images, and the convolutional neural network automatically extracts relevant visual attributes, eliminating the need for users to understand or articulate complex visual characteristics through keywords.
Solution Approach 2:
The system performs self-service by automatically extracting visual attributes from input images without requiring user intervention to specify search parameters. The convolutional neural network autonomously identifies and extracts relevant visual features, and the similarity search automatically retrieves matching content based on these extracted attributes.
3Measurement precision
If keyword-based search is used, then search results can be returned, but the accuracy of classifying and refining results based on visual attributes deteriorates
Solution Approach 1:
The patent replaces keyword-based classification with visual attribute-based classification using convolutional neural networks. The system processes images directly to extract and compare visual attributes, substituting the text-based classification mechanism with a visual recognition system that preserves and utilizes actual visual information rather than relying on potentially inaccurate textual descriptions.
Solution Approach 2:
The patent transforms visual attribute data into an n-dimensional embedding space where similarity can be measured geometrically. This dimensional transformation allows the system to capture complex visual relationships that cannot be expressed through traditional keyword categories, preserving nuanced visual information through coordinate representations in high-dimensional space.
Data Source
AI summary
A set of training images is obtained by analyzing text associated with various images to identify images likely demonstrating a visual attribute. Localization can be used to extract patches corresponding to these attributes, which can then have features or feature vectors determined to train, for example, a convolutional neural network. A query image can be received and analyzed using the trained network to determine a set of items whose images demonstrate visual similarity to the query image at least with respect to the attribute of interest. The similarity can be output from the network or determined using distances in attribute space. Content for at least a determined number of highest ranked, or most similar, items can then be provided in response to the query image.