Category-Based Image Relevance Filtering for Outlier Removal
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current web search systems often provide irrelevant images in search results due to reliance on visual similarity, failing to accurately determine semantic relevance and removing unrelated images.
Innovation Solution
An image relevance system using category-specific embeddings and relevance thresholds to identify and remove outlier images by generating text and image embeddings, determining similarity scores, and applying category-specific relevance thresholds to ensure semantic relevance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If visual similarity is used to identify related images, then image retrieval speed is improved, but image relevance accuracy deteriorates
Solution Approach 1:
The patent segments the image relevance determination into multiple independent components: category label embedding, image embedding, and similarity score calculation. Each component processes specific aspects of relevance independently, allowing for optimized performance in each segment while maintaining overall accuracy.
Solution Approach 2:
The patent introduces an intermediary similarity score calculation layer between category labels and image embeddings. This intermediary step computes cosine similarity scores that mediate the relationship between textual categories and visual data, enabling accurate relevance determination without direct visual comparison.
2Measurement precision
If category-based embedding models are used to determine semantic relevance, then image relevance accuracy is improved, but computational cost increases
Solution Approach 1:
The patent applies partial action by computing embeddings and similarity scores only for images that pass initial filtering thresholds, rather than processing all images uniformly. This selective computation reduces overall computational cost while maintaining high accuracy for relevant images.
Solution Approach 2:
The patent changes parameters by using category-specific relevance thresholds that are dynamically adjusted based on the particular category being searched. This allows the system to optimize computational resources by applying different threshold stringent levels to different categories, reducing unnecessary computations for less ambiguous categories.
3Quantity of substance
If current web search systems provide images tagged to a topic, then image quantity is improved, but image relevance quality deteriorates
Solution Approach 1:
The patent extracts and removes irrelevant images from the search results by computing similarity scores and comparing them against category-specific thresholds. Images below the threshold are extracted (removed) from the final results, ensuring only relevant images are provided to the user while maintaining adequate quantity through targeted retrieval.
Data Source
AI summary
This disclosure describes a framework for determining the category-based image relevance of digital images associated with entities or topics. Specifically, this disclosure describes an image relevance system that determines outlier images within a set of images associated with an entity or topic by correlating semantic content with visual content. For example, the image relevance system ensures that only images relevant to the entity or topic are provided in response to a user query about the entity or topic. The image relevance system can also filter out images from an image set that do not correspond to user input in a search query before providing the image set. Furthermore, the image relevance system can prevent irrelevant images from being added to an image set associated with an entity or topic.