Object-Sensitive Image Search Using Multi-Instance Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Text-based search engines fail to accurately retrieve images based on visual content, leading to inaccurate results for image-based queries as they rely solely on textual information rather than visual information.
Innovation Solution
Implementing object-sensitive image search systems that utilize multi-instance learning (MIL) to re-rank images based on the prominence of objects within the images, either through an off-line system building pre-trained MIL models or an on-line system training models 'on the fly' using pseudo-positive and pseudo-negative samples.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If text-based search engines are used to retrieve images, then the system is simple and fast, but the search accuracy is low because visual information is not considered
Solution Approach 1:
The patent segments the image search problem into two independent components: text-based retrieval (using traditional search engines) and visual-based re-ranking (using MIL models). This segmentation allows each component to specialize - the text-based system provides initial candidates efficiently, while the visual system refines results accurately, resolving the contradiction between simplicity and accuracy.
Solution Approach 2:
The patent introduces multi-instance learning (MIL) models as an intermediary component between the text-based search engine and the final results. The MIL model acts as a mediator that takes text-based search results and re-ranks them based on visual content analysis, enabling accurate image search without requiring complete redesign of the entire search system.
2Measurement precision
If multi-instance learning models are trained offline using free public information, then the model can provide high precision, but the training time and computational resources are significant
Solution Approach 1:
The patent applies preliminary action by training MIL models offline using free public image information before actual query processing. This pre-training phase, though time-consuming, creates ready-to-use models that can quickly and accurately re-rank search results during online operations, separating the heavy computational burden from real-time service.
Solution Approach 2:
The patent changes the operational parameters of the MIL model by switching between offline training mode (using free public data with high computational resources) and online inference mode (using pre-trained models with minimal computational overhead). This parameter switching allows the system to achieve high precision while managing training time constraints.
3Adaptability or versatility
If online model training is performed using pseudo-positive and pseudo-negative samples, then the system adapts to user queries in real-time, but the quality of training data may be lower
Solution Approach 1:
The patent implements dynamics by enabling the MIL model to be retrained online with pseudo-positive and pseudo-negative samples generated from actual user queries. This dynamic training approach allows the system to adapt to emerging search patterns and user preferences in real-time, making the search system versatile and responsive to changing user needs.
Solution Approach 2:
The patent changes the data quality parameter by accepting lower-quality pseudo-labels in exchange for real-time adaptability. The system trades off some training data quality (using automatically generated pseudo-positive and pseudo-negative samples) to achieve the benefit of continuous adaptation to user queries without requiring manual annotation of training data.
Data Source
AI summary
Methods and systems for object-sensitive image searches are described herein. These methods and systems are usable for receiving a query for an image of an object and providing a ranked list of query results to the user based on a ranking of the images. The object-sensitive image searches may generate a pre-trained multi-instance learning (MIL) model trained from free training data from users sharing images at websites to identify a common pattern of the object, and/or may generate a MIL model “on the fly” trained from pseudo-positive and pseudo-negative samples of query results to identify a common pattern of the object. As such, the user is presented with query results that include images that prominently display the object near the top of the results.