Image Search Using Multi-Model Semantic Ranking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional image searching methods are inadequate for efficiently finding similar images due to their reliance on keyword queries, which can be inaccurate and computationally expensive, and fail to capture granular level details such as object relationships and positions.
Innovation Solution
The method employs a multi-model approach using scene classification, attribute classification, object detection, and image statistics models to rank candidate images based on similarity to a query image, incorporating weighted aggregates of similarity measures to capture detailed semantic and spatial information.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional keyword query methods are used for image searching, then text-based semantic concepts can be associated with images, but the search accuracy deteriorates and computational cost increases
Solution Approach 1:
The patent segments the image search process into multiple specialized models: scene classification model for spatial relationships, object detection model for object identification, attribute classification model for visual characteristics, and image statistics model for global features. Each model processes specific aspects of image similarity independently, improving overall search accuracy while distributing computational load efficiently across specialized components rather than using a single resource-intensive keyword matching system.
Solution Approach 2:
The patent transforms the search approach by changing from text-based keyword parameters to multi-dimensional visual parameters including scene type, object presence, attribute characteristics, and statistical features. This parameter transformation enables more accurate semantic matching by comparing actual visual content properties rather than relying on text descriptions, thereby improving search precision without proportionally increasing computational cost.
2Quantity of substance
If manual searching of large image repositories is performed, then complete coverage of images can be achieved, but time consumption increases significantly
Solution Approach 1:
The patent applies preliminary action by pre-processing images through multiple classification and detection models to extract scene types, objects, attributes, and statistical features before the actual search query is executed. These pre-computed visual parameters are stored and can be rapidly compared against query images, enabling fast full-repository search without manual intervention. The preliminary extraction of semantic features from all images allows comprehensive coverage while maintaining low search time through efficient parameter comparison.
3Measurement precision
If conventional search algorithms are used, then simple text matching can be performed, but the ability to capture granular details such as object relationships and positions deteriorates
Solution Approach 1:
The patent divides the complex task of semantic similarity assessment into four specialized models: scene classification for spatial relationships and layout, object detection for identifying and localizing objects, attribute classification for visual characteristics, and image statistics for global patterns. Each model focuses on specific granular details, enabling precise capture of object relationships and positions without requiring a single overly complex algorithm. This segmentation allows the system to achieve high semantic accuracy through coordinated specialized components.
Solution Approach 2:
The patent combines multiple specialized models into a composite search system that integrates scene classification, object detection, attribute classification, and image statistics. This composite approach leverages the strengths of each individual model to achieve comprehensive semantic understanding, capturing granular details like object relationships and positions that any single model might miss. The composite structure enables high measurement precision by synthesizing information from multiple specialized sources.
Data Source
AI summary
Methods, systems, and media for image searching are described. Images comprising one query image and a plurality of candidate images are received. For each candidate image, a first model similarity measure from an output of a first model configured for scene classification to perceive scenes in the images is determined. Further, for each candidate image of the plurality of candidate images, a second model similarity measure from the output of a second model configured for attribute classification to perceive attributes in the images is determined. For each candidate image of the plurality of candidate images, a similarity agglomerate index of a weighted aggregate of the first model similarity measure and the second model similarity measure is computed. The plurality of candidate images based on the respective similarity agglomerate index of each candidate image are ranked and a first ranked candidate images corresponding to the searched images are generated.