Facial Image Search Re-ranking via Inlier Scoring
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Image search engines often rank images of famous persons inaccurately due to sensitivity to external data such as image tags and user feedback, leading to outlier images being incorrectly ranked higher than intended subjects.
Innovation Solution
A method that detects faces in images, generates facial templates, clusters images based on similarity, determines an inlier score for each image by comparing it to the largest cluster, and re-ranks images based on these scores to boost relevant images and demote outliers, using a computer vision-based approach.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If image search engines use external data such as image tags and user feedback for ranking, then the search results can be influenced by popular culture and user behavior, but outlier images are incorrectly ranked higher than intended subjects
Solution Approach 1:
The patent segments the image ranking process into two independent components: external data factors (tags, user feedback) and internal visual features (facial templates). By separating these components, the system can evaluate images based on their visual similarity to the query subject without being unduly influenced by external data that may lead to outliers, thus resolving the contradiction between adaptability and ranking accuracy.
Solution Approach 2:
The patent introduces facial templates as an intermediary mechanism between the query image and the ranking decision. The facial template serves as a mediator that captures the essential visual characteristics of the query subject and uses this intermediate representation to evaluate all candidate images, ensuring that ranking accuracy is maintained while still considering relevant visual features.
2Measurement precision
If facial template clustering is performed on all images, then ranking accuracy is improved, but computational resources and processing time increase
Solution Approach 1:
The patent applies partial action by performing facial template clustering only on a subset of images rather than all images. Specifically, the system clusters images that are visually similar to the query subject based on facial templates, but limits the scope to only those images that pass initial filtering criteria. This reduces computational complexity while maintaining ranking accuracy for the most relevant results.
Solution Approach 2:
The patent performs preliminary actions by pre-processing images to extract facial templates and pre-clustering images based on visual similarity before the final ranking stage. This preliminary organization of data reduces the computational burden during the actual ranking process, as the system only needs to compare facial templates within pre-formed clusters rather than analyzing all images from scratch.
Data Source
AI summary
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for adjusting facial image search results. In one aspect, a method includes detecting, in images that are responsive to a query, faces in the images and building facial templates. The images are then clustered according to similarity measures of the facial templates. The cluster with the highest quantity is selected, and each image in the set of returned images is compared to each image in the selected cluster. The similarity of each face with the faces in the largest cluster, based on the facial templates, is determined as an inlier score for the image in which the face is depicted. The system then increases the rank of images with high inlier scores and decreases the rank of images with low inlier scores.