Personalized Image Search via Query Anonymization and Re-scoring
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current image searching technologies fail to effectively process natural language queries, including proper names, and often return inappropriate results due to limitations in detector failures and inability to handle semantic-based searches.
Innovation Solution
A method and apparatus for personalized image retrieval that involves anonymizing natural language queries by replacing proper names with generic terms, using a trained scoring model for initial ranking and a re-scoring model to determine the probability of face matches, thereby enhancing semantic search capabilities.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If natural language queries with proper names are used for image searching, then search accuracy and personalization are improved, but system complexity increases due to the need for multiple processing stages including anonymization, semantic scoring, and re-scoring
Solution Approach 1:
The search system is divided into distinct functional modules: a query processing module that anonymizes proper names, a scoring model for semantic-based initial ranking, and a re-scoring model for refining results. This segmentation allows each component to specialize in specific tasks, improving overall accuracy while making the complex system more manageable and maintainable
Solution Approach 2:
The patent introduces intermediate processing steps including query anonymization (replacing proper names with generic terms) and attention weight calculation as mediators between the user query and final image ranking. These intermediaries bridge the gap between natural language input and machine processing, enabling accurate personalized search without requiring the entire system to handle all aspects of name resolution simultaneously
2Measurement precision
If semantic-based searching with multiple models is implemented, then relevance of search results is improved, but processing time increases due to multiple scoring and re-scoring stages
Solution Approach 1:
The scoring model performs preliminary semantic-based ranking and generates attention weights before the re-scoring stage. By pre-processing queries and images to extract meaningful features and initial rankings, the system reduces the computational burden on the final re-scoring model, thereby decreasing overall processing time while maintaining high relevance
Solution Approach 2:
The system applies attention mechanisms selectively to specific regions and features rather than processing all image data uniformly. The re-scoring model focuses computational resources on adjusting rankings for images with high initial scores or those containing proper names, rather than re-evaluating every image, thus reducing processing time while preserving result quality
3Adaptability or versatility
If proper names are processed directly without anonymization, then personalized search capability is improved, but detector reliability decreases due to failures in handling proper names
Solution Approach 1:
The patent introduces an anonymization intermediary that replaces proper names with generic terms before processing through detectors. This intermediary layer protects the system from detector failures when handling proper names, as the generic terms are more robust to detection errors. The original proper names are preserved in the query metadata and applied during the re-scoring stage to maintain personalized search capability
Solution Approach 2:
By anonymizing proper names before detection, the system creates a buffer against detector failures. The generic terms serve as a backup representation that ensures basic search functionality even when proper name detection fails, thereby cushioning the system against reliability issues while still enabling personalized search through the original query
Data Source
AI summary
A method of personalized image retrieval includes obtaining a natural language query including a name; replacing the name in the natural language query with a generic term to provide an anonymized query and named entity information; obtaining a plurality of initial ranking scores and a plurality of attention weights corresponding to a plurality of images using a trained scoring model that inputs the anonymized query and the plurality of images; obtaining a plurality of delta scores corresponding to the plurality of images using a re-scoring model that inputs the plurality of attention weights and the named entity information; and obtaining a plurality of final ranking scores by modifying the plurality of initial ranking scores based on the plurality of delta scores. The trained scoring model performs semantic based searching and the re-scoring model determines a probability that faces detected in the plurality of images correspond to the name.