Query Embedding Whitelist Filtering for Search Relevance
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing online systems struggle to accurately retrieve items relevant to a user's query due to embedding-based retrieval methods including irrelevant items, which can discourage user interaction and prevent item selection.
Innovation Solution
An online system generates item embeddings and query embeddings in a latent space, determines distances or similarities between them, and uses a whitelist of attribute values to filter out irrelevant items based on specific attributes, ensuring higher relevance to the query.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If embedding based retrieval is used to retrieve items, then the system can identify items with higher relevance to query intent, but irrelevant items may still be included in the results
Solution Approach 1:
The patent segments the retrieval process into multiple stages: initial embedding-based retrieval to capture potentially relevant items, followed by attribute-based filtering to segment and remove irrelevant items. This multi-stage approach allows the system to benefit from the semantic understanding of embeddings while eliminating false positives through attribute verification.
Solution Approach 2:
The patent introduces attribute values as an intermediary layer between the query embedding and final item selection. Instead of directly selecting items based solely on embedding distance, the system uses attribute values as a mediator to verify relevance, thereby improving result quality while maintaining the benefits of embedding-based retrieval.
2Productivity
If text description matching is used to retrieve items, then the system can identify items with text matching the query, but it fails to identify certain relevant items or identifies less relevant items
Solution Approach 1:
The patent replaces the mechanical text-matching system with an embedding-based semantic representation system. Instead of comparing text strings character-by-character or word-by-word, the system transforms both queries and items into vector embeddings in a semantic space, allowing for more accurate identification of relevant items based on meaning rather than surface-level text matching.
3Adaptability or versatility
If irrelevant items are included in the search results, then the system provides more comprehensive coverage, but user interaction is discouraged and item selection is prevented
Solution Approach 1:
The patent performs preliminary attribute-based filtering before presenting results to the user. By pre-screening items based on their attribute values against the query requirements, the system eliminates irrelevant items in advance, ensuring that only potentially relevant items are presented to the user. This preliminary action maintains comprehensive coverage of relevant items while removing irrelevant ones that would hinder user interaction.
Data Source
AI summary
An online system maintains various items and maintains values for different attributes of the items, as well as an item embedding for each item. When the online system receives a query for retrieving one or more items, the online system generates an embedding for the query. Based on measures of similarity between the embedding for the query and item embeddings, the online system selects a set of items. The online system identifies a specific attribute of items and generates a whitelist of values for the specific attribute based on measures of similarity between item embeddings for items in the selected set and the embedding for the query. The online system removes items having values for the selected attribute outside of the whitelist of values from the selected set of items to identify items more likely to be relevant to the query.