Novel Keyword Embedding Fusion for Zero-Shot Search Retrieval
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional search engines struggle to accurately match query text with novel keywords that have little or no interaction data, leading to poor retrieval performance for new products, future events, or breaking news, and existing methods suffer from computational overhead and catastrophic forgetting.
Innovation Solution
A computing system uses an encoder module to generate textual embeddings for query and novel keywords, identifies similar seen keywords through classifiers trained on interaction data, and employs a combiner model to enrich the novel keyword representation with collaborative signals, enabling accurate retrieval of zero-shot and one-shot keywords.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional encoder-based approaches are used to match query text with keywords, then lexical similarity can be determined, but higher order semantic similarities are not captured and catastrophic forgetting occurs when models are retrained
Solution Approach 1:
The system segments the keyword matching process into two distinct components: (1) an encoder-based component for lexical similarity matching, and (2) a similarity model component for semantic similarity assessment. This segmentation allows each component to specialize in its strength while mitigating the weaknesses of individual approaches.
Solution Approach 2:
The patent merges the outputs of the encoder-based lexical similarity assessment with the similarity model's semantic understanding to produce a comprehensive keyword matching decision. This combination enables the system to capture both surface-level text matches and deeper semantic relationships.
2Adaptability or versatility
If models are constantly retrained with new interaction data to improve keyword matching, then newer keywords are better matched, but older keywords are forgotten and computational overhead increases
Solution Approach 1:
The similarity model acts as an intermediary between the encoder-based lexical matching and the final keyword selection. It provides semantic context that helps retain knowledge of older keywords without requiring constant retraining of the entire model, thus reducing computational overhead while maintaining adaptability.
Solution Approach 2:
The system performs preliminary semantic analysis using the similarity model before final keyword matching decisions are made. This preliminary action allows the system to preserve semantic knowledge of older keywords without requiring full model retraining, reducing computational overhead while maintaining adaptability to new keywords.
3Measurement precision
If learned keyword classifiers are used for keyword matching, then user interaction data can improve matching accuracy, but classifiers cannot be utilized for novel keywords with little or no interaction data
Solution Approach 1:
The system copies the semantic understanding capability from the similarity model and applies it to novel keywords that lack interaction data. This allows novel keywords to benefit from semantic similarity assessment without requiring the extensive interaction data that learned classifiers need.
Solution Approach 2:
The similarity model serves multiple functions: it enhances lexical matching for keywords with interaction data and provides standalone semantic matching for novel keywords without interaction data. This multi-functionality allows a single component to address both scenarios effectively.
4Productivity
If textual similarity alone is used to match queries with keywords, then the process is simple and fast, but retrieval performance is poor for novel keywords and new products
Solution Approach 1:
The system dynamically adjusts the matching approach based on keyword characteristics. For novel keywords, it relies more heavily on semantic similarity assessment, while for established keywords with interaction data, it can leverage learned classifier patterns. This dynamic adaptation maintains speed while improving accuracy for novel terms.
Data Source
AI summary
A computing system obtains a novel keyword from a keyword database comprising a plurality of keywords. The computing system provides the novel keyword as input into an encoder and obtains a textual embedding of the novel keyword. The computing system identifies classifiers for similar seen keywords to the novel keyword. The novel keyword embedding and the classifiers are provided as input into a combiner model which outputs a textual embedding representative of the novel keyword and the seen keyword classifiers. The computing system compares the output of the combiner model to a textual embedding of query text. The computing system then identifies content based upon a computed similarity between the query text and the output of the combiner model. The computing system causes presentation of search results at a client computing device wherein the results include content selected based upon the computed similarity.