Search Engine Embeddings and Semantic Graph Traversal
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing information retrieval systems face challenges in achieving high predictive accuracy and efficient search results without sacrificing training speed, particularly in retrieving relevant items for user queries across various content categories.
Innovation Solution
The implementation of a search engine machine learning model that generates query input embeddings for each content category, performs k-Nearest-Neighbor searches, and utilizes a semantic graph to generate related search results, while training the model with content category labels and similarity-based embeddings to improve search efficiency and accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If existing information retrieval systems perform comprehensive searches across all content categories, then search completeness is improved, but computational operations and training time increase significantly
Solution Approach 1:
The patent segments the search process into multiple stages: initial KNN search for candidate retrieval, semantic graph traversal for relationship-based expansion, and re-ranking for final optimization. Each stage operates on a subset of data with specific objectives, reducing the computational burden compared to comprehensive search while maintaining result quality
Solution Approach 2:
The patent performs preliminary actions by pre-computing semantic graphs from training data, pre-processing query embeddings, and preparing candidate item pools before the actual search query is executed. This shifts computational work from query-time to offline preparation, improving online search speed without sacrificing completeness
2Measurement precision
If the search engine uses multiple content categories and semantic graphs to improve search accuracy, then predictive accuracy is enhanced, but device complexity increases
Solution Approach 1:
The patent implements a universal semantic graph structure that serves multiple functions: it represents relationships between items across different content categories, enables semantic expansion during search, and provides contextual information for re-ranking. This single multi-functional structure reduces the need for separate complex systems for each function
Solution Approach 2:
The patent introduces query embeddings and item embeddings as intermediary representations that bridge between user queries and the semantic graph. These vector representations serve as mediators that enable efficient comparison and matching without requiring direct complex interactions between all system components
3Productivity
If k-Nearest-Neighbor search is performed for each query input embedding to generate initial search results, then search efficiency is improved, but computational operations increase
Solution Approach 1:
The patent applies partial action by performing KNN search only to generate an initial candidate pool rather than retrieving all possible results. The semantic graph traversal then selectively expands from these candidates, and re-ranking processes only the final subset. This partial application of search operations reduces computational operations while maintaining efficiency
Solution Approach 2:
The patent implements a nested search architecture where KNN search results are nested within semantic graph traversal, which is in turn nested within the final re-ranking process. Each layer operates on the output of the previous layer, creating a nested filtering mechanism that reduces the data volume at each stage and thereby reduces total computational operations
Data Source
AI summary
Various embodiments of the present disclosure provide methods, apparatus, systems, computing devices, computing entities, and/or the like for retrieving relevant items for user queries by generating, using a search engine machine learning model, a prediction-based action for the query input wherein query input embeddings of the query input are generated. For each query input embedding, a k-Nearest-Neighbor (KNN) search is performed with respect to search engine repository item embeddings to generate initial search results, and for each initial set result, performing N hops within a semantic graph starting from nodes associated with the initial search result to generate related search results. The search engine machine learning model is trained by generating a search engine repository item embeddings according to embedding techniques for respective content categories and generating the semantic graph based at least in part on a measure of similarity for pairs of search engine repository item embeddings.