Semantic Search Retrieval Model Without Query-Time Re-Ranking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Semantic searches in search engines consume excessive computational bandwidth, leading to increased latency and user dissatisfaction due to the need for compute-intensive re-ranking steps.
Innovation Solution
A trained retrieval model is used to generate candidate record lists with high accuracy, eliminating the need for a re-ranking step by using a late interaction model to sort records based on similarity scores that satisfy a threshold, thereby reducing bandwidth consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If semantic search is used to understand query meaning and improve search accuracy, then search accuracy is improved, but computational bandwidth consumption increases
Solution Approach 1:
The patent segments the search process into two distinct phases: (1) an offline training phase where the retrieval model is trained using query-record pairs and similarity scores, and (2) an online search phase where the trained model performs fast inference. This segmentation allows the computationally intensive training to be done beforehand, while the actual search operations use the pre-trained model for efficient processing.
Solution Approach 2:
The retrieval model is trained in advance using preliminary action, where the model learns to generate accurate candidate record lists before actual search queries are processed. The training phase prepares the model with pre-computed knowledge from query-record pairs, enabling fast and accurate inference during online search without requiring intensive computational resources at query time.
2Measurement precision
If compute-intensive re-ranking step is performed to improve result accuracy, then result accuracy is improved, but processing time increases
Solution Approach 1:
The patent extracts and eliminates the compute-intensive re-ranking step from the online search process. Instead of performing re-ranking at query time, the system uses the trained retrieval model to directly generate accurately ranked candidate record lists during inference, removing the time-consuming re-ranking operation from the critical search path.
Solution Approach 2:
The trained retrieval model captures and stores the ranking knowledge learned during training. The model copies the ranking patterns and relationships discovered in the training phase into its parameters, enabling it to perform accurate ranking during inference without requiring the original computationally intensive re-ranking process to be repeated at query time.
3Ease of operation
If semantic search processes are used to understand query intent, then user satisfaction is improved, but bandwidth usage increases unfavorably
Solution Approach 1:
The patent changes the operational parameters of the search system by using a pre-trained retrieval model that operates with optimized inference-time parameters. The model transitions from the training phase with high computational requirements to the inference phase with reduced bandwidth usage, maintaining semantic understanding capabilities while consuming fewer resources during actual search operations.
Data Source
AI summary
Techniques are disclosed relating to operating, by a computer system, a semantic search engine to retrieve records from a data store. The technique includes training, by the computer system using a plurality of training data sets that include queries and corresponding records, a retrieval model for use in the semantic search engine. The technique may further include generating, by the trained retrieval model, a particular output vector representing a received semantic search query, and generating, using the particular output vector, a respective similarity score for ones of candidate records identified in the data store. The trained retrieval model may send the particular output vector to a late interaction model, and the late interaction model may sort, using the particular output vector, candidate records with respective similarity scores that satisfy a threshold score.