Semantic Search Architecture With Late Interaction Ranking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Semantic searches in search engines consume excessive computational bandwidth, leading to increased latency and user dissatisfaction due to the need for compute-intensive re-ranking steps.
Innovation Solution
A trained retrieval model is used to generate similarity scores for candidate records, allowing a late interaction model to sort records based on these scores, thereby reducing the need for real-time re-ranking and minimizing bandwidth consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If semantic search is implemented to understand query meaning and improve search accuracy, then search accuracy is improved, but computational bandwidth consumption increases significantly
Solution Approach 1:
The patent segments the search process into two distinct phases: (1) an offline training phase where the retrieval model is trained using query-record pairs and query-query pairs to learn semantic relationships, and (2) an online search phase where the trained model performs efficient inference. This segmentation allows the computationally intensive training to be done beforehand, making the online phase much lighter and faster.
Solution Approach 2:
The patent performs preliminary action by pre-training the retrieval model offline using large datasets of query-record pairs and query-query pairs. This preliminary training establishes the semantic understanding capability before actual search operations, so that during online search, the model can quickly retrieve relevant documents without performing heavy computations in real-time.
2Measurement precision
If compute-intensive re-ranking is performed to improve result accuracy, then result accuracy is improved, but latency increases leading to user dissatisfaction
Solution Approach 1:
The patent extracts the semantic understanding and re-ranking functionality into a separate trained retrieval model that operates independently during online search. Instead of performing full re-ranking computations for every query, the system uses the pre-trained model to generate similarity scores and retrieve top-k documents directly, taking out the heavy computational burden from the real-time processing path.
Solution Approach 2:
The patent changes the operational parameters of the search system by using a pre-trained retrieval model that outputs similarity scores in a computationally efficient manner. The model transforms the search problem from requiring intensive re-ranking computations to a faster similarity-based retrieval process, changing the time and computational resource parameters favorably.
Data Source
AI summary
Techniques are disclosed relating to operating, by a computer system, a semantic search engine to retrieve records from a data store. The technique includes training, by the computer system using a plurality of training data sets that include queries and corresponding records, a retrieval model for use in the semantic search engine. The technique may further include generating, by the trained retrieval model, a particular output vector representing a received semantic search query, and generating, using the particular output vector, a respective similarity score for ones of candidate records identified in the data store. The trained retrieval model may send the particular output vector to a late interaction model, and the late interaction model may sort, using the particular output vector, candidate records with respective similarity scores that satisfy a threshold score.