Search Query Completion Using Reused Language Model Candidates
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing search query suggestion systems face challenges in accurately predicting user intent due to misspellings and limited data, while large language models introduce unacceptable latency, making them inefficient for real-time query completion.
Innovation Solution
A language model generates autosuggest candidates that are sequentially expanded by reusing previous candidates relevant to the revised partial query, allowing parallel processing and maintaining low runtime latency, and incorporates scoring to ensure accuracy and relevance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If large language models are used to generate search query suggestions, then the accuracy and flexibility of suggestions improve, but the runtime latency becomes unacceptable
Solution Approach 1:
The system pre-generates a comprehensive set of autosuggest candidates using the large language model before the user finishes typing. These candidates are stored and ready for rapid retrieval and filtering, eliminating the need to run the complex model in real-time during user input
Solution Approach 2:
The suggestion generation process is divided into two stages: an offline preprocessing stage where the large language model generates comprehensive candidates, and an online real-time stage where candidates are filtered and selected based on user input. This segmentation allows the complex model work to be done ahead of time
2Measurement precision
If the language model re-generates all autosuggest candidates when the user revises the partial query, then the accuracy of suggestions improves, but the runtime latency increases
Solution Approach 1:
When the user revises their partial query, the system discards only the irrelevant previous candidates and recovers/maintains the relevant ones. This selective retention avoids regenerating all candidates from scratch while maintaining accuracy for relevant suggestions
Solution Approach 2:
The candidate set is dynamically adjusted based on user input revisions. The system adaptively maintains or discards candidates based on their relevance to the revised query, allowing flexible response to user actions without complete regeneration
3Device complexity
If suggestions are based exclusively on previous queries, then the system complexity is reduced, but the flexibility and coverage of possible terms is limited
Solution Approach 1:
The system uses the large language model to generate candidate suggestions that are then filtered against previous query data. This copying approach allows the benefits of comprehensive model-generated candidates while leveraging historical query patterns to guide selection
Data Source
AI summary
A language model is used to generate autosuggestions to complete or revise a user's partial search query. An initial partial query is applied to the language model to generate query candidates for completing the search query. The language model may generate the query candidates as additional or alternate tokens for the partial search query. When the user revises the partial query, the previously-generated candidates can be re-used to reduce subsequent processing time for generating additional candidates. The previously-generated candidates are compared with the revised partial query to select which of the candidates to be re-used and expanded for generating additional tokens. Additional tokens can be generated in parallel for the previously-generated candidates or with model values from the previous generation, enabling the tokens to be generated effectively with reduced latency consistent with user expectations for search-related autosuggestions.