Query Replay for Personalized LLM Responses
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing conversational assistants powered by large language models struggle to accurately determine which user preferences are permanent, which change over time, and which are context-dependent, leading to difficulties in providing personalized responses.
Innovation Solution
A query replay technique that identifies a relevant preferred refinement query from a user's history to prompt the assistant LLM, generating personalized responses by embedding the current query and retrieving relevant refinement queries, and structuring prompts to enhance personalization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If the assistant LLM uses historical refinement queries to personalize responses, then response personalization is improved, but system complexity increases
Solution Approach 1:
The system pre-processes and stores historical refinement queries in a database before they are needed. When a new query arrives, the system retrieves relevant historical refinement queries from the pre-stored database and uses them to personalize the response. This preliminary action avoids complex real-time analysis of historical data while still achieving personalization.
Solution Approach 2:
The patent introduces an intermediary layer between the user query and the LLM response. This intermediary component (the query replay mechanism) retrieves relevant historical refinement queries and formats them as prompts for the LLM. This intermediary simplifies the overall system architecture by separating the personalization logic from the core LLM processing.
2Measurement precision
If the system retrieves and applies relevant refinement queries from history, then response relevance to user preferences is improved, but processing time increases
Solution Approach 1:
Historical refinement queries are pre-retrieved and stored in a database structure optimized for quick access. When a new query arrives, the system performs a fast database lookup to retrieve relevant historical refinement queries, avoiding the need to search through entire historical logs in real-time. This preliminary organization of data reduces processing time while maintaining response relevance.
Solution Approach 2:
The system retrieves only the necessary portion of historical data (relevant refinement queries) rather than processing all historical queries. By using query relevance filtering and selective retrieval, the system minimizes the amount of data processed while still achieving sufficient personalization, thus reducing processing time.
3Adaptability or versatility
If the assistant LLM processes multiple refinement queries for personalization, then response personalization quality is improved, but computational costs increase
Solution Approach 1:
The system processes only the necessary refinement queries (partial action) rather than all historical queries. By filtering and selecting only the most relevant historical refinement queries based on the current query context, the system reduces the computational load on the LLM while still achieving adequate personalization quality.
Solution Approach 2:
The patent extracts and separates the personalization function from the main LLM processing. By retrieving historical refinement queries separately and using them as prompts rather than processing them through the full LLM pipeline, the system reduces computational costs while maintaining personalization quality.
Data Source
AI summary
A method for personalizing responses includes receiving a natural language query from a user that solicits a response from an assistant large language model (LLM) and processing the natural language query to identify, from a log of refinement queries each previously input by the user for instructing the assistant LLM to refine a corresponding previous response generated by the assistant LLM, a preferred refinement query that is relevant to the natural language query. The method also includes prompting, using the natural language query and the preferred refinement query identified as being relevant to the natural language query, the assistant LLM to generate a personalized response to the natural language query. The method also includes providing, for output from a user device, the personalized response to the natural language query.