Role-Based LLM Query Routing for Privacy-Safe Response Accuracy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Users are hesitant to engage with large language models (LLMs) due to concerns about data privacy and confidentiality, leading to inefficiencies and increased latency as they avoid using LLMs for certain tasks, necessitating manual workarounds or using non-LLM tools, which disrupt workflows and consume additional computing resources.
Innovation Solution
Implementing a role-based large language model (LLM) platform that distinguishes between open and closed domain dialog systems, using privacy status to route queries appropriately, ensuring sensitive data is not reused for training and maintaining accuracy by segregating data sets.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If LLMs continuously re-train based on user queries to improve accuracy, then response accuracy improves, but user data privacy and confidentiality are compromised
Solution Approach 1:
The patent segments the dialog system into open domain and closed domain components. Open domain LLMs are used for general tasks where re-training can occur, while closed domain LLMs handle sensitive tasks where data is not reused for training. This segmentation allows the system to pursue accuracy improvements through re-training where safe, while protecting privacy where needed.
Solution Approach 2:
The patent introduces an intermediary mechanism (privacy status classification and routing system) that determines whether user queries should be processed by open or closed domain LLMs. This intermediary layer mediates between the conflicting goals of re-training for accuracy and data protection, routing sensitive queries to closed domain models that do not reuse data for training.
2Object-affected harmful factors
If users avoid using LLMs for confidential tasks, then data privacy is protected, but workflow efficiency decreases and latency increases
Solution Approach 1:
The patent creates a universal dialog system that can handle both open and closed domain tasks through a single interface. Users interact with one system that automatically routes queries appropriately, eliminating the need for users to manually select different tools or workflows for sensitive versus non-sensitive tasks, thereby maintaining productivity while ensuring privacy.
Solution Approach 2:
The system performs self-service by automatically classifying the privacy status of user queries and routing them to the appropriate domain (open or closed) without user intervention. This automated classification and routing eliminates workflow disruptions and allows users to maintain their natural interaction patterns while the system handles privacy protection transparently.
3Reliability
If separate closed domain LLMs are used for sensitive tasks, then data security is improved, but system complexity increases
Solution Approach 1:
The patent merges the open domain and closed domain LLM systems into a unified dialog system with a single interface and automated routing mechanism. While the backend contains separate models for different domains, the user experiences a single integrated system, reducing the perceived complexity and eliminating the need for users to manage multiple separate tools.
Solution Approach 2:
The system implements feedback mechanisms where user queries are automatically classified by privacy status, routed to appropriate domains, and the results are fed back to users through a unified interface. This feedback loop allows the system to maintain separate processing paths for security while presenting a simplified unified experience to users, effectively managing system complexity.
Data Source
AI summary
A first query having a first privacy status is received. A response to the first query is obtained based on output(s) of one or more machine learning (ML) models of an open domain dialog system. Each ML model is trained to predict responses to queries having the first privacy status. Data associated with the first query and the response is provided as training data for the ML models, in view of the first privacy status. A second query having a second privacy status is received. A closed domain dialog system associated with a context of the second query and having a privacy status corresponding to the second privacy status is identified. The second query is forwarded to the closed domain dialog system for obtaining a response to the second query. Data associated with the second query is not provided to train the ML models of the open domain dialog system.