Hybrid LLM Task Routing for Privacy and Token Limits
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large language models (LLMs) face limitations in data privacy concerns, token length constraints, and inefficiencies in handling domain-specific knowledge, while private services lack adaptability and capability to handle complex tasks due to their small scale.
Innovation Solution
Integrate public language models with private services by splitting tasks into sub-tasks based on the capabilities of an operation pool, using a public language model to pair these sub-tasks with respective private services, and executing them efficiently.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If public language models are used to handle complex tasks, then task processing capability is improved, but data privacy concerns arise
Solution Approach 1:
The patent divides complex tasks into multiple sub-tasks and distributes them across different language models. Some sub-tasks are handled by public LLMs while others are processed by private LLMs, allowing the system to leverage the strong capabilities of public models while keeping sensitive data processing within private models, thus resolving the contradiction between task processing capability and data privacy.
Solution Approach 2:
The patent introduces a task allocation mechanism that acts as an intermediary between public LLMs and private services. This intermediary intelligently routes tasks based on sensitivity and capability requirements, enabling public LLMs to process non-sensitive tasks while private models handle sensitive data, thereby maintaining both high capability and data privacy.
2Adaptability or versatility
If public language models are used, then capability to handle complex tasks is improved, but token length constraints limit processing scope
Solution Approach 1:
The patent segments long-context tasks into multiple sub-tasks that can be processed in parallel or sequentially across different model instances. This allows the system to overcome the token length limitation of individual models by distributing the processing of large volumes of data across multiple smaller processing units, each operating within their token limits.
Solution Approach 2:
The patent transitions from a single-model sequential processing approach to a multi-model parallel processing architecture. By adding the dimension of multiple model instances working simultaneously, the system can process much larger amounts of data than any single model could handle independently, effectively overcoming token length constraints.
3Object-affected harmful factors
If private services are used for local processing, then data privacy is maintained, but adaptability and capability to handle complex tasks are reduced
Solution Approach 1:
The patent merges private and public language model services into a unified hybrid architecture. Private models handle sensitive local processing while public models provide enhanced capabilities for non-sensitive tasks. The system dynamically combines the strengths of both approaches, allowing private services to maintain data privacy while gaining access to the superior capabilities of public models through coordinated collaboration.
4Use of energy by moving object
If small-scale private language models are deployed, then resource usage is optimized, but capability to handle complex tasks is limited
Solution Approach 1:
The patent segments complex tasks into smaller sub-tasks that can be efficiently handled by small-scale private models. By breaking down computationally intensive tasks into manageable pieces, the system enables resource-constrained models to contribute meaningfully while maintaining overall system capability through coordination with other model instances that handle more demanding sub-tasks.
Data Source
AI summary
A method of this disclosure may comprise receiving a task described at least in part with natural language; instructing a public language model to split the task into a plurality of sub-tasks based on a capability of an operation pool which includes a plurality of private services, and to pair the plurality of sub-tasks with respective private services; and instructing the respective private services to perform the plurality of sub-tasks so as to complete the task.