On-Device AI Task Routing via Dynamic Expert Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing AI systems face challenges in efficiently processing large datasets on-device due to high computational requirements and limited resource management, particularly with methods like Mixture of Experts (MoE) that require significant VRAM and are resource-intensive.
Innovation Solution
A method and system that automatically determine the type of tasks requested by users through analyzing user dialogue inputs, and systematically perform these tasks using task context data obtained from the dialogue inputs, optimizing resource usage and improving efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If Mixture of Experts (MoE) architecture is used to process complex tasks, then task processing capability is improved, but resource consumption (VRAM and computational power) increases significantly
Solution Approach 1:
The patent segments the monolithic AI model into multiple specialized expert models, each trained to handle specific task types. The gating network divides incoming tasks and routes them to appropriate expert models, allowing the system to process complex tasks with high capability while consuming resources only for the specific expert model needed, rather than activating the entire large model.
Solution Approach 2:
The patent implements dynamic task routing where the gating network evaluates each incoming task and dynamically selects which expert model to activate. This dynamic selection ensures that only the necessary computational resources are consumed for each specific task, rather than statically allocating resources for all possible task types.
2Adaptability or versatility
If large AI models are deployed for comprehensive task handling, then service coverage is improved, but on-device deployment becomes difficult due to resource constraints
Solution Approach 1:
The patent divides a large AI model into multiple smaller expert models that can be individually deployed on resource-constrained devices. Each expert model handles a specific subset of tasks, allowing the system to maintain comprehensive service coverage through the collection of specialized models while keeping individual model sizes suitable for on-device deployment.
Solution Approach 2:
The patent creates a universal AI assistant framework where a gating network coordinates multiple specialized expert models to handle diverse task types. This multi-functional architecture allows a single system to provide comprehensive service coverage across various domains while maintaining the flexibility to deploy only the necessary expert models on each device based on available resources.
3Productivity
If multiple expert models are activated simultaneously in MoE architecture, then processing capability is improved, but resource management efficiency deteriorates
Solution Approach 1:
The patent implements dynamic task routing where the gating network evaluates each incoming task and selectively activates only the relevant expert model(s) needed for that specific task. This dynamic activation ensures high processing capability by having multiple expert models available while maintaining resource management efficiency by activating only one or a few models at a time rather than all models simultaneously.
Solution Approach 2:
The gating network autonomously evaluates incoming tasks and self-manages the selection and activation of appropriate expert models without external intervention. This self-service mechanism optimizes resource utilization by automatically determining which expert models are needed for each task, avoiding unnecessary activation of models and thereby improving resource management efficiency.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method for automatically performing a task may perform a task based on a user dialogue input by a computing system including a memory and a processor. The method may include: receiving a user dialogue input; determining a type of a task requested by a user by analyzing the user dialogue input; obtaining task context data required to perform the task based on the user dialogue input; and performing the task of which type is determined based on the task context data.