On-Device AI Agent Model Selection Using Mixture-of-Experts
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing AI models face limitations in on-device environments due to high computing costs and resource inefficiencies, particularly when processing large datasets, and struggle to quickly adapt to specific contexts.
Innovation Solution
A method and system for providing an on-device AI agent that utilizes a mixture of experts (MoE) architecture to optimize application models based on a large language model (LLM), incorporating a master model and secondary models like sLLM, MoELM, and SMs to efficiently process tasks based on domain-specific data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a large language model (LLM) is used to process complex tasks and large datasets, then processing capability and accuracy are improved, but computing cost and time consumption increase significantly
Solution Approach 1:
The patent applies segmentation by dividing the LLM into multiple expert models, each specialized in processing specific types of data or tasks. The gating network segments the input data and routes it to the appropriate expert model, avoiding the need to process all data through the entire LLM, thus reducing time consumption while maintaining processing accuracy for specific domains.
Solution Approach 2:
The patent implements partial action by activating only the necessary expert models based on the input data characteristics. Instead of running the complete LLM for every task, the system selectively engages only the relevant expert models identified by the gating network, reducing unnecessary computing operations and time consumption while maintaining adequate processing accuracy.
2Measurement precision
If a large language model (LLM) is used to process complex tasks and large datasets, then processing capability and accuracy are improved, but computing cost increases
Solution Approach 1:
The patent segments the computationally expensive LLM into multiple smaller expert models, each handling specific task types. The gating network efficiently routes inputs to appropriate experts, reducing the total computational load and energy consumption compared to running the full LLM for all tasks, while maintaining processing accuracy through specialized expert models.
Solution Approach 2:
The patent applies local quality by creating expert models with specialized capabilities tailored to specific domains or task types. Each expert model is optimized for its particular function, allowing the system to achieve high processing accuracy for specific tasks with lower computing costs compared to using a general-purpose large language model for all tasks.
3Productivity
If mixture of experts (MoE) architecture is used to improve efficiency, then resource concentration and performance are improved, but high-level video random access memory (VRAM) is required and fine tuning complexity increases
Solution Approach 1:
The patent segments the MoE architecture into a gating network and multiple independent expert models. This segmentation allows for simplified fine-tuning processes where each expert model can be trained or updated independently based on its specific domain, reducing the overall complexity compared to fine-tuning a monolithic model, while maintaining high productivity through efficient resource concentration.
4Productivity
If mixture of experts (MoE) architecture is used to manage large size model, then model efficiency is improved, but support for remaining non-activated resources is limited
Solution Approach 1:
The patent implements dynamics by making the MoE architecture adaptive through the gating network, which dynamically selects and activates expert models based on the characteristics of the input data. This dynamic selection mechanism allows the system to flexibly utilize available resources by activating only the necessary experts for each task, improving model efficiency while maintaining adaptability and versatility in resource utilization.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method for providing an AI agent based on an LLM applying an artificial intelligence model including a plurality of models according to an embodiment of the present invention pertains to a method for a computing system including a memory and a processor to provide an AI agent based on an LLM applying an artificial intelligence model including a plurality of models, wherein the method includes: executing an on-device AI agent service; acquiring predetermined input data based on the executed on-device AI agent service; determining a domain according to the acquired input data; deciding an application model, which is an artificial intelligence model that will process a task according to the determined domain; generating output data for the input data based on the decided application model; and providing the generated output data based on the on-device AI agent service.