On-Device Generative AI Task Routing for Privacy and Resource Balance
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing AI models consume significant computational resources, limiting their execution on smaller devices and leading to inefficiencies and resource conflicts, while server-based execution increases infrastructure and network usage, and raises data privacy concerns.
Innovation Solution
A system and method that determines which tasks can be performed by AI models on client devices versus server devices, optimizing resource usage and reducing network data transfer by executing tasks locally when possible.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If AI models are executed on client devices, then data privacy and security are improved, but computational resource consumption on client devices increases
Solution Approach 1:
The patent segments AI tasks into two categories: those executed locally on client devices and those executed remotely on server devices. The system dynamically determines which tasks to process locally based on device capabilities, task complexity, and resource availability, thereby balancing data privacy requirements with computational resource constraints.
2Productivity
If AI models are executed on server devices, then computational resource availability is improved, but network data transfer and infrastructure usage increase
Solution Approach 1:
The patent implements local quality by enabling client devices to execute AI tasks locally when capable, rather than uniformly processing all tasks through centralized servers. This reduces network data transfer and infrastructure usage for tasks that can be handled locally, while still maintaining access to server resources when needed.
3Loss of energy
If AI models are executed on client devices, then network usage is reduced, but device complexity and resource requirements increase
Solution Approach 1:
The patent employs dynamic task routing that adjusts execution location based on real-time device capabilities, task characteristics, and resource availability. The system can dynamically switch between local and remote execution modes, allowing client devices to handle simpler tasks locally while offloading complex tasks to servers, thereby managing device complexity adaptively.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method for on-device artificial intelligence (AI) includes obtaining, at a client device, a request to perform a task. The method includes providing the request as input to a generative AI model operating on the client device. The generative AI model may be trained to identify one or more applications capable of performing the task and to generate instructions related to the task for the one or more applications. The method includes causing the instructions related to the task to be provided to the one or more applications. The method includes receiving an indication of completion of at least a portion of the task. The method includes providing, at the client device, information regarding the completion of the at least a portion of the task.