Edge Agentic Manager With On-Demand AI Model Switching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Agentic AI systems require large memory footprints and computation resources, making it infeasible to deploy large AI models like GPT-3 or BERT on edge devices due to their billions of parameters.
Innovation Solution
A device agentic framework that decouples apps and models, allowing on-demand downloading of AI models from the cloud, utilizing an agentic manager to orchestrate app interactions and manage edge and cloud models, with vector embeddings for efficient search and switching between models based on resource availability and user requests.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If large AI models like GPT-3 or BERT are deployed on edge devices, then agentic AI capabilities are improved, but memory footprint and computation resources required become excessively large
Solution Approach 1:
The system segments AI model functionality by separating the agentic manager (running locally on edge devices) from the actual AI models (hosted in cloud). The manager handles task orchestration and context management locally, while only necessary model components are downloaded from cloud when needed, enabling agentic AI capabilities without requiring large models to be permanently stored on edge devices
Solution Approach 2:
The patent introduces a cloud-based AI model service as an intermediary between the edge device and the AI models. The service includes a model downloader that fetches models from cloud, a model manager that handles model lifecycle, and a tokenizer that processes inputs. This intermediary architecture allows edge devices to access powerful AI models without storing them locally, resolving the contradiction between capability and memory requirements
2Adaptability or versatility
If large AI models are stored on edge devices, then agentic AI functionality is enhanced, but computation resources become insufficient
Solution Approach 1:
The system divides computation workload into two parts: lightweight tasks executed locally by the agentic manager on the edge device (task understanding, context retrieval, action orchestration), and heavy AI model inference tasks executed in cloud. This segmentation enables enhanced agentic AI functionality without requiring the edge device to provide sufficient computation resources for running large models
Solution Approach 2:
The cloud-based AI model service acts as a computation intermediary, providing powerful AI inference capabilities remotely. The service includes a model downloader for fetching models, a model manager for resource allocation, and a tokenizer for processing. This allows the edge device to access computationally intensive AI functions without needing to host them locally
3Adaptability or versatility
If multiple AI models are stored on edge devices for diverse purposes, then model versatility is improved, but device memory becomes insufficient
Solution Approach 1:
The system segments model storage by moving all AI models to cloud-based storage. The edge device maintains only a lightweight agentic manager that can request specific models from cloud when needed. This enables the device to support multiple AI models for diverse purposes (language processing, image analysis, etc.) without consuming device memory, as models are dynamically downloaded only when required
Solution Approach 2:
The patent implements dynamic model loading where the agentic manager can request and download specific AI models from cloud based on current task requirements. The model manager handles dynamic model lifecycle operations including downloading, loading, unloading, and deleting models. This dynamic approach enables model versatility while keeping device memory usage minimal, as models are only present on device when actively needed
4Quantity of substance
If AI models are downloaded on-demand from cloud, then memory usage is optimized, but system complexity increases
Solution Approach 1:
The patent merges multiple functions into the agentic manager: task reception from users, context information retrieval from local database, action plan generation by calling AI models, and response delivery to users. By combining these functions in a single coordinated component, the system achieves on-demand model downloading and optimized memory usage without proportionally increasing overall system complexity
Solution Approach 2:
The cloud-based AI model service serves as a comprehensive intermediary that handles model management operations. The service includes a model downloader for fetching models, a model manager for coordinating model lifecycle, and a tokenizer for processing inputs and outputs. This intermediary abstraction handles the complexity of on-demand model management, allowing the edge device to achieve optimized memory usage without managing model operations locally
Data Source
AI summary
A device provides an agentic experience for a user. An agentic manager, which is a management app on the device for providing the agentic experience, receives a request from the user via a user interface. The agentic manager sends a prompt incorporating contextual information of the request to an agentic artificial intelligence (AI) model. The contextual information is retrieved from an on-device database and identifies one or more apps on the device. The agentic manager receives from the agentic AI model an action plan for calling a target app among the one or more of apps. The agentic manager sends action requests according to the action plan to the target app to invoke functionalities of the target app, and sends to the user via the user interface an output that incorporates a response generated by the target app.