Edge Model Management With Dynamic AI Model Switching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Agentic AI systems face challenges in deploying large AI models like GPT-3 and BERT on edge devices due to memory and computation resource constraints, making it infeasible to store diverse AI models for various purposes.
Innovation Solution
A device agentic framework that decouples apps and models, allowing on-demand downloading of AI models from the cloud, utilizing an agentic manager to orchestrate app operations and switch between edge and cloud models based on resource availability and requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If large AI models are stored on edge devices to provide diverse AI capabilities, then the agentic AI system can function effectively across various domains, but the memory footprint and computation resources required become infeasible for edge devices
Solution Approach 1:
The system segments AI models into two categories: edge models stored locally on the device and cloud models accessed remotely. This segmentation allows the edge device to maintain only a small set of lightweight models while accessing diverse large models through the cloud, resolving the contradiction between model diversity and memory footprint.
Solution Approach 2:
The patent introduces a cloud server as an intermediary to store and provide AI models. The edge device communicates with the cloud server to download and execute models on demand, eliminating the need for the edge device to store all models locally and thus reducing memory requirements while maintaining access to diverse models.
2Adaptability or versatility
If multiple AI models are installed on edge devices to satisfy diverse app requirements, then apps can function with various AI capabilities, but the computation resources and memory become insufficient
Solution Approach 1:
The system dynamically loads and unloads AI models from the cloud based on app requirements and available resources. Instead of having all models permanently installed, the edge device dynamically acquires models only when needed, reducing the permanent resource footprint while maintaining the capability to handle diverse AI tasks.
Solution Approach 2:
The cloud server acts as a universal model repository that serves multiple edge devices. A single copy of a large AI model stored in the cloud can be accessed by multiple edge devices simultaneously, eliminating the need for each edge device to maintain its own complete model library and thus reducing overall resource consumption.
3Speed
If the edge device stores all required AI models locally to ensure fast access, then model loading speed is improved, but the memory capacity required becomes excessively large
Solution Approach 1:
The system performs preliminary actions by caching frequently used edge models locally and pre-configuring model loading mechanisms. For models that need to be accessed frequently, the system maintains them in a local cache, while less frequently used models are accessed from the cloud, balancing speed and storage requirements.
Data Source
AI summary
An edge device provides an agentic framework to manage apps and artificial intelligence (AI) models used by the apps. An app and app metadata are downloaded from a cloud of servers to the device. The app metadata describes requirements of the app for AI models to be used by the app. The agentic framework performs a search in an on-device database that stores the app metadata and model metadata of edge models installed on the device. The search is performed to determine whether one of the edge models satisfies the requirements of the app. Following the search, the agentic framework sets a given edge model already installed on the device as a target model of the app, where the target model satisfies the requirements of the app. The agentic framework then directs the app to use the target model in response to a request for service.