Edge-Cloud AI Model Switching for Memory-Limited Apps
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Agentic AI systems face challenges in deploying large AI models like GPT-3 and BERT on edge devices due to memory and computation resource constraints, making it infeasible to store diverse AI models for various purposes.
Innovation Solution
A device agentic framework that decouples apps and models, allowing on-demand downloading of AI models from the cloud, with vector embeddings stored in a Retrieval Augmented Generation (RAG) database for efficient search and model switching between edge and cloud models based on resource availability and user requests.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If large AI models like GPT-3 and BERT are stored on edge devices, then AI functionality and capabilities are improved, but memory footprint and computation resources are excessively consumed
Solution Approach 1:
The patent extracts the large AI models from the edge device and places them in the cloud. Only lightweight client applications are stored on the edge device, while the heavy computational workload is offloaded to cloud-based AI models. This extraction resolves the memory footprint constraint by removing the bulk of the model data from the device.
Solution Approach 2:
The patent introduces a cloud-based intermediary system that mediates between the edge device and the AI functionality. The cloud serves as an intermediary that hosts the large AI models and provides them to edge devices through network communication, allowing access to powerful AI capabilities without requiring the models to be stored locally.
2Adaptability or versatility
If diverse AI models are stored on edge devices for various purposes, then service versatility is improved, but device storage capacity is exhausted
Solution Approach 1:
The patent implements a universal cloud-based AI model repository that serves multiple purposes. A single cloud infrastructure can host diverse AI models for different services (image processing, natural language processing, data analysis, etc.), and the edge device can access any of these models as needed without requiring dedicated local storage for each function.
Solution Approach 2:
The patent moves the storage dimension from local device storage to cloud-based remote storage. Instead of storing models locally on the edge device, the system leverages the cloud's vast storage capacity, effectively adding a new spatial dimension to the storage architecture that resolves the device storage capacity constraint.
3Quantity of substance
If cloud-based AI models are used instead of local models, then memory requirements are reduced, but network dependency and latency increase
Solution Approach 1:
The patent implements a dynamic model selection mechanism that can switch between cloud-based models and locally cached models based on real-time conditions. When network conditions are good and model diversity is needed, the system uses cloud models; when network conditions are poor or speed is critical, the system switches to locally cached models, creating a dynamic adaptive system.
Solution Approach 2:
The patent performs preliminary actions by pre-downloading and caching frequently used AI models on the edge device before they are actually needed. This allows the system to have local copies of commonly accessed models readily available, reducing latency when these models are needed while still maintaining the option to use cloud models for less frequent or more specialized functions.
Data Source
AI summary
A device is in collaboration with a cloud. In response to a request for service, the device identifies one or more apps to serve the request by searching an on-device database that stores vector embeddings of features of on-device apps. The device ranks the features of the one or more apps based on similarities to requested features indicated in the request, and identifies an app having a highest-ranking feature as a target app. The device further identifies a target model for the target app. The target model is an edge artificial intelligence (AI) model when the highest-ranking feature is a local feature, or the target model is a cloud AI model when the highest-ranking feature is a non-local feature. The device then requests the target app to use the target model to serve the request.