Terminal AI Model Management for Memory-Constrained Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing user terminal environments are unable to effectively integrate and manage large AI models due to limitations in memory and inference-specific chips, leading to privacy concerns and network delays when using external AI models, while lightweight models lack sufficient functionality and integration capabilities.
Innovation Solution
A method and system for managing AI models on user terminals by storing device information, determining compatibility, and setting access and operation parameters, including secure data management and model verification, to ensure safe and efficient installation and operation of AI models.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If large AI models are installed on user terminals, then AI performance and functionality are improved, but device memory and processing capabilities are exceeded
Solution Approach 1:
The patent segments the AI model into multiple parts: a smaller base model that fits within device memory and larger parameter files that can be stored externally or in cloud storage. The system loads only necessary portions of the model into device memory during operation, enabling large model functionality while respecting device memory constraints.
Solution Approach 2:
The patent introduces a new dimension of storage by utilizing external storage resources (cloud storage, external drives) beyond the device's internal memory. This allows the system to access AI model parameters that exceed the physical memory capacity of the device, effectively expanding the available resource space.
2Adaptability or versatility
If external AI models are used, then advanced AI capabilities are achieved, but user data privacy is compromised and network delays occur
Solution Approach 1:
The patent implements local quality by keeping the base AI model and user data processing localized on the user's device, ensuring data privacy and fast response times. Only non-sensitive model parameters are stored externally, while the actual inference and data processing occur locally on the device, combining the benefits of both local and external resources.
3Ease of operation
If lightweight AI models are installed on terminals, then device compatibility is improved, but AI functionality and performance are limited
Solution Approach 1:
The patent creates a universal AI model architecture that can function across devices of varying capabilities. The base model is designed to run on resource-constrained devices, while optionally loading additional parameters from external storage to enhance functionality. This allows the same model to provide appropriate AI capabilities on both mobile phones and powerful workstations.
Data Source
AI summary
Method and system for managing artificial intelligence model installed and operating in terminal environment are disclosed. A method for managing an artificial intelligence model according to one embodiment may include storing and updating device information of each of a plurality of user terminals that download and install an artificial intelligence model from an artificial intelligence model store, and determining whether or not a first artificial intelligence model can be installed on a first user terminal based on device information of a first user terminal according to an installation request for the first artificial intelligence model of the first user terminal among the plurality of user terminals.