AI Model Compression for Limited Memory Devices
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing artificial intelligence systems require significant time and storage capacity, making it infeasible to efficiently use multiple AI models on user terminal devices like smartphones due to limited memory resources.
Innovation Solution
An electronic apparatus compresses AI models based on available memory size, using methods like Pruning, Quantization, Decomposition, and Knowledge Distillation, to load them efficiently into smaller memory spaces while maintaining performance criteria, and dynamically adjusts compression based on usage and memory changes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If multiple artificial intelligence models are stored and loaded into memory, then the functionality and intelligence of the electronic apparatus is improved, but the memory size requirement increases significantly
Solution Approach 1:
The patent divides AI models into multiple segments or versions with different compression levels. Multiple versions of the same AI model are stored in the first memory, each optimized for different memory constraints. The system selectively loads appropriate segments into the second memory based on available space, allowing functionality to be maintained while reducing individual model size requirements.
Solution Approach 2:
The patent applies various compression techniques that change the parameters of AI models to reduce their size. Different compression methods (e.g., pruning, quantization, decomposition, knowledge distillation) transform the original models into compressed versions with reduced memory footprint while attempting to preserve performance. This allows multiple models to coexist in limited memory space.
2Volume of stationary object
If AI models are compressed to fit limited memory, then the memory usage is reduced, but the performance of the AI models deteriorates
Solution Approach 1:
The patent implements a dynamic model selection and loading mechanism that adapts to current memory conditions and usage patterns. The system monitors available memory space and dynamically selects which AI model versions to load into the second memory. When memory is abundant, higher-performance versions are loaded; when memory is constrained, compressed versions are used. This dynamic adaptation helps maintain performance while accommodating memory limitations.
Solution Approach 2:
The system incorporates feedback mechanisms to evaluate the performance of compressed AI models and adjust compression strategies accordingly. By monitoring the actual performance impact of compression and memory usage patterns, the system can refine its model selection and compression approach to minimize performance degradation while maintaining memory efficiency.
3Volume of stationary object
If compression techniques are applied to AI models, then the model size is reduced, but the complexity of the processing increases
Solution Approach 1:
The patent performs compression operations in advance during the model preparation and storage phase, rather than in real-time during execution. Multiple pre-compressed versions of AI models are prepared and stored in the first memory before deployment. This preliminary compression eliminates the need for complex real-time compression processing during runtime, reducing the processing complexity burden on the electronic apparatus while achieving the desired size reduction.
Data Source
AI summary
A method for controlling an electronic apparatus includes storing a plurality of artificial intelligence models in a first memory, based on receiving a control signal for loading a first artificial intelligence model among the plurality of stored artificial intelligence models into a second memory, identifying an available memory size of the second memory, and based on a size of the first artificial intelligence model being larger than the available memory size of the second memory, obtaining a first compression artificial intelligence model by compressing the first artificial intelligence model based om the available memory size of the second memory, and loading the first compression artificial intelligence model into the second memory.