AI Model RAM Loading Based on Application Usage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The local execution of machine learning models at information handling systems consumes hardware resources needed for other applications, leading to resource conflicts and reduced system performance.
Innovation Solution
An OTB AI productivity tool automatically manages the loading and unloading of machine learning models in RAM based on their usage by AI productivity tool enableable software applications, balancing resource consumption by moving inactive models to SSD storage when not in use.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If machine learning models are stored in RAM for local execution, then model access speed and application functionality are improved, but hardware resource utilization deteriorates due to conflicts with other applications
Solution Approach 1:
The system dynamically manages machine learning model storage by automatically loading models into RAM when needed by AI productivity tool enableable software applications and unloading them when not in use. This dynamic loading/unloading mechanism allows the system to adapt RAM allocation based on actual demand, resolving the contradiction between maintaining fast model access and preserving RAM resources for other applications.
Solution Approach 2:
The system performs preliminary actions by pre-loading machine learning models into RAM when AI productivity tool enableable software applications are launched or when models are predicted to be needed. This anticipatory loading ensures models are readily available when required, improving access speed without requiring all models to remain permanently in RAM.
2Adaptability or versatility
If multiple machine learning models are kept in RAM simultaneously, then application functionality and model availability are improved, but system performance deteriorates due to resource conflicts
Solution Approach 1:
The system implements a discard and recover mechanism where machine learning models are automatically unloaded from RAM when no longer needed by AI productivity tool enableable software applications. The models are preserved in storage and can be reloaded when needed again, ensuring model availability while freeing up RAM resources to maintain system performance.
Solution Approach 2:
The RAM memory is designed to serve multiple functions by dynamically allocating space for machine learning models only when AI productivity tool enableable software applications require them. The same RAM resources can be used for other applications when machine learning models are not in use, achieving universal resource utilization that balances model availability with overall system performance.
3Ease of operation
If machine learning models are automatically loaded into RAM, then application responsiveness is improved, but hardware resource consumption increases
Solution Approach 1:
The system employs self-service mechanisms where the machine learning model access coordination module automatically monitors usage patterns and manages model loading/unloading without manual intervention. This automated resource management ensures applications remain responsive by keeping needed models in RAM while minimizing hardware resource consumption by unloading unused models, eliminating the need for users to manually balance these competing requirements.
Data Source
AI summary
An information handling system operating an On the Box (OTB) Artificial Intelligence (AI) productivity tool may comprise a first solid state data storage device for storing a machine learning model, and a hardware processor for executing code instructions of a software application and of a machine learning model access coordination module to receive a request for the software application to access the machine learning model, store the machine learning model in a second random access memory (RAM) data storage device, direct the software application to provide input into the machine learning model, detect a period of time exceeding a machine learning model unloading countdown timer has elapsed since the software application has last provided input values into the machine learning model, and remove the machine learning model from RAM to decrease hardware component resource consumption at the information handling system when the machine learning model is unused.