Volatile Memory Space Recovery for Loading Large AI Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The increasing complexity and size of information stored in electronic devices, particularly for executing artificial intelligence models, often leads to memory saturation, causing delays and interruptions due to insufficient available space in volatile memory.
Innovation Solution
The electronic device employs a processor to identify events requiring loading of trained models into volatile memory, determines the required space, and uses single or multi-thread operations to obtain additional space through processes like compression swapping and terminating or compressing existing processes, ensuring the memory is not saturated.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If the size of information stored in electronic devices is increased to execute various functions including AI models, then the functionality and complexity of the device is improved, but the available space in volatile memory becomes insufficient leading to memory saturation
Solution Approach 1:
The system proactively identifies events that will require loading trained models into volatile memory before the memory becomes saturated. It calculates the required space in advance and initiates space recovery operations (compression swapping, terminating processes) beforehand to ensure sufficient space is available when the model loading event occurs, preventing memory saturation.
Solution Approach 2:
The system dynamically changes the state of memory resources by compressing data in non-volatile memory (changing density), swapping compressed data between memory tiers, and selectively terminating or suspending processes to free up volatile memory space. These parameter changes allow the same physical memory to accommodate larger AI models.
2Quantity of substance
If compression swapping and process termination are used to free up memory space, then available volatile memory space is increased, but device complexity and processing overhead increase
Solution Approach 1:
The system implements self-service memory management where the processor automatically monitors volatile memory usage, identifies when space is needed for AI model loading, and autonomously executes space recovery operations including compression swapping and process termination without requiring external intervention or complex manual management.
Solution Approach 2:
The system continuously monitors the usage state of volatile memory and uses this feedback to dynamically adjust memory management actions. When the monitor detects that available space is insufficient for an upcoming AI model load event, it triggers appropriate recovery operations, creating a closed-loop control system that adapts to real-time memory conditions.
3Quantity of substance
If multi-thread operations are used to obtain additional memory space rapidly, then memory space is secured faster, but processor execution state and energy consumption are affected
Solution Approach 1:
The system dynamically adjusts the degree of parallelism in memory space acquisition operations based on processor execution state. When the processor is in a low-utilization state, it can safely employ multi-threaded compression swapping operations to rapidly free up memory space. When the processor is highly utilized, it reduces parallelism to minimize energy consumption and avoid interfering with critical processing tasks.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
An electronic device identifies an event to load a trained model into volatile memory while executing a software application. The electronic device identifies a command, indicating a size of the trained model to be loaded into the volatile memory, based on the event. The command is generated by the software application. The electronic device identifies an execution state of a processor based on the size of the trained model indicated by the command being larger than a size of an available space within the volatile memory. The electronic device obtains an additional available space within the volatile memory for loading the trained model using a single thread based on an execution state of the processor in a first execution state. The electronic device obtains an additional available space using a multi-thread based on an execution state of the processor in a second execution state.