ML Model Memory Partitioning for Low-Power SRAM Execution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional techniques for storing machine learning models in SRAM consume significant power, leading to delays and increased power consumption, while storing in secondary storage results in longer loading times.
Innovation Solution
A system that partitions a model into an active segment in SRAM and a low-power segment, maintaining a mapping between the model identifier and the low-power segment's starting address, allowing seamless transition of the low-power segment to active state during execution, thereby hiding latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If SRAM is kept active to store the model, then model access speed is improved, but power consumption increases significantly
Solution Approach 1:
The model is divided into two segments: a first segment stored in active SRAM for fast access during execution, and a second segment stored in low-power memory (such as retain mode SRAM or MRAM) for persistent storage. This segmentation allows the system to maintain fast model access during operation while reducing overall power consumption by keeping less frequently accessed model data in low-power memory.
Solution Approach 2:
The system dynamically switches between active and low-power memory states based on execution needs. During model execution, the active SRAM segment is accessed for fast read operations. When execution completes or pauses, the system transitions the second segment from low-power memory to active memory, enabling dynamic adaptation between performance and power saving modes.
2Use of energy by moving object
If SRAM is power gated to reduce power consumption, then power usage is reduced, but data retention is lost causing loading delays
Solution Approach 1:
The model is segmented such that the first segment remains in active SRAM (maintaining data retention without power gating), while the second segment is stored in retain mode memory that can be power-gated but still retains data. This allows the system to power gate the second segment to save energy while avoiding the need to reload the entire model, thus reducing loading delays.
Solution Approach 2:
The system performs preliminary action by pre-loading the first segment of the model into active SRAM before execution begins. This ensures that the most frequently accessed parts of the model are immediately available in fast memory, eliminating the need to wait for full model loading during execution and reducing perceived loading delays.
3Use of energy by moving object
If the entire model is stored in secondary storage to reduce power usage, then power consumption is reduced, but loading time increases significantly
Solution Approach 1:
The model is divided into a first segment for active SRAM storage and a second segment for low-power memory storage. By storing only the essential first segment in active memory, the system reduces the amount of data that needs to be quickly loaded, thereby reducing loading time while still achieving lower power consumption compared to keeping the entire model in active SRAM.
Solution Approach 2:
The system uses partial action by loading only the necessary first segment of the model into active SRAM rather than the entire model. This partial loading approach reduces the immediate memory bandwidth requirements and loading time while maintaining sufficient functionality for execution, achieving a balance between speed and power consumption.
Data Source
Figure 1
Figure 2
Figure 3A
AI summary
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for reducing the power usage of storing a model in memory. One of the methods includes storing a first portion of a machine learning model in an active segment of memory and a second portion of the model in a low-power segment of the memory; maintaining a mapping between a model identifier of the model and a starting address of the second portion of the model; and maintaining a power state mapping. The method further includes receiving a request to execute the model. In response, the method includes initiating execution of the model using the first portion of the model and determining the power state of the low-power segment. In response to determining that the low-power segment is in a low-power state, the method further includes transitioning the low-power segment to being another active segment.