Deep Learning Memory Allocation Controller
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current deep learning models processed using only dynamic random access memory (DRAM) are unable to minimize the time required for processing due to the limitations of DRAM in handling complex neural networks efficiently.
Innovation Solution
An apparatus and method that detect memory usage and processing times for both static random access memory (SRAM) and DRAM for each layer of a deep learning model, determining an optimal memory configuration using an objective function to minimize processing time, which includes a controller to manage memory allocation and usage between SRAM and DRAM.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If only DRAM is used to process deep learning models, then memory capacity is sufficient, but processing time cannot be minimized
Solution Approach 1:
The patent divides the memory system into two distinct segments: SRAM (first memory) for high-speed processing and DRAM (second memory) for large-capacity storage. Each layer of the deep learning model is segmented to use appropriate memory types based on their specific requirements, allowing simultaneous optimization of both speed and capacity without requiring a single complex memory system.
Solution Approach 2:
The patent introduces a new dimension of memory hierarchy by combining two different memory types (SRAM and DRAM) with fundamentally different characteristics into a unified system. This dimensional approach allows the system to access both the speed advantage of SRAM and the capacity advantage of DRAM, resolving the contradiction between processing time and memory capacity.
2Speed
If SRAM is used for all layers, then processing speed increases, but memory capacity becomes insufficient
Solution Approach 1:
The patent applies local quality by assigning different memory types to different layers of the deep learning model based on their specific needs. Layers requiring high-speed access use SRAM, while layers requiring large capacity use DRAM. This localized optimization allows the system to achieve high processing speed where needed without sacrificing overall memory capacity.
Solution Approach 2:
The patent implements dynamic memory allocation where the controller can flexibly assign SRAM or DRAM to different layers based on real-time requirements. This dynamic approach allows the system to adapt to varying processing demands, allocating high-speed SRAM to critical layers while using capacity-oriented DRAM for less time-sensitive layers, thus balancing speed and capacity requirements.
3Productivity
If memory allocation is optimized per layer, then processing time is minimized, but system complexity increases
Solution Approach 1:
The patent introduces a controller as an intermediary component that manages the complex task of allocating SRAM and DRAM to different layers. This intermediary handles the complexity of memory management, allowing individual layers to benefit from optimized memory assignment without requiring each layer to independently manage the complexity. The controller centralizes the decision-making process, improving processing efficiency while containing system complexity in a dedicated management component.
Data Source
AI summary
An apparatus for processing a deep learning model includes a first memory, a second memory, and a controller. The controller is configured to, for each layer of the deep learning model, detect memory usage, a first processing time corresponding to the first memory being used, and a second processing time corresponding to the second memory being used, and determine an optimal memory for each layer of the deep learning model based on an objective function.