DNN Memory Buffer Reuse Across Layers to Cut Memory Footprint
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep neural networks (DNNs) face challenges in deploying on low-power devices due to high memory bandwidth and computational requirements, resulting in increased memory allocation and computational demands, leading to inefficient memory usage and performance issues.
Innovation Solution
A method and apparatus for determining memory requirements by identifying reusable memory buffers and buffer overlap across processing layers of DNN models, using a colored network graph to optimize memory allocation on embedded devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If conventional DNN processing is used with separate memory buffers for each processing layer, then each layer can be processed independently, but the total memory requirement increases significantly
Solution Approach 1:
The patent merges memory buffers across multiple processing layers by identifying that certain buffers can be reused. Instead of allocating separate buffers for each layer, the system combines buffer usage so that a single buffer can serve multiple layers sequentially, reducing total memory footprint while maintaining processing functionality.
Solution Approach 2:
The patent makes memory buffers universal by enabling them to perform multiple functions across different processing layers. A buffer allocated for one layer can be reused for another layer after its data is no longer needed, making the same memory resource serve multiple purposes throughout the DNN processing pipeline.
2Speed
If memory buffers are allocated for all processing layers simultaneously, then all layers can access memory at once, but memory bandwidth requirements increase
Solution Approach 1:
The patent performs preliminary analysis of the DNN processing graph to identify buffer reuse opportunities before actual processing begins. By pre-determining which buffers can be shared and when they can be safely reused, the system prepares a memory allocation plan that minimizes bandwidth usage while ensuring all layers can access memory when needed.
Solution Approach 2:
The patent implements dynamic memory buffer management where buffer allocation and reuse decisions are made based on the actual execution timeline of processing layers. Instead of static allocation, the system dynamically reuses buffers as layers complete their processing, optimizing memory bandwidth usage while maintaining the speed needed for active layers.
3Ease of operation
If independent memory buffers are assigned to each processing layer, then memory management is simplified, but overall memory footprint increases
Solution Approach 1:
The patent segments the DNN processing graph into independent units where buffer reuse can be identified and implemented. By dividing the processing workflow into discrete layers with clear input-output dependencies, the system can determine which segments can share buffers without complicating memory management, as each segment's memory needs are clearly defined.
Solution Approach 2:
The patent changes the memory allocation parameters from fixed per-layer buffers to dynamic shared buffers based on processing timelines. This parameter change allows the same memory region to be allocated to different layers at different times, reducing total memory footprint while maintaining ease of operation through automated buffer management.
Data Source
Figure 1
Figure 2A
Figure 2B
AI summary
The present disclosure disclose method and apparatus for determining memory requirement for processing a DNN model on a device, a method includes receiving a DNN model for an input, wherein the DNN model includes a plurality of processing layers. The method includes generating a network graph of the DNN model. The method includes creating a colored network graph of the DNN model based on the identified execution order of the plurality of processing layers. The colored network graph indicates assignment of at least one memory buffer for storing at least one output of at least one processing layer. The method includes determining at least one buffer reuse overlap possibility across the plurality of processing layers. Based on the determined at least one buffer reuse overlap possibility, the method includes determining and assigning the memory required for processing the DNN model.