Automated Memory Allocation for Neural Networks on Microcontrollers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Memory allocation in artificial neural networks (ANNs) is inefficient, particularly in memory-constrained micro-controller units (MCUs), where existing solutions require manual hand-coding and are limited by NP-hard optimization problems, leading to increased implementation time and costs, and are not optimized for varying data types and sizes.
Innovation Solution
A computer-implemented method that couples data buffers in virtual memory circuitry to processing layers of ANNs, generates a virtual memory allocation schedule with instructions for memory allocation and deallocation, and computes a physical memory configuration dataset, using a directed acyclic graph to optimize memory usage and reduce memory usage at inference time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If manual hand-coding memory allocation is used, then memory allocation can be optimized for specific architectures, but implementation time and costs increase significantly
Solution Approach 1:
The system automatically generates memory allocation schedules by analyzing the neural network graph structure and computing resource requirements. The memory management unit autonomously creates allocation plans without manual intervention, selecting optimal memory regions and timing based on the computational graph dependencies, thereby eliminating the time-consuming manual hand-coding process while maintaining allocation optimization
Solution Approach 2:
The system performs preliminary analysis of the neural network architecture and computes resource requirements before actual memory allocation occurs. By pre-calculating the memory allocation schedule based on the computational graph and data flow dependencies, the system prepares optimized allocation plans in advance, enabling efficient automatic allocation during execution without manual intervention
2Device complexity
If fixed-size blocks allocation is used, then memory allocation is simplified, but memory fragmentation occurs especially with long memory addresses
Solution Approach 1:
The system segments memory allocation into variable-sized blocks based on actual neural network layer requirements rather than using uniform fixed-size blocks. Each memory allocation request is customized to match the specific data buffer size needed for particular computational operations, allowing flexible allocation that adapts to varying memory needs and prevents fragmentation while maintaining manageable complexity through automated allocation scheduling
3Loss of time
If automatic memory allocation is implemented, then implementation time is reduced, but optimization for varying data types and sizes becomes computationally hard
Solution Approach 1:
The system dynamically adjusts memory allocation strategies based on the actual computational graph structure and data flow requirements. Rather than using static fixed-size allocations, the memory management unit adapts allocation sizes and timing based on the specific operations identified in the neural network graph, enabling automatic allocation that handles varying data types and sizes efficiently without excessive computational complexity
Solution Approach 2:
The system changes allocation parameters such as block size, timing, and location based on the analyzed computational graph. By adjusting these parameters dynamically according to the neural network's operational requirements, the system achieves optimized memory allocation automatically, handling varying data types and sizes through parameter adaptation rather than complex optimization algorithms
4Ease of manufacture
If memory resources are limited in MCUs, then device cost is reduced, but the number of models that can be mapped is limited
Solution Approach 1:
The system performs preliminary analysis of the neural network model to identify critical computational layers and data flow dependencies before allocation. By pre-calculating the optimal memory allocation schedule based on the computational graph, the system can efficiently map models onto limited MCU memory resources, maximizing the number of supported models through intelligent scheduling rather than raw memory capacity
Solution Approach 2:
The memory allocation system dynamically adjusts the memory schedule based on the specific model requirements and available hardware resources. This dynamic scheduling allows the same limited memory capacity to support multiple different neural network models by optimizing allocation timing and size for each model's specific computational graph, thereby increasing adaptability without requiring additional hardware
Data Source
AI summary
Methods, microprocessors, and systems are provided for implementing an artificial neural network. Data buffers in virtual memory are coupled to respective processing layers in the artificial neural network. An ordered visiting sequence of layers of the artificial neural network is obtained. A virtual memory allocation schedule is produced as a function of the ordered visiting sequence of layers of the artificial neural network, the schedule including a set of instructions for memory allocation and deallocation operations applicable to the data buffers. A physical memory configuration dataset is computed as a function of the virtual memory allocation schedule for the artificial neural network, the dataset including sizes and addresses of physical memory locations for the artificial neural network.