Neural Network Compiler Static Memory Allocation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deploying neural networks on resource-constrained specialized processors is challenging due to incompatibility with structural and operational requirements, often necessitating additional computing power or memory, which is not feasible on low-power devices like Always On Processors (AOPs).
Innovation Solution
The technology reduces the memory footprint of neural networks by reusing memory portions and allocating all memory at compile time based on resource constraints, avoiding dynamic memory allocation, enabling neural networks to run on specialized processors by generating optimized code that is compatible with the target device's architecture.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by stationary object
If neural networks are deployed on resource-constrained specialized processors, then power consumption is reduced and device portability is improved, but the neural network cannot execute due to incompatibility with structural and operational requirements
Solution Approach 1:
The compiler transforms the neural network model by changing parameters such as data types (e.g., from float32 to int8), precision requirements, and memory allocation strategies to match the capabilities and constraints of the specialized processor, enabling execution while maintaining acceptable accuracy
Solution Approach 2:
The neural network model is segmented into discrete operations and layers that can be independently optimized and mapped to specific processor units, allowing the compiler to distribute computations across available hardware resources efficiently
2Reliability
If additional computing power or memory is allocated to neural networks, then execution capability is improved, but device complexity increases and resource constraints are violated
Solution Approach 1:
The compiler applies partial precision calculations where full precision is not necessary, using techniques like mixed precision computation and selective rounding to reduce memory and computational requirements while maintaining sufficient accuracy for the application
Solution Approach 2:
The compiler performs preliminary optimization during the compilation phase, pre-calculating and pre-allocating memory structures, fusing operations that can be combined, and eliminating redundant computations before the neural network is deployed to the resource-constrained device
3Adaptability or versatility
If dynamic memory allocation is used for neural networks, then flexibility is improved, but execution on specialized processors fails due to lack of dynamic allocation support
Solution Approach 1:
The compiler performs preliminary analysis of the neural network's memory requirements and generates static memory allocation tables during compilation, assigning fixed memory addresses and sizes to all intermediate activations and parameters before deployment, eliminating the need for runtime memory management
Solution Approach 2:
Instead of allocating memory dynamically during execution and then managing it, the compiler inverts the approach by pre-allocating all memory structures during compilation and generating code that directly accesses these predetermined memory locations, reversing the traditional memory management paradigm
Data Source
AI summary
The subject technology receives a neural network model in a model format, the model format including information for a set of layers of the neural network model, each layer of the set of layers including a set of respective operations. The subject technology generates neural network (NN) code from the neural network model, the NN code being in a programming language distinct from the model format, and the NN code comprising a respective memory allocation for each respective layer of the set of layers of the neural network model, where the generating comprises determining the respective memory allocation for each respective layer based at least in part on a resource constraint of a target device. The subject technology compiles the NN code into a binary format. The subject technology generates a package for deploying the compiled NN code on the target device.