Neural Network Code Generation with Layer-Specific Lookup Tables
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network processing on hardware platforms like microcontrollers is limited by the need for efficient data processing without expanding the hardware, which is constrained by manufacturing costs, space, power consumption, and heat generation, with activation function evaluation being a significant bottleneck.
Innovation Solution
Generating program code that creates a neural network with multiple lookup tables specific to each layer or group of neurons, allowing for efficient processing by precalculating and storing activation function values, thereby increasing flexibility and reducing computational time while managing memory effectively.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If a single shared lookup table is used for all neurons, then memory usage is reduced, but processing flexibility and speed are compromised due to complex index calculations
Solution Approach 1:
The patent divides the single shared lookup table into multiple layer-specific lookup tables, where each layer has its own dedicated lookup table. This segmentation eliminates the need for complex index calculations when switching between different activation functions across layers, as each layer's lookup table is optimized for its specific activation function. The segmentation resolves the contradiction by sacrificing some memory (using multiple tables instead of one) to gain significant processing speed through simplified index operations.
2Productivity
If hardware platform is expanded to improve processing power, then data processing speed increases, but manufacturing costs, installation space, power consumption, and heat generation increase
Solution Approach 1:
The patent changes the parameter organization of activation function data from a single shared structure to multiple layer-specific structures. This parameter reorganization enables the system to achieve faster processing speeds by eliminating complex index calculations, thereby improving productivity without requiring hardware expansion. The solution addresses the contradiction by optimizing software parameters rather than increasing hardware capacity.
3Productivity
If layer-specific lookup tables are created for each neuron group, then processing flexibility and speed improve, but memory consumption increases
Solution Approach 1:
The patent applies preliminary action by pre-calculating and storing activation function values in layer-specific lookup tables during the model compilation phase. This advance preparation enables the inference phase to simply retrieve pre-computed values through simple index operations, significantly improving processing efficiency during runtime. The memory overhead is accepted as a trade-off for the substantial speed improvement during actual neural network inference.
Data Source
AI summary
A method for generating program code which, when executed on a hardware platform, creates a neural network having a given architecture. In the method: for at least one layer and/or group of neurons, a non-linear activation function of the neurons in that layer and/or group is ascertained from the given architecture; possible values that can be assumed by the activation function are precalculated and stored in a lookup table; program code is generated which, for all neurons in the layer and/or group respectively: aggregates the inputs of the respective neuron to form an argument of the activation function in accordance with the given architecture, ascertains an index from this argument, under which the associated value of the activation function is stored in the lookup table for the respective layer or group, and ascertains the output of the neuron by retrieving the value from the lookup table with this index.