Neural Processor Activation Parameter Reuse for Lower Task Bandwidth
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning systems rely heavily on central processing units (CPUs) for neural network operations, leading to significant bandwidth consumption and increased power usage due to the extensive computing requirements of operations like convolution and accumulation.
Innovation Solution
A neural processor circuit with programmable memory circuits stores parameters representing nonlinear activation functions, allowing these parameters to be reused across tasks, reducing the need to include them in subsequent task descriptors and optimizing CPU usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If parameters representing nonlinear functions are included in every task descriptor, then the neural processor can execute tasks independently, but the data transmission bandwidth and power consumption increase significantly
Solution Approach 1:
The patent implements a universal parameter storage mechanism where a single set of parameters representing nonlinear functions is shared across multiple task descriptors. Instead of duplicating parameters in each task descriptor, the system uses a common storage location that can be referenced by multiple tasks, reducing redundant data transmission and power consumption while maintaining task execution independence
Solution Approach 2:
The patent pre-loads and stores parameters representing nonlinear functions in a dedicated storage location before task execution begins. This preliminary action allows subsequent tasks to reference these pre-stored parameters without requiring repeated data transmission, thereby reducing bandwidth consumption and power usage during actual task execution
2Loss of energy
If parameters are stored in programmable memory circuits for reuse, then power consumption is reduced, but the device complexity increases
Solution Approach 1:
The patent segments the memory system into distinct functional components: programmable memory circuits for storing parameters representing nonlinear functions, and separate task descriptor storage. This segmentation allows the system to optimize power consumption by selectively accessing the programmable memory only when parameters need to be loaded, while subsequent tasks can reference these stored parameters efficiently, balancing power savings with manageable device complexity
3Adaptability or versatility
If CPU is used to instantiate and execute machine learning systems, then system configuration is flexible and easy, but bandwidth consumption and power usage increase significantly
Solution Approach 1:
The patent introduces a neural processor as an intermediary between the CPU and the actual neural network execution. The CPU retains its role in high-level system configuration and task management, while the neural processor handles the computationally intensive neural network operations using dedicated hardware including programmable memory circuits for parameter storage. This intermediary architecture maintains the flexibility of CPU-based configuration while dramatically reducing power consumption during actual neural network execution
Data Source
AI summary
Embodiments of the present disclosure relate to storing parameters representing nonlinear functions in programmable memory circuits of a neural processor circuit and reusing the stored parameters across multiple tasks. The parameters are initially included in a task descriptor defining the configuration of the neural processor circuit for a task and are programmed into programmable memory circuits. Parameters for other nonlinear functions are stored in non-programmable memory circuits. In subsequent tasks, the stored parameters are reused to generate activation values for applying to processed output from multiply-accumulate (MAC) circuit by indicating, in task descriptors for the subsequent tasks, programmable or nonprogrammable memory circuits from which the parameters are to be retrieved. By replacing the parameters of the nonlinear functions with the indication of the memory circuits in the subsequent tasks, the amount of data to be included in the task descriptors of the subsequent tasks is reduced.