Concurrent LSTM Cell Calculation in Neural Network Unit
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current processors lack efficient mechanisms for performing neural network computations, particularly in deep learning applications, due to limitations in handling large datasets and complex operations within artificial neural networks.
Innovation Solution
A processor architecture incorporating a neural network unit (NNU) with multiple neural processing units (NPUs) that perform concurrent operations, including multiply-accumulate functions and activation functions, and utilize shared resources like weight and data RAMs, along with architectural instructions for efficient data processing and memory management.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional processors are used for neural network computations, then general-purpose processing capability is maintained, but computation performance and efficiency for deep learning applications deteriorates
Solution Approach 1:
The processor is segmented into separate functional domains: a general-purpose processor for control and coordination, and dedicated neural network units for computation. Each NNU contains multiple NPUs organized in arrays, allowing specialized optimization for neural network operations while maintaining overall system manageability through modular architecture.
Solution Approach 2:
The neural network units are designed with multi-functionality to handle various neural network operations including multiply-accumulate, activation functions, and concurrent LSTM cell calculations. The NPUs can be dynamically configured to process different data sizes and perform multiple functions within the same hardware structure.
2Loss of time
If sequential processing is used for neural network operations, then simplicity of execution is maintained, but computation time and clock cycles increase
Solution Approach 1:
Multiple neural processing units are merged into arrays that can execute operations concurrently. The NPUs are organized in structures that allow simultaneous multiply-accumulate operations across multiple neurons, reducing total computation time while sharing common resources like weight RAM and data RAM to manage complexity.
Solution Approach 2:
The neural network units implement continuous pipelined operation where NPUs can perform multiply-accumulate operations while others are loading weights or performing activation functions. This overlapping of operations ensures continuous useful computation without idle cycles, significantly reducing total execution time.
3Productivity
If dedicated neural network units are added to processors, then neural network computation efficiency is improved, but overall processor complexity increases
Solution Approach 1:
The processor is segmented into separate functional domains: a general-purpose processor for control and coordination, and dedicated neural network units for computation. Each NNU contains multiple NPUs organized in arrays, allowing specialized optimization for neural network operations while maintaining overall system manageability through modular architecture.
Solution Approach 2:
The neural network units are designed with multi-functionality to handle various neural network operations including multiply-accumulate, activation functions, and concurrent LSTM cell calculations. The NPUs can be dynamically configured to process different data sizes and perform multiple functions within the same hardware structure.
4Speed
If large datasets are processed sequentially, then memory access simplicity is maintained, but processing speed and throughput deteriorate
Solution Approach 1:
Multiple neural processing units are merged into arrays that can execute operations concurrently. The NPUs are organized in structures that allow simultaneous multiply-accumulate operations across multiple neurons, reducing total computation time while sharing common resources like weight RAM and data RAM to manage complexity.
Solution Approach 2:
The architecture transitions from sequential one-dimensional processing to parallel multi-dimensional processing. NPUs are arranged in two-dimensional arrays that can simultaneously process multiple data elements across different spatial dimensions, enabling batch processing of large datasets and significantly increasing throughput.
Data Source
AI summary
An output buffer holds N words arranged as N/J mutually exclusive output buffer word groups (OBWG) of J words each of the N words. N processing units (PU) are arranged as N/J mutually exclusive PU groups. Each PU group has an associated OBWG. Each PU includes an accumulator and an arithmetic unit that performs operations on inputs, which include the accumulator output, to generate a first result for accumulation into the accumulator. Activation function units selectively perform an activation function on the accumulator outputs to generate results for provision to the N output buffer words. For each PU group, four of the J PUs and at least one of the activation function units compute an input gate, a forget gate, an output gate and a candidate state of a Long Short Term Memory (LSTM) cell, respectively, for writing to respective first, second, third and fourth words of the associated OBWG.