Neural Network Vector Computation Unit with Activation and Normalization Circuitry
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies face challenges in efficiently computing neural network inferences, particularly in handling multiple activation values and normalizing outputs across layers in a timely manner.
Innovation Solution
A special-purpose hardware circuit is designed to perform neural network computations, featuring activation circuitry for applying functions to accumulated values and normalization circuitry for generating normalized values, enabling pipelined computations across layers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If general-purpose processors are used for neural network computations, then flexibility and adaptability are maintained, but computation speed and processing efficiency deteriorate
Solution Approach 1:
The processor is divided into multiple independent processing elements (PEs) organized in a grid, where each PE can independently perform multiply-accumulate operations. This segmentation enables parallel processing of neural network computations while maintaining the ability to handle different network configurations through selective activation of PEs.
Solution Approach 2:
The patent introduces a spatial dimension by organizing PEs in a two-dimensional grid and using three-dimensional memory structures (depth, rows, columns) to store weights and activations. This multi-dimensional organization enables efficient data access patterns for various neural network operations while maintaining computational flexibility.
2Productivity
If more processing elements are added to increase computation throughput, then productivity improves, but device complexity and resource requirements worsen
Solution Approach 1:
Each processing element is designed as a universal unit capable of performing the same set of operations (multiply-accumulate, activation functions) on different data. This uniformity simplifies the control logic and resource management compared to having specialized units for different operations, reducing overall system complexity while maintaining high throughput.
Solution Approach 2:
Multiple processing elements share common resources including activation function units, normalization units, and memory controllers. By merging these resources, the patent reduces the total number of components needed compared to having dedicated resources for each PE, thereby reducing device complexity while maintaining computational throughput.
3Measurement precision
If activation functions and normalization are computed for each neuron individually, then processing accuracy is maintained, but computation time and processing delay increase
Solution Approach 1:
The patent combines multiple PEs into processing groups that share activation function units and normalization units. Multiple neurons within a group can have their activation functions and normalization computed simultaneously by shared units, reducing total computation time while maintaining individual neuron processing accuracy through precise floating-point operations.
Solution Approach 2:
The processor performs preliminary computations by calculating activation values and normalization factors in parallel across multiple PEs before final output generation. This preliminary parallel processing reduces the critical path delay and overall computation time while maintaining accuracy through consistent mathematical operations.
Data Source
AI summary
A circuit for performing neural network computations for a neural network comprising a plurality of layers, the circuit comprising: activation circuitry configured to receive a vector of accumulated values and configured to apply a function to each accumulated value to generate a vector of activation values; and normalization circuitry coupled to the activation circuitry and configured to generate a respective normalized value from each activation value.