Parallel Neural Inference Elements With Pipelined Partial-Sum Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current neural network inference processing architectures have a sequential nature that limits parallelism, despite the underlying neural network's potential for massive parallelism, leading to slower computation and increased latency.
Innovation Solution
The implementation of a massively parallel neural inference computing system with multiple multipliers and adders operating in parallel, along with dedicated partial sum operators and nonlinear activation circuits, enables efficient computation of neuron activations by distributing activations and employing low-precision ternary vector-matrix multiplication and pipelining.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If a sequential processing architecture is used for neural network inference, then device complexity is reduced and ease of manufacture is improved, but computation speed and parallelism are limited
Solution Approach 1:
The patent divides the neural network computation into multiple independent processing elements (PEs), each capable of performing multiply-accumulate operations in parallel. The computation is segmented across spatial dimensions (multiple PEs) and temporal dimensions (pipelining), allowing simultaneous execution of multiple neural network operations without increasing individual PE complexity
Solution Approach 2:
The patent introduces temporal dimension through pipelining, allowing multiple stages of computation to overlap in time. Data flows through multiple pipeline stages (e.g., weight storage, computation, accumulation) simultaneously, effectively increasing parallelism without requiring proportionally more hardware resources
2Productivity
If low-precision ternary vector-matrix multiplication is implemented, then computation speed and parallelism are increased, but manufacturing precision requirements are reduced
Solution Approach 1:
The patent changes the precision parameter from standard floating-point to low-precision ternary representation for weights and activations. This parameter change enables more efficient hardware implementation with simpler multipliers and adders, increasing throughput while the precision loss is compensated through architectural techniques like increased parallelism and pipelining
3Measurement precision
If dedicated partial sum operators and activation circuits are added for each neuron, then computation accuracy is improved, but device complexity increases
Solution Approach 1:
The patent designs universal processing elements that can perform multiple functions: weight storage, multiply-accumulate operations, partial sum accumulation, and activation function application. Each PE is a multi-functional unit that handles different stages of neural network computation, reducing overall system complexity through functional integration rather than separate dedicated circuits
Data Source
AI summary
Massively parallel neural inference computing elements are provided. A plurality of multipliers is arranged in a plurality of equal-sized groups. Each of the plurality of multipliers is adapted to, in parallel, apply a weight to an input activation to generate an output. A plurality of adders is operatively coupled to one of the groups of multipliers. Each of the plurality of adders is adapted to, in parallel, add the outputs of the multipliers within its associated group to generate a partial sum. A plurality of function blocks is operatively coupled to one of the plurality of adders. Each of the plurality of function blocks is adapted to, in parallel, apply a function to the partial sum of its associated adder to generate an output value.