Parallel Neural Inference Elements With Dendritic Tree Architecture
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current neural network inference processing architectures have a sequential nature, limiting the degree of parallelism despite the fundamentally parallel nature of neural network computations, which restricts the ability to perform massive parallel computations.
Innovation Solution
The implementation of a massively parallel neural inference computing system with multiple multipliers and adders operating in parallel, along with dedicated partial sum operators and nonlinear activation circuits, enables efficient parallel computation of neuron activations using a dendritic tree architecture and pipelined circuits.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a sequential processing architecture is used for neural network inference, then the device complexity is reduced and ease of manufacture is improved, but the productivity and degree of parallelism are significantly limited
Solution Approach 1:
The neural network inference computation is segmented into multiple independent processing elements (PEs), each capable of performing multiply-accumulate operations in parallel. The architecture divides the computational workload across numerous PEs organized in a grid structure, with each PE handling specific neuron computations independently, thereby achieving massive parallelism while maintaining manageable complexity through modular design
Solution Approach 2:
The architecture transitions from sequential one-dimensional processing to two-dimensional parallel processing by organizing PEs in a grid layout. This spatial dimensionality enables simultaneous execution of multiple computations across different PEs, dramatically increasing productivity without proportionally increasing control complexity
2Productivity
If more multipliers and adders are added to increase parallelism, then the productivity increases, but the device complexity and manufacturing difficulty increase
Solution Approach 1:
Each processing element is designed as a universal module that can perform multiple functions (multiplication, accumulation, activation function application) within a standardized architecture. This universality allows the same PE design to be replicated numerous times across the chip, simplifying manufacturing processes while enabling scalable parallelism through simple replication of the proven modular unit
Solution Approach 2:
The architecture employs a hierarchical nested structure where individual PEs are nested within larger processing blocks, which are in turn nested within the complete neural network accelerator. This nested organization allows complex functionality to be built from simpler modular components, making the overall system manufacturable through systematic assembly of standardized units
Data Source
AI summary
Massively parallel neural inference computing elements are provided. A plurality of multipliers is arranged in a plurality of equal-sized groups. Each of the plurality of multipliers is adapted to, in parallel, apply a weight to an input activation to generate an output. A plurality of adders is operatively coupled to one of the groups of multipliers. Each of the plurality of adders is adapted to, in parallel, add the outputs of the multipliers within its associated group to generate a partial sum. A plurality of function blocks is operatively coupled to one of the plurality of adders. Each of the plurality of function blocks is adapted to, in parallel, apply a function to the partial sum of its associated adder to generate an output value.