Multiple Busses in Grouped Systolic Array for Parallel Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Systolic arrays used in artificial neural networks face challenges in achieving high inference throughput while minimizing power consumption and latency, as increasing clock speed leads to increased power consumption and cooling requirements.
Innovation Solution
Implementing multiple busses within the columns and rows of a systolic array to increase parallelism and reduce latency, allowing for longer latency in individual operations and enabling the use of lower-cost, lower-power components without compromising overall latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If clock speed is increased to achieve high inference throughput, then processing speed is improved, but power consumption increases
Solution Approach 1:
The patent divides the data path into multiple parallel busses (first bus, second bus, third bus, fourth bus) that process data simultaneously. Each bus handles a portion of the computation, allowing the system to achieve high throughput without requiring all components to operate at maximum speed, thus reducing overall power consumption while maintaining performance.
2Loss of time
If clock speed is increased to reduce latency, then inference time is improved, but cooling requirements increase
Solution Approach 1:
By segmenting the data flow into multiple parallel busses, the patent reduces the critical path length and enables lower latency without requiring excessive clock speeds. This distributed approach allows each bus to operate at moderate speeds, generating less heat and reducing cooling requirements while still achieving fast inference.
3Loss of time
If multiple busses are implemented to reduce latency, then parallelism is improved, but device complexity increases
Solution Approach 1:
The patent implements a unified systolic array structure where each processing element can operate on multiple busses simultaneously. The same array architecture serves both single-bus and multi-bus modes, allowing the system to achieve reduced latency through multi-bus operation without requiring completely different hardware designs, thus managing complexity while improving performance.
Data Source
AI summary
Systems and methods are provided to enable parallelized multiply-accumulate operations in a systolic array. Each row of the systolic array can include multiple busses enabling independent transmission of inputs along the respective bus. Each processing element of a given row-oriented bus can receive an input from a prior element of the given row-oriented bus, and perform arithmetic operations on the input. The systolic array can be divided into a plurality of sub-arrays corresponding to a row-oriented bus where each sub-array is separated by a shifter. Each shifter can shift a row-oriented bus into the active bus position for a given sub-array. Use of row-oriented busses can enable parallelization to increase speed or enable increased latency at individual processing elements.