Systolic Array Multiple Busses Interleaved
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Systolic arrays used in artificial neural networks face challenges in reducing latency while minimizing power consumption, as increasing clock speed leads to increased power consumption and cooling requirements, limiting performance improvements.
Innovation Solution
Implementing multiple busses within the columns and rows of a systolic array allows for sequence skipping, reducing latency and enabling the use of lower-cost, lower-power processing elements by decreasing the total path-length of calculations and increasing parallelism, thereby maintaining overall latency without significant power consumption increases.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If clock speed is increased to reduce latency, then operational speed is improved, but power consumption increases
Solution Approach 1:
The patent divides the systolic array into multiple independent busses (first bus, second bus, third bus, fourth bus) that operate in parallel. Each bus handles a subset of data flow, allowing the system to achieve higher throughput without increasing the clock speed of individual processing elements. This segmentation enables the array to process multiple data streams simultaneously, improving effective operational speed while maintaining lower power consumption per element.
Solution Approach 2:
The patent introduces an additional dimension of parallelism by implementing multiple busses within each row and column of the systolic array. Instead of increasing the speed of a single data path, the invention creates multiple concurrent data paths (first bus, second bus, third bus, fourth bus) that operate simultaneously. This multi-dimensional approach to data flow allows the system to achieve higher overall throughput without requiring higher clock speeds, thereby avoiding increased power consumption.
2Loss of time
If multiple busses are implemented to increase parallelism, then latency is reduced, but device complexity increases
Solution Approach 1:
The patent designs processing elements that can interface with multiple busses (first bus, second bus, third bus, fourth bus) using the same basic architecture. Each processing element is configured to receive inputs from and provide outputs to multiple busses without requiring fundamentally different circuitry. This universal design allows the system to achieve high parallelism and reduced latency while avoiding a proportional increase in device complexity, as the same processing element type can serve multiple bus functions.
Solution Approach 2:
The patent implements multiple busses (first bus, second bus, third bus, fourth bus) selectively within specific rows and columns of the systolic array rather than uniformly across the entire array. This localized approach allows the system to reduce latency in critical data paths while maintaining simpler architecture in less critical areas. The selective implementation of multiple busses optimizes the balance between latency reduction and complexity management by applying parallelism only where most beneficial.
Data Source
AI summary
Systems and methods are provided to enable parallelized multiply-accumulate operations in a systolic array. Each row of the systolic array can include multiple busses enabling independent transmission of inputs along the respective bus. Each processing element of a given row-oriented bus can receive an input from a prior element of the given row-oriented bus, and perform arithmetic operations on the input. Each processing element can generate an output partial sum based on the arithmetic operations, provide the input to a next processing element of the given row-oriented bus, without the input being processed by a processing element of the row located between the two processing elements that uses a different row-oriented bus. Use of row-oriented busses can enable parallelization to increase speed or enable increased latency at individual processing elements.