Multiple Accumulate Busses Systolic Array Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Systolic arrays face challenges in reducing latency while maintaining low power consumption, which is crucial for efficient neural network inference operations, as increased clock speed leads to higher power consumption and cooling requirements.
Innovation Solution
Implementing multiple busses within the columns and rows of a systolic array allows for sequence skipping and parallelism, reducing the overall latency and enabling the use of lower-cost, lower-power processing elements by decreasing the path length of calculations and increasing parallelism.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If clock speed is increased to reduce latency, then inference performance is improved, but power consumption increases
Solution Approach 1:
The patent divides the data flow path into multiple parallel busses (e.g., multiple accumulate busses) within the systolic array. This segmentation allows data to traverse shorter parallel paths simultaneously, reducing the effective latency without requiring higher clock speeds, thereby avoiding increased power consumption.
Solution Approach 2:
The patent introduces multiple busses as an additional dimensional resource for data transmission. Instead of increasing speed along a single data path, the system creates multiple parallel dimensions for data flow, effectively reducing latency through parallelism rather than speed increase.
2Loss of time
If multiple busses are implemented to reduce latency, then parallelism increases, but device complexity increases
Solution Approach 1:
The patent designs processing elements that can operate with multiple busses in a uniform manner. Each processing element is configured to handle data from multiple input busses and route to multiple output busses using the same basic operations, making the complex multi-bus structure manageable through standardized multi-functional processing units.
3Loss of time
If multiple busses are used to decrease path length, then latency is reduced, but manufacturing complexity increases
Solution Approach 1:
The patent employs identical or highly similar processing elements throughout the systolic array, each configured to work with the multi-bus architecture in the same way. This homogeneity simplifies manufacturing by allowing repeated use of the same design template, reducing fabrication complexity despite the increased number of busses.
Data Source
AI summary
Systems and methods are provided to enable parallelized multiply-accumulate operations in a systolic array. Each column of the systolic array can include multiple busses enabling independent transmission of input partial sums along the respective bus. Each processing element of a given columnar bus can receive an input partial sum from a prior element of the given columnar bus, and perform arithmetic operations on the input partial sum. Each processing element can generate an output partial sum based on the arithmetic operations, provide the output partial sum to a next processing element of the given columnar bus, without the output partial sum being processed by a processing element of the column located between the two processing elements that uses a different columnar bus. Use of columnar busses can enable parallelization to increase speed or enable increased latency at individual processing elements.