Systolic Array Processing Elements for Matrix Multiplication Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current systolic arrays for matrix multiplication introduce significant latency due to processing only a single pair of operands per cycle, which hampers the efficiency of high-speed parallel computations required for complex tasks like image classification and automatic speech recognition.
Innovation Solution
The design of a systolic array with processing elements capable of receiving, multiplying, and accumulating multiple pairs of operands in each cycle, reducing latency by half while maintaining efficient data transfer and accumulation through synchronization logic and multiple accumulators.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If each processing element processes only a single pair of operands per cycle, then the data transfer and accumulation logic remains simple, but the processing latency increases significantly
Solution Approach 1:
The patent divides the processing element into multiple processing blocks (e.g., four processing blocks per processing element), where each block can independently process a pair of operands. This segmentation allows parallel processing of multiple operand pairs within the same processing element, reducing overall latency while keeping individual block complexity manageable.
Solution Approach 2:
The patent combines multiple processing blocks within a single processing element, sharing common resources such as accumulators and control logic. This merging approach enables simultaneous processing of multiple operand pairs while avoiding the need for completely separate processing elements, thus reducing latency without proportionally increasing device complexity.
2Productivity
If processing elements handle multiple pairs of operands simultaneously, then throughput increases and latency reduces, but the complexity of data transfer and synchronization logic increases
Solution Approach 1:
The patent designs processing elements with universal interfaces and control logic that can handle multiple operand pairs through the same data transfer pathways. The synchronization logic is designed to work with any number of processing blocks within a processing element, making it multi-functional and reducing the need for separate specialized logic for each operand pair.
Solution Approach 2:
The patent organizes processing blocks within processing elements in a structured array format, adding a spatial dimension to the processing architecture. This dimensional organization allows systematic data transfer and synchronization across multiple blocks, making the complexity manageable through regular patterns rather than arbitrary connections.
3Productivity
If processing elements accumulate products from multiple operand pairs, then the accumulation of outer products becomes more efficient, but the accumulator management complexity increases
Solution Approach 1:
The patent implements a hierarchical accumulator structure where first accumulators within each processing block accumulate products locally, and second accumulators at the processing element level accumulate results from multiple processing blocks. This nested arrangement efficiently manages accumulation from multiple operand pairs while organizing complexity in a hierarchical manner.
Solution Approach 2:
The patent introduces second accumulators as intermediary structures that collect results from multiple first accumulators before final output. This intermediary layer simplifies the management of multiple accumulation streams by providing a centralized collection point, reducing the complexity of directly managing all accumulator operations.
Data Source
AI summary
Computational apparatus includes a systolic array of processing elements. In each of a sequence of processing cycles, the processing elements in a first row of the array each receive a respective first plurality of first operands, while the processing elements in a first column of the array each receive a respective second plurality of second operands. Each processing element, except in the first row and first column, receives the respective first and second pluralities of the operands from adjacent processing elements in a preceding row and column of the array. Each processing element multiplies pairs of the first and second operands together to generate multiple respective products, and accumulates the products in accumulators. Synchronization logic loads a succession of first and second vectors of the operands into the array, and upon completion of processing triggers the processing elements to transfer respective data values from the accumulators out of the array.