Systolic Array Conveyor Queue Dynamic Skip
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional systolic array-based neural network processing units are unable to dynamically skip unnecessary calculations, leading to inefficiencies in sparse matrix multiplication and requiring larger chip areas and higher power consumption.
Innovation Solution
The proposed method and apparatus utilize a conveyor queue and synchronous processing units with a search window design, allowing for dynamic skipping of zero-value calculations and synchronization of data transfer between processing elements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional systolic array architecture is used, then data flow is maintained through two-dimensional pipeline, but unnecessary zero value calculations cannot be dynamically skipped
Solution Approach 1:
The patent introduces dynamic skipping capability in systolic array processing elements, allowing them to adaptively bypass unnecessary zero-value calculations based on sparsity patterns. This dynamic behavior enables the system to adjust its operation mode between dense and sparse matrix multiplication, improving calculation efficiency without requiring structural changes to the overall architecture.
Solution Approach 2:
The patent applies different processing strategies to different regions of the systolic array based on local sparsity characteristics. Processing elements detect zero values in incoming data and locally adapt their computation behavior, performing multiplication only when necessary while skipping operations on zero values. This local quality approach allows selective optimization without affecting the global data flow pattern.
2Speed
If FIFO buffers are added between processing elements to allow data transmission regardless of calculation completion, then speed differences between PEs are alleviated, but chip area and power overhead increase significantly
Solution Approach 1:
The patent extracts and removes the FIFO buffer component from the systolic array architecture by implementing direct data transmission pathways between processing elements. The data flow is synchronized through control signals and timing mechanisms rather than through buffer storage, eliminating the need for additional buffer memory and reducing chip area overhead while maintaining data transmission flexibility.
Solution Approach 2:
The patent introduces control signals and synchronization mechanisms as intermediaries between processing elements to coordinate data transmission without requiring physical buffer storage. These control intermediaries manage the timing and flow of data through the array, allowing faster processing elements to transmit data ahead while slower elements receive data when ready, without the need for FIFO buffers.
3Productivity
If multiple operand data chunks are received by each PE for dynamic operand pair generation, then unnecessary calculations can be skipped, but pipeline stalls occur due to speed differences between PEs
Solution Approach 1:
The patent ensures continuous useful action in the systolic array by implementing mechanisms that prevent pipeline stalls. Processing elements continuously receive and process multiple operand data chunks, dynamically generating operand pairs and skipping zero-value calculations without interrupting the data flow. The architecture maintains continuous operation by coordinating data transmission and processing to eliminate idle time between operations.
Data Source
AI summary
There is provided an apparatus for calculations based on a systolic array, comprising: a memory in which one or more operand data chunks are stored, a conveyor queue configured to shift the one or more operand data chunks in sequence and one or more synchronous processing units (SPUs) including one or more processing elements (PEs), and configured to access the operand data chunks shifted along the conveyor queue, wherein the one or more processing elements perform calculations based on the operand data chunks that the synchronous processing unit including each processing element has accessed.