Systolic Array Row Input Data Buses for Matrix Multiplication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Systolic arrays used in neural network processors often experience inefficiencies due to underutilization when performing matrix multiplications with smaller matrix sizes, as they require single input buses for feature map and weight values, leading to idle processing elements and prolonged weight loading times, which can limit the throughput of neural network computations.
Innovation Solution
Increasing the number of row input data buses to enable parallel loading of feature map and weight values, and implementing general-purpose input data buses that can switch between transferring different types of data based on matrix size, allowing for concurrent matrix multiplications and improved loading times.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If single input buses are used for feature map and weight values, then device complexity is reduced, but processing elements become idle and utilization decreases
Solution Approach 1:
The patent implements general-purpose input data buses that can dynamically switch between transferring feature map data and weight values based on the current operation requirements. This multi-functionality allows the same physical bus infrastructure to support different data transfer needs, enabling full utilization of processing elements without requiring separate dedicated buses for each data type.
2Device complexity
If single input buses are used for feature map and weight values, then device complexity is reduced, but weight loading time increases
Solution Approach 1:
The patent enables background weight loading to occur concurrently with feature map processing by utilizing the same input buses for different purposes at different times. Weights can be pre-loaded into processing elements during periods when they are not actively computing, thereby reducing the overall weight loading time without requiring additional dedicated loading pathways.
3Productivity
If general-purpose input data buses are implemented, then systolic array utilization improves, but device complexity increases
Solution Approach 1:
The patent employs dynamic switching of input data bus functionality based on operational context. The same physical buses can be configured to transfer feature maps during computation phases and load weights during idle or background phases. This dynamic reconfiguration is controlled through timing signals and control logic that switch the bus mode appropriately, maximizing utilization while managing complexity through time-multiplexed resource sharing.
Data Source
AI summary
To improve utilization of a systolic array, each row of the array is provided with a number of general purpose row input data buses. Each of the general purpose row input data buses can be operable to transfer either feature map (FMAP) input elements or weight values into the processing elements of the corresponding row of the array. By using such general purpose row input data buses, concurrent matrix multiplications as well as faster background weight loading can be achieved in the array.