Configurable Weight Registers for Faster Systolic Array Flushing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The flushing latency in systolic arrays for neural network processing adds significant latency and degrades performance by propagating old weights to unused columns, affecting throughput.
Innovation Solution
Implement a configurable pipeline register that selects between propagating weight values or flush values based on control signals, reducing flushing latency by not propagating weights to non-computation-involved columns.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If weights are propagated through all columns in a systolic array, then weight loading is simplified, but flushing latency increases and throughput degrades
Solution Approach 1:
The pipeline register is made configurable to dynamically switch between two modes: propagating weight values during weight loading phase, and propagating flush values during flushing phase. This dynamic reconfiguration allows the system to adapt its behavior based on the operational phase, eliminating the need to propagate weights through unused columns and thereby reducing flushing latency while maintaining simplified weight loading mechanisms.
Solution Approach 2:
The system changes the parameter being propagated through the pipeline register based on the operational phase. During weight loading, the register propagates weight values; during flushing, it propagates flush values (e.g., zero values). This parameter change allows the flushing operation to clear weights from the array without unnecessarily propagating them through all columns, thus reducing flushing latency and improving throughput.
2Loss of time
If a configurable pipeline register is implemented to select between weight propagation and flush value propagation, then flushing latency is reduced, but device complexity increases
Solution Approach 1:
The pipeline register is designed to serve multiple functions: it acts as a weight propagation register during weight loading and as a flush value propagation register during flushing operations. By making the register configurable, a single hardware component performs multiple roles, avoiding the need for separate dedicated registers for each function. This multi-functionality reduces overall device complexity while achieving the goal of reduced flushing latency.
Data Source
AI summary
Disclosed herein are techniques for obtaining weights for neural network computations. In one embodiment, an integrated circuit may include memory configured to store a first weight and a second weight; a row of processing elements comprising a first processing element and a second processing element, the first processing element comprising a first weight register, the second processing element comprising a second weight register, both of the first weight register and the second weight register being controllable by a weight load signal; and a controller configured to: provide the first weight from the memory to the row of processing elements; set the weight load signal to enable the first weight to propagate through the row to reach the first processing element; and set the weight load signal to store the first weight at the first weight register and a flush value at the second weight register.