Broadcast Data Multiply-Accumulate Shared Unload
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing tensor processing unit (TPU) architectures face challenges in reducing processing latency and improving timing margins due to cycle-to-cycle data exchange and skewed weighting operand storage, which increases latency and complicates filter matrix storage.
Innovation Solution
The proposed broadcast-data TPU architecture allows a shared input data operand to be loaded in parallel into all MAC processors, eliminating the need for cycle-to-cycle data exchange and enabling non-skewed weighting operand storage, thereby reducing processing latency and improving timing margins.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If cycle-to-cycle data exchange is implemented in existing TPU architectures, then data can be exchanged between MAC processors, but processing latency increases by N-1 clock cycles
Solution Approach 1:
The patent extracts the data exchange function from the MAC processor cycle-to-cycle operation and implements it through a separate broadcast data register that loads data in parallel into all MAC processors simultaneously. This eliminates the need for iterative data exchange between processors, reducing latency from N-1 clock cycles to a single parallel load operation.
Solution Approach 2:
The patent transitions from a sequential time-based data exchange approach (cycle-to-cycle) to a parallel spatial approach where a single data value is broadcast simultaneously to all N MAC processors through a broadcast data register, effectively changing the dimension of data distribution from temporal to spatial parallelism.
2Ease of operation
If skewed weighting operand storage is used to enable data exchange, then filtering operations can be performed, but filter matrix storage becomes complicated and timing margins are reduced
Solution Approach 1:
The patent applies local quality by allowing each MAC processor to independently store its weighting operands in a local weighting operand register without requiring global synchronization or skewed storage patterns. Each processor operates with locally available data, eliminating the need for complicated filter matrix storage schemes and improving timing margins.
Solution Approach 2:
The patent implements preliminary action by pre-loading the broadcast data value into a broadcast data register before the MAC processing cycle begins. This allows all MAC processors to simultaneously access the required input data without waiting for cycle-to-cycle exchanges, thereby improving timing margins and simplifying the storage requirements for filter matrices.
Data Source
AI summary
An integrated circuit device includes broadcast data paths, a weighting-value memory, multiply-accumulate (MAC) units, and shared shift-out circuitry. The MAC units are coupled in common to each of the broadcast data paths and coupled to receive respective weighting values from the weighting-value memory via respective weighting-value paths. Each of the MAC units includes MAC circuits that each receive an input data value via a respective one of the broadcast data paths and a shared one of the weighting values via a shared one of the respective weighting-value paths; generate a sequence of multiplication products by multiplying the input data value with the shared one of the weighting values; accumulate a sum of the multiplication products; and output the sum of the multiplication products to a respective one of a plurality of serially coupled storage elements within the shared shift-out path.