Broadcast-Data TPU Architecture for CNN Latency Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing tensor processing unit (TPU) architectures face challenges in reducing processing latency and improving timing margins due to cycle-to-cycle data exchange and skewed weighting operand storage, which hinder efficient execution of convolutional neural network (CNN) operations.
Innovation Solution
The implementation of a broadcast-data TPU architecture, where a shared input data operand is broadcast to all MAC processors, reducing latency and eliminating the need for cycle-to-cycle data exchange, and allowing for non-skewed weighting operand storage, enabling more flexible tensor mapping and faster CNN operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If cycle-to-cycle data exchange is implemented in conventional TPU architectures, then data can be exchanged between MAC processors, but processing latency increases and timing margins deteriorate
Solution Approach 1:
The patent applies preliminary action by pre-loading all input data operands into a shared input data operand register before the MAC computation begins. This allows the data to be ready for immediate use by all MAC processors simultaneously, eliminating the need for cycle-to-cycle data exchange during computation and thereby reducing processing latency while maintaining data exchange capability.
2Quantity of substance
If skewed weighting operand storage is used in conventional TPUs, then weighting operands can be stored, but device complexity increases and timing margins worsen
Solution Approach 1:
The patent applies homogeneity by using a uniform, non-skewed storage architecture where all weighting operands are stored in a regular array structure without complex reordering. Each MAC processor accesses its weighting operands from this uniform storage in a straightforward manner, eliminating the need for skewed storage patterns and reducing device complexity while maintaining full weighting operand storage capability.
3Ease of operation
If conventional TPU architectures are used, then standard MAC operations can be performed, but the number of MAC cycles required increases
Solution Approach 1:
The patent applies segmentation by dividing the CNN computation into independent MAC operations that can be executed in parallel. By organizing the computation so that each MAC processor works on independent data segments simultaneously with a shared input data operand, the overall computation is completed in fewer MAC cycles, increasing productivity while maintaining ease of operation through modular processing.
Data Source
AI summary
An integrated circuit device includes a broadcast data path, a weighting-value memory, Winograd conversion circuitry and multiply-accumulate units. The Winograd conversion circuitry executes a first Winograd conversion function with respect to an input data set to render a converted input data set onto the broadcast data path and executes a second Winograd conversion function with respect to a filter-weight data set to store a converted weighting data set within the weighting-value memory. The multiply-accumulate units, coupled in common to the broadcast data path to receive the converted input data set and coupled to receive respective converted weighting data values from the weighting-value memory, execute a parallel sequence of multiply-accumulate operations to generate an interim output data set that is, in turn, converted to a final output data set through execution of a third Winograd conversion function within the Winograd conversion circuitry.