Configurable NPU Dataflow for Low-Latency Transpose Convolution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural processing units (NPUs) face inefficiencies in performing transpose convolution operations due to increased latency and unnecessary multiply-and-accumulate (MAC) operations when zeros are inserted in feature maps, leading to increased computational load.
Innovation Solution
The NPU is designed to divide transpose convolution operations into multiple sub-kernel convolutions, eliminating the need to calculate zeros and reducing MAC operations by performing separate convolutions with each sub-kernel.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If transpose convolution operation is performed by inserting zeros in feature map and using general convolution operation, then the operation can be implemented using existing convolution hardware, but latency increases and unnecessary MAC operations are performed
Solution Approach 1:
The kernel is divided into multiple sub-kernels, and the transpose convolution operation is segmented into multiple separate convolution operations. Each sub-kernel operates on a specific region of the feature map, avoiding unnecessary MAC operations with zero elements. This segmentation reduces the total computational load and latency while maintaining compatibility with standard convolution hardware through multiple targeted operations.
2Adaptability or versatility
If transpose convolution operation is performed by inserting zeros in feature map and using general convolution operation, then the operation can be implemented using existing convolution hardware, but computational load increases due to unnecessary MAC operations
Solution Approach 1:
The invention extracts and identifies zero elements in the feature map before performing convolution operations. By detecting the positions of zero elements, the system excludes these regions from MAC operations, thereby removing unnecessary computational steps. This extraction approach maintains hardware compatibility while significantly improving computational efficiency by focusing resources only on non-zero elements that contribute to the output.
3Device complexity
If channels decrease in image enhancement neural network, then the network can reduce computational complexity in subsequent layers, but the width and height of feature map must increase which requires transpose convolution operation
Solution Approach 1:
The kernel is divided into multiple sub-kernels, and the transpose convolution operation is segmented into multiple separate convolution operations. Each sub-kernel operates on a specific region of the feature map, avoiding unnecessary MAC operations with zero elements. This segmentation reduces the total computational load and latency while maintaining compatibility with standard convolution hardware through multiple targeted operations.
Data Source
AI summary
A method for operating a configurable neural processing unit is disclosed. The method includes dynamically selecting, by a controller, one of multiple data flow modes for a transpose convolution operation within a systolic array of processing elements. The selection is based on an efficiency calculation for a given data set. The method further includes configuring the systolic array to operate in the selected data flow mode, wherein each data flow mode specifies a pattern of data reuse and transfer among the processing elements, and wherein the pattern defines stationary reuse of a particular data type.