1xN Kernel Decomposition and Activation Broadcasting for DNN Accelerators
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep Neural Networks (DNNs) face significant computation costs and inefficiencies in energy and area usage, particularly in mobile and embedded systems, due to high demands for MAC operations and weight operand storage, which current technologies fail to effectively address through sparsity methods that limit PE utilization and bandwidth demands.
Innovation Solution
Implementing 1 × N kernel decomposition and activation broadcasting in DNN accelerators, where a H × W kernel is converted into 1 × N kernel vectors, allowing for reduced read operations and increased PE utilization by folding the kernel-X loop into the spatial X loop and broadcasting activations between processing elements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If sparsity methods are used to reduce computation cost, then energy consumption is reduced, but PE utilization is limited
Solution Approach 1:
The kernel is decomposed into multiple 1×N kernel vectors, where N is an integer greater than 1. This segmentation allows the convolution operation to be divided into multiple smaller operations that can be distributed across PEs, increasing utilization while maintaining sparsity benefits
Solution Approach 2:
The kernel decomposition transforms the traditional 2D kernel structure into 1×N vectors, changing the dimensional representation. This allows activations to be broadcast across multiple PEs in the N dimension, improving utilization without sacrificing energy efficiency
2Reliability
If traditional convolution methods are used, then accuracy is maintained, but memory bandwidth demands are high
Solution Approach 1:
Activations are broadcast to multiple PEs in advance before the convolution computation begins. This preliminary action ensures that all necessary data is available at each PE, reducing the need for repeated memory accesses during computation and lowering overall bandwidth demands
Solution Approach 2:
Activations are copied and distributed to multiple PEs through broadcasting. Each PE receives a copy of the activation data it needs, eliminating the need for multiple read operations from the same memory location and reducing bandwidth requirements
3Productivity
If 1×N kernel decomposition is implemented, then PE utilization is increased, but device complexity increases
Solution Approach 1:
The same PE array infrastructure is used to perform both traditional convolution and the new 1×N decomposed convolution. By making the system multi-functional, it can handle different kernel sizes and decomposition factors without requiring completely separate hardware, thus managing complexity while improving utilization
Data Source
Figure 1
Figure 2
Figure 3
AI summary
An DNN accelerator may perform 1 × N kernel decomposition to decompose a convolutional kernel into kernel vectors, each of which includes multiple weights. Through the kernel decomposition, a weight operand may be generated from a filter. The DNN accelerator converts an input tensor into input operands. An input operand includes activations and has the same size as the weight operand. The DNN accelerator may read a first activation in the input operand from memory to an internal memory of a first PE and read a second activation in the input operand from the memory to an internal memory of a second PE. The first PE may receive the second activation from the second PE through activation broadcasting between the two PEs and perform MAC operations on the input operand and weight operand. The second PE may perform MAC operations on another input operand in the input tensor and the weight operand.