DNN Accelerator Gradient Computation via Group Convolution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for training deep neural networks (DNNs) via back-propagation are inefficient when implemented in hardware, particularly in DNN accelerators, as they rely on general-purpose computers for determining gradients of the loss metric with respect to convolution layer weights through matrix multiplication, which is not efficiently scalable.
Innovation Solution
Implementing group convolutions in hardware, specifically in DNN accelerators with convolution engines, to efficiently calculate the gradients of the loss metric with respect to the weights of convolution layers by performing group convolutions between input data and output gradient tensors, allowing for efficient hardware-based computation of weight gradients.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If general-purpose computers are used for matrix multiplication to determine gradients, then flexibility and programmability are maintained, but computation efficiency and scalability deteriorate
Solution Approach 1:
The patent replaces general-purpose computer matrix multiplication with dedicated hardware convolution engines that perform group convolution operations. This substitution transforms software-based gradient computation into hardware-accelerated computation, achieving both improved efficiency and scalability while maintaining the ability to handle different DNN architectures through configurable hardware parameters.
2Measurement precision
If matrix multiplication is used for gradient computation, then mathematical accuracy is maintained, but hardware scalability and performance deteriorate
Solution Approach 1:
The patent transforms the gradient computation problem from matrix multiplication parameters to group convolution parameters. By changing the mathematical operation parameters to match hardware convolution engine capabilities, the system achieves accurate gradient computation through hardware-optimized operations rather than general-purpose matrix multiplication.
3Productivity
If dedicated convolution engines are used for gradient computation, then computation speed and hardware efficiency improve, but system complexity increases
Solution Approach 1:
The patent designs convolution engines that serve dual purposes: performing forward propagation convolutions during DNN execution and computing gradients during training. This multi-functionality allows the same hardware infrastructure to handle both inference and training workloads, improving resource utilization without proportionally increasing system complexity.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Methods and systems of generating gradients of a loss metric for a neural network (NN) with respect to weights of a convolution layer of the NN, the convolution layer of the NN configured to receive an input tensor of input values and a weight tensor of weights, and generate an output tensor of output values. The methods comprise performing, using hardware logic, a group convolution between a first y-dimensional tensor and a second z-dimensional tensor wherein z = y + 1, the first y-dimensional tensor being formed of a set of values from the input tensor, and the second z-dimensional tensor being formed of a set of values from an output gradient tensor comprising gradients of the loss metric for the NN with respect to the output values; wherein the first y-dimensional tensor, the second z-dimensional tensor and the group convolutions are configured to generate an output of a convolution operation between each channel of the set of values of the input tensor and each channel of the set of values of the output gradient tensor.