Adapting Winograd Algorithm for Neural Network Accelerators
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network accelerators face inefficiencies in implementing the Winograd algorithm due to their optimization for standard convolution operations rather than the small matrix multiplications required by the algorithm, leading to impractical performance with millions of small matrix multiplications.
Innovation Solution
The method adapts the Winograd algorithm by using hardware with linear operation engines to perform convolution operations efficiently, mapping sandwich matrix multiplications and element-wise operations into equivalent standard convolution and convolution transpose operations, allowing for efficient implementation on neural network accelerators without dedicated Winograd convolution support.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If standard convolution operations are used on neural network accelerators, then hardware efficiency is improved, but the ability to implement Winograd algorithm with small matrix multiplications deteriorates
Solution Approach 1:
The patent introduces an intermediary transformation layer that converts Winograd algorithm operations into standard convolution operations. By using constant matrices A, B, and G to transform the input tensor and weight tensor before standard convolution, the system enables Winograd algorithm implementation on hardware optimized for standard convolutions, resolving the contradiction between hardware efficiency and algorithmic versatility
Solution Approach 2:
The patent changes the parameters of the convolution operation by introducing transformation matrices A, B, and G that modify the input tensor and weight tensor. These parameter transformations convert the small matrix multiplications required by Winograd algorithm into equivalent standard convolution operations with different kernel sizes and padding, allowing the same hardware to execute both standard convolutions and Winograd algorithm efficiently
2Device complexity
If Winograd algorithm is implemented with small matrix multiplications, then computational complexity is reduced, but hardware utilization efficiency deteriorates
Solution Approach 1:
The patent substitutes the mechanical system of performing millions of small matrix multiplications with a different mechanical system of standard convolution operations. By replacing the Winograd algorithm's small matrix multiplication mechanism with standard convolution using transformed tensors, the system maintains reduced computational complexity while improving hardware utilization efficiency through better parallelization and memory access patterns
3Device complexity
If standard convolution hardware is used, then hardware simplicity is maintained, but Winograd algorithm performance becomes impractical
Solution Approach 1:
The patent segments the Winograd algorithm implementation into three distinct stages: (1) transforming the input tensor using matrix A and B, (2) performing standard convolution with transformed weights, and (3) transforming the output using matrix A. This segmentation allows the hardware to remain simple for standard convolutions while achieving Winograd algorithm performance through the mathematical transformation of the data and weights before and after the convolution operation
Data Source
AI summary
Systems and methods of performing convolution efficiently adapting the Winograd algorithm are provided. Methods of convolving an input tensor with weights w use hardware comprising a plurality of linear operation engines as part of performing adaptations of a Winograd algorithm, the Winograd algorithm splitting each input channel i of a total of Cin input channels into one or more tiles di and calculating a result A[Σi=1Cin(GwjiGT)∘(BTdiB)]AT for each output channel j, wherein G, B and A are constant matrices. The methods comprise determining a first filter F1 from matrix B wherein the filter F1 comprises n kernels, each kernel being an outer product of two columns of the matrix B; and using the linear operation engines to perform a convolution of the input tensor with the first filter F1.