Neural Network Matrix Multiplication Using Convolution Hardware
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network accelerators (NNAs) often lack native support for efficient matrix multiplications, leading to inefficiencies and additional overheads when using general-purpose processors, which can slow down computations and waste resources.
Innovation Solution
Map matrix multiplications to a graph of neural network operations, including transformations and convolutions, utilizing fixed-function hardware in NNAs to perform these operations efficiently, leveraging existing specialized hardware without the need for additional modules.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If general-purpose processors are used to perform matrix multiplications on neural network accelerators, then computational flexibility is maintained, but computation speed decreases and resource overhead increases
Solution Approach 1:
The patent makes existing convolutional hardware perform matrix multiplication operations by reconfiguring it to handle matrix multiplication as a specialized case of convolution. This allows the same hardware to serve both convolution and matrix multiplication functions, eliminating the need for separate dedicated matrix multiplication units while maintaining high computational efficiency.
Solution Approach 2:
The patent changes the operational parameters of the convolutional hardware by setting specific convolution parameters (such as kernel size, stride, and padding) to values that transform the convolution operation into an equivalent matrix multiplication operation. This parameter reconfiguration enables the hardware to efficiently execute matrix multiplication without requiring structural modifications.
2Productivity
If dedicated matrix multiplication hardware is added to neural network accelerators, then computation efficiency improves, but device complexity and manufacturing cost increase
Solution Approach 1:
The patent achieves dedicated matrix multiplication functionality by reconfiguring existing convolutional hardware parameters rather than adding separate dedicated hardware. This multi-functional approach allows the same physical hardware to efficiently perform both convolution and matrix multiplication, avoiding increased manufacturing complexity and cost while maintaining high computation efficiency.
Solution Approach 2:
The patent merges the matrix multiplication function into the existing convolutional processing pipeline by treating matrix multiplication as a specialized convolution operation. This consolidation eliminates the need for separate hardware paths and integrates matrix multiplication capabilities into the existing convolutional hardware architecture, simplifying manufacturing while improving efficiency.
3Device complexity
If matrix multiplications are performed using existing convolutional hardware, then additional hardware overhead is avoided, but operational complexity increases
Solution Approach 1:
The patent simplifies operational complexity by establishing clear parameter mapping rules that automatically transform matrix multiplication operations into convolution operations. The system configures convolution parameters (kernel size, stride, padding) based on the matrix dimensions, providing a systematic and automated approach that reduces operational complexity despite the conceptual transformation.
Solution Approach 2:
The patent introduces a mapping layer or interface that mediates between matrix multiplication operations and convolutional hardware execution. This intermediary automatically handles the transformation and parameter configuration, shielding users from operational complexity while enabling efficient hardware utilization.
Data Source
AI summary
A method and data processing system implement a neural network containing at least one matrix multiplication operation. The matrix multiplication operation is mapped to a graph of neural network operations including at least one transformation and at least one convolution. The at least one convolution is implemented in fixed-function hardware of a neural network accelerator.