Neural Network Matrix Multiplication via Convolution Mapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network accelerators (NNAs) often lack native support for efficient matrix multiplications, leading to inefficiencies and additional overheads when using general-purpose processors, which can slow down computation and occupy unnecessary resources.
Innovation Solution
Implement matrix multiplications by mapping them to a graph of neural network operations, including transformations and convolutions, utilizing the fixed-function hardware of NNAs to perform these operations efficiently, avoiding the need for additional dedicated hardware or external processors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If general-purpose processors are used to perform matrix multiplications in neural network accelerators, then computational flexibility is improved, but computation speed deteriorates and resource utilization worsens
Solution Approach 1:
The patent implements matrix multiplication functionality within the existing convolutional processing framework of the neural network accelerator. By treating matrix multiplication as a specialized case of convolution operations, the system achieves multi-functionality without requiring separate dedicated hardware, thus maintaining computational flexibility while improving computation speed through efficient hardware utilization
2Ease of manufacture
If general-purpose processors are used for matrix multiplications, then implementation simplicity is improved, but resource utilization deteriorates due to unnecessary overhead
Solution Approach 1:
The system reuses the existing convolution processing pipeline for matrix multiplication operations, eliminating the need for separate dedicated hardware modules. This approach maintains implementation simplicity while significantly improving resource utilization by avoiding unnecessary overhead from general-purpose processors and specialized hardware additions
Solution Approach 2:
The neural network accelerator performs matrix multiplication using its own existing convolution processing capabilities, without requiring external general-purpose processors or additional specialized hardware. This self-service approach reduces resource utilization overhead while maintaining ease of implementation within the existing system architecture
3Device complexity
If external general-purpose processors are used for matrix multiplications, then hardware design simplicity is improved, but memory-access overhead increases and computational efficiency deteriorates
Solution Approach 1:
The patent merges matrix multiplication operations with the existing convolution processing pipeline within the neural network accelerator. By combining these operations into a single integrated processing flow, the system eliminates the need for separate memory access paths and data transfers to external processors, thereby reducing memory-access overhead and energy loss while maintaining hardware design simplicity
Data Source
AI summary
A method and data processing system using a processor implement a neural network containing at least one matrix multiplication operation. A set of operations to which the matrix multiplication operations are mapped is evaluated, the set of operations including at least one convolution operation, to thereby evaluate the matrix multiplication operation, whereby the processor is adapted to multiply the same set of weights simultaneously by multiple sets of input data elements in parallel at multiple processing elements.