Convolution Engine Matrix Multiplication With Transposed Inputs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing processors, such as GPUs, are not designed to efficiently perform matrix multiplication operations required for neural networks, particularly convolutional neural networks, due to the additional hardware and software complexity needed for processing convolutions.
Innovation Solution
A matrix processor, such as a convolution engine, is configured to perform matrix multiplication by adjusting the format of matrices to align with convolution parameters, including transposing and padding, allowing efficient computation of convolutions and matrix multiplications using a grid of processing elements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If a GPU is used for neural network processing, then computation speed is improved, but device complexity increases due to additional hardware needed for convolution operations
Solution Approach 1:
The patent makes the convolution engine universal by enabling it to perform both convolution operations and matrix multiplication operations. The same processing elements, memory structures, and control logic are used for both types of operations, eliminating the need for separate dedicated hardware for matrix multiplication while maintaining high computation speed for neural network processing
2Device complexity
If a convolution engine is used for matrix multiplication, then device complexity is reduced, but computation speed may worsen without proper optimization
Solution Approach 1:
The patent changes the parameters and formatting of matrix data to align with convolution engine expectations. Matrices are transposed and padded to match convolution kernel dimensions, allowing the convolution engine to perform matrix multiplication at its full computational speed without requiring additional hardware optimization
3Ease of operation
If matrices are directly input to convolution engine, then ease of operation is improved, but manufacturing precision worsens due to format mismatches
Solution Approach 1:
The patent applies preliminary actions to the matrix data before inputting it to the convolution engine. The second matrix is transposed and padded in advance to match the expected convolution kernel format, ensuring precise data alignment and eliminating format mismatches before the computation begins
Data Source
AI summary
An example matrix processor includes processing elements arranged as a grid, with the matrix processor being configured to receive a first matrix and a second matrix, wherein the first matrix is to be multiplied by the second matrix; transpose the second matrix; organize the second matrix into a plurality of columns, wherein each column is a row of the second matrix; and over one or more cycles, sequentially provide the columns of the second matrix and the rows of the first matrix to the processing elements, wherein the processing elements are configured as multiply-accumulate units (MAC units), and wherein a processing result is stored in the processing elements.