Matrix Multiplication Engine Using Vector Outer-Product Summation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing computing devices face inefficiencies in performing matrix-matrix multiplication due to the repeated loading of elements from memory, which leads to high memory access latency and data transmission overhead.
Innovation Solution
The proposed solution involves a computing device with an array of processing elements that compute matrix dot products as a summation of vector-vector outer-products, reducing data transmission by keeping partial products stationary within processing elements and allowing full output reuse.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional vector-vector multiplication is used with sequential loading from memory, then computation can be performed, but memory access latency and data transmission overhead increase significantly
Solution Approach 1:
The matrix multiplication operation is segmented into multiple micro-batches, where each micro-batch processes a portion of the outer-product computations. This segmentation allows the system to interleave computation and data loading, keeping processing elements busy while reducing peak memory bandwidth requirements and improving overall throughput despite memory latency
Solution Approach 2:
Data is pre-loaded into on-chip buffers before processing begins. The system performs preliminary data movement operations to load matrix elements into faster on-chip memory, reducing the impact of main memory access latency during the actual computation phase
2Reliability
If repeated loading of matrix elements from memory is performed, then complete matrix multiplication can be achieved, but data transmission overhead increases
Solution Approach 1:
The patent transforms the computation from traditional row-by-row or column-by-column multiplication into a sequence of vector-vector outer-product computations. This dimensional change in the computation approach allows for better data reuse patterns, where intermediate results are accumulated in on-chip buffers, reducing the need for repeated data transmission between main memory and processing elements
3Productivity
If conventional processing elements are used with full data movement, then matrix dot product can be computed, but inter-processing element data movement increases
Solution Approach 1:
The patent extracts and separates the data movement operations from the computation operations. By using dedicated buffer memory and separating data loading from processing, the system reduces inter-processing element data movement while maintaining computational throughput. The buffer acts as an intermediary that decouples data movement from computation
Data Source
AI summary
A matrix multiplication engine can include a plurality of processing elements configured to compute a matrix dot product as a summation of a sequence of vector-vector outer-products.