Affine Transformation Accelerator Using Outer Products for Lower Bandwidth
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional matrix multiplication processes in machine learning and scientific computing become bandwidth-bound due to heavy data movement, especially in affine transformations used by neural networks, leading to inefficient memory usage.
Innovation Solution
An integrated circuit (IC) that performs matrix multiplication using outer products, storing block matrices in dedicated memory and computing partial matrices, which are then summed to generate the final result, reducing memory bandwidth requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional inner product matrix multiplication is used, then computational accuracy is maintained, but memory bandwidth consumption increases making the application bandwidth-bound
Solution Approach 1:
The patent inverts the traditional inner product computation approach by using outer products instead. Instead of computing the dot product of rows and columns to get scalar values, the system computes outer products of column vectors to generate matrices, then sums these matrices to obtain the final result. This inversion fundamentally changes the data access pattern from element-by-element multiplication to matrix-level operations, reducing bandwidth requirements.
Solution Approach 2:
The patent segments the matrix multiplication process into discrete outer product computations. Each outer product operation processes a specific column vector from the first matrix with a specific row vector from the second matrix, producing a partial result matrix. These partial results are then accumulated to form the final matrix product. This segmentation allows for optimized memory access patterns and reduced bandwidth consumption compared to traditional approaches.
2Loss of energy
If outer product-based matrix multiplication is used, then memory bandwidth requirements are reduced, but computational complexity increases
Solution Approach 1:
The patent employs copying operations to replicate column vectors and row vectors during the outer product computation. By copying and reusing these vector components across multiple operations, the system reduces the need for repeated memory accesses. The column vectors are copied and reused for multiple row vector multiplications, and similarly for row vectors with column vectors, thereby reducing overall bandwidth consumption despite the increased computational steps.
Solution Approach 2:
The patent performs preliminary actions by pre-computing and storing outer product results in intermediate memory structures. Before the final matrix summation, the system prepares partial result matrices by computing outer products and storing them in temporary memory. This preliminary computation and storage allows for more efficient final accumulation operations and reduces the bandwidth required for the complete matrix multiplication process.
Data Source
AI summary
Examples herein describe a hardware accelerator for affine transformations (matrix multiplications followed by additions) using an outer products process. In general, the hardware accelerator reduces memory bandwidth by computing matrix multiplications as a sum of outer products. Moreover, the sum of outer products benefits parallel hardware that accelerates matrix multiplication, and is compatible with both scalar and block affine transformations, and more generally, both scalar and block matrix multiplications.