Multi-Dimensional Block-Scaled Acceleration Unit for Lower Memory Use
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing processing systems require multiple partitioned versions of matrices for matrix multiplication operations, leading to increased memory footprint and processing inefficiencies due to the need to accommodate matrices on either side of the operand or after transposition.
Innovation Solution
The use of block-scaled dot-product circuitry that partitions matrices into multi-dimensional scaled blocks, allowing a single partitioned version to be used on either side of the operand and after transposition, reducing the need for multiple versions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If matrices are partitioned into one-dimensional row-wise scaled blocks or column-wise scaled blocks before multiplication, then matrix multiplication operations can be performed efficiently, but multiple partitioned versions of matrices are required when matrices need to be used on either side of the operand or after transposition, increasing memory footprint and processing resources
Solution Approach 1:
The patent applies universality by creating a single partitioned matrix version that can serve multiple functions: it can be used on the left side of the operand, on the right side of the operand, and after transposition operations. This multi-functional partitioned matrix structure eliminates the need to generate separate partitioned versions for different operational contexts, thereby reducing memory footprint while maintaining matrix multiplication efficiency
Solution Approach 2:
The patent merges the functionality of multiple partitioned matrix versions into a single unified partitioned matrix structure. By combining the capabilities of row-wise and column-wise partitioned versions into one versatile structure, the system reduces the quantity of stored data while preserving the ability to perform various matrix operations efficiently
2Adaptability or versatility
If two partitioned versions of matrices are generated to accommodate matrices on either side of the operand or after transposition, then versatility in matrix operations is improved, but device complexity and processing resources increase
Solution Approach 1:
The unified partitioned matrix structure provides universality by being adaptable to multiple operational scenarios including left-side operations, right-side operations, and transposition. This single versatile structure replaces the need for multiple specialized partitioned versions, maintaining operational flexibility while reducing device complexity and processing resource requirements
3Productivity
If one-dimensional row-wise scaled blocks or column-wise scaled blocks are used for matrix partitioning, then processing efficiency is improved, but the same partitioned matrix cannot be reused on different sides of the operand or after transposition
Solution Approach 1:
The patent creates a universally applicable partitioned matrix structure that maintains processing efficiency while enabling reuse across different operational contexts. This unified structure can be efficiently processed while serving multiple purposes including left-side operations, right-side operations, and transposition, thereby simultaneously achieving high productivity and adaptability
Data Source
AI summary
To perform matrix multiplication operations for one or more applications, a processing system includes an acceleration unit (AU) having a block-scaled dot-product circuitry configured to multiply a first matrix by a second matrix. To this end, the block-scaled dot-product circuitry first partitions the first matrix into one or more multi-dimensional scaled blocks and the second matrix also into one or more multi-dimensional scaled blocks. The block-scaled dot-product circuitry next determines dot products of respective portions of the first matrix and corresponding portions of the second matrix using the multi-dimensional scaled blocks of the matrices and then combines these dot products to determine the dot product of the first matrix and the second matrix.