In-Memory MAC Unit Layout for Matrix-Vector Output Without Temp Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing memory technologies require temporary storage of intermediate results during matrix-vector multiplication, leading to increased manufacturing costs, larger die size, and delayed output due to additional operations needed to combine these results.
Innovation Solution
Perform matrix-vector multiplication by multiplying data values of columns of a matrix with a vector using multiple accumulate (MAC) units, retaining final results in these units without the need for external temporary storage, allowing direct output through the same data path as reading from memory arrays.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If temporary storage is used for intermediate results during matrix-vector multiplication, then manufacturing cost and die size increase, but calculation accuracy is maintained
Solution Approach 1:
The patent extracts the temporary storage function from external memory and relocates it to internal registers within the processing unit. This allows intermediate results to be stored locally during matrix-vector multiplication, maintaining calculation accuracy while reducing the need for external temporary storage and associated manufacturing complexity
Solution Approach 2:
The patent nests the temporary storage registers within the processing unit structure, creating a hierarchical memory organization where fast internal storage is embedded within the computational unit. This nested approach allows intermediate results to be held during calculation without requiring separate external storage infrastructure
2Manufacturing precision
If temporary storage is used for intermediate results, then die size increases, but calculation accuracy is maintained
Solution Approach 1:
The patent extracts the temporary storage function from external memory and relocates it to internal registers within the processing unit. This allows intermediate results to be stored locally during matrix-vector multiplication, maintaining calculation accuracy while reducing the need for external temporary storage and associated manufacturing complexity
Solution Approach 2:
The patent changes the spatial dimension of temporary storage by moving from external memory space to internal register space within the processing unit. This dimensional shift allows intermediate results to be stored in a compact, integrated manner that minimizes die size impact while preserving computational accuracy
3Reliability
If additional operations are performed to combine intermediate results, then calculation completeness is improved, but output time increases
Solution Approach 1:
The patent performs preliminary accumulation of intermediate results in internal registers during the matrix-vector multiplication process. By preparing the final result incrementally throughout the calculation rather than requiring separate combination operations afterward, the system maintains calculation completeness while minimizing additional output delays
Solution Approach 2:
The patent maintains continuous useful action by accumulating intermediate results in real-time during the multiplication process rather than pausing to combine them separately. The MAC units continuously update the result vector in internal storage, ensuring calculation completeness is achieved without interrupting the computational flow for additional combination operations
4Productivity
If multiple MAC units are used for parallel multiplication, then productivity increases, but device complexity increases
Solution Approach 1:
The patent segments the matrix-vector multiplication task across multiple MAC units, with each unit handling a portion of the calculation in parallel. This segmentation increases productivity by distributing the computational workload while managing complexity through modular, standardized unit designs that can be replicated and coordinated systematically
Solution Approach 2:
The patent employs universal MAC units that can handle different elements of the matrix-vector multiplication through standardized interfaces and control logic. Each MAC unit is designed to perform the same fundamental multiply-accumulate operation on different data, providing multi-functionality that increases productivity while controlling complexity through design uniformity
Data Source
AI summary
The processing unit (PU) PU of a memory device can receive a matrix of data values and a vector of data values stored in a bank. The PU can perform a first plurality of multiplication operations on a first data value of the vector utilizing a first plurality of data values of a first column of the matrix. The first plurality of multiplication operations can be performed by a plurality of multiply-accumulate (MAC) units. Each of the first plurality of multiplication operations can be performed by a different MAC unit of the plurality of MAC units. The PU can perform a second plurality of multiplication operations on a second data value of the vector utilizing a second plurality of data values of a second column of the matrix. Each of the second plurality of multiplication operations can be performed by a different MAC unit of the plurality of MAC units.