1.5D MMA Calculator Layout for Lower RF Access and Power
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing matrix multiply-accumulate (MMA) calculators face inefficiencies in power consumption and RF access when dealing with low-precision floating-point numbers, particularly in AI applications, due to high power usage and increased RF access for non-reusable matrix elements.
Innovation Solution
A 1.5-dimensional MMA calculator is introduced, which performs outer product calculations in a pipelined manner, reducing RF access by dividing matrix elements into groups and executing calculations in multiple steps, thereby optimizing power efficiency and reducing RF access.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If traditional MMA calculator architectures are used, then calculation functionality is provided, but power consumption increases and RF access efficiency decreases
Solution Approach 1:
The patent introduces a 1.5-dimensional MMA calculator architecture that combines elements of both 1D and 2D configurations. This intermediate dimensionality allows for optimized data flow patterns that reduce RF access frequency while maintaining calculation throughput, specifically by arranging accumulators in a configuration that enables more efficient reuse of matrix elements from the register file.
Solution Approach 2:
The calculator array is segmented into multiple independent MMA units, each with its own set of accumulators and control logic. This segmentation allows parallel processing of different matrix elements while reducing the contention for shared RF resources, thereby lowering overall power consumption and improving access efficiency.
2Measurement precision
If matrix elements are accessed frequently from RF, then calculation accuracy is maintained, but power consumption increases
Solution Approach 1:
Matrix elements are pre-loaded into the register file in optimized patterns that anticipate future access needs. The 1.5-dimensional architecture enables preliminary organization of data such that elements requiring multiple uses are positioned for efficient access, reducing the need for repeated high-power RF accesses while maintaining full calculation accuracy.
Solution Approach 2:
The architecture implements selective retention of matrix elements in the register file based on their reuse potential. Elements that will be used multiple times are kept in the faster register file, while elements used only once are accessed directly from RF when needed, optimizing the trade-off between access speed and power consumption without compromising accuracy.
Data Source
AI summary
A calculator comprising an array of l×m multiply-accumulate calculators configured to perform, when L and M are both integers of 2 or more and N is an integer of 1 or more, an L×M×N matrix product calculation C=A*B or an L×M×N matrix multiply-accumulate calculation C=A*B+Cin by performing accumulations of outer products Ok of k-th column of A and k-th row of B in an array of L×M accumulators for an integer k of 0 or more and less than N, with respect to an L×N matrix A, an N×M matrix B, an L×M matrix C, and an L×M matrix Cin, wherein any one of l or m is l=L or m=M and the other one is an integer of 2≤l<L or 2≤m<M, and the l×m multiply-accumulate calculators perform each of the accumulations of the outer product Ok by a plurality of steps.