Wave Matrix Multiply Accumulate Instruction Reducing Data Movement
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Matrix multiplication operations in machine learning algorithms are inefficient, consuming significant power and time, and repeatedly fetching data from vector register files, leading to increased data storage and decreased throughput.
Innovation Solution
A parallel data processing circuit with a SIMD microarchitecture performs matrix multiplication using source operands accessed only once from a vector register file, utilizing multiple vector ALUs for fused multiply-add and dot product operations, and implementing a Wave Matrix Multiply Accumulate (WMMA) instruction to reduce data movement and power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If matrix multiplication operations are performed using conventional SIMD processors with multiple fetch operations from vector register file, then the operations can be executed in parallel, but power consumption increases and data movement overhead increases
Solution Approach 1:
The patent applies preliminary action by pre-loading matrix elements into the vector register file before the matrix multiplication operation begins. The system fetches and stores multiple elements of matrix A and matrix B into the vector register file in advance, so that during the actual computation phase, data can be reused without repeated fetch operations. This reduces power consumption by minimizing active data movement during computation while maintaining high throughput through parallel processing.
2Productivity
If matrix multiplication operations perform multiple fetch operations from vector register file, then data can be accessed for computation, but data storage requirements increase and throughput decreases
Solution Approach 1:
The system performs preliminary action by fetching and storing multiple matrix elements into the vector register file before computation begins. Specifically, it loads elements such as A[0], A[1], B[0], B[1] into the register file in advance, allowing subsequent multiply-accumulate operations to reuse this data without additional fetches. This approach reduces the total quantity of data that needs to be stored and repeatedly accessed, thereby decreasing storage requirements while maintaining high throughput.
3Adaptability or versatility
If conventional FMA and dot product operations are repeatedly used in machine learning algorithms, then the algorithms can be implemented, but data storage increases and power consumption increases
Solution Approach 1:
The patent applies merging by combining multiple FMA operations into a single unified matrix multiplication instruction. Instead of executing separate FMA operations that each require individual data fetches from the vector register file, the system performs a consolidated matrix multiplication that reuses the same loaded data across multiple accumulate operations. This merging reduces redundant data movement and associated power consumption while maintaining the adaptability to implement various machine learning algorithms that require repeated FMA and dot product operations.
Data Source
AI summary
An apparatus and method for efficiently processing multiplication and accumulate operations for matrices in applications. In various implementations, a computing system includes a parallel data processing circuit and a memory. The memory stores the instructions (or translated commands) of a parallel data application. The circuitry of the parallel data processing circuit performs a matrix multiplication operation using source operands accessed only once from a vector register file and multiple instantiations of a vector processing circuit capable of performing multiple matrix multiplication operations corresponding to multiple different types of instructions. The multiplier circuit and the adder circuit of the vector processing circuit perform each of the fused multiply add (FMA) operation and the dot product (inner product) operation without independent, dedicated execution pipelines with one execution pipeline for the FMA operation and the other separate execution pipeline for the dot product operation.