Matrix Operations Accelerator Transpose Instructions for FMA Tiles
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing processors face challenges in efficiently handling large matrices due to instruction intensity and difficulty in managing rows of matrices, which is not acceptable with larger matrices, especially in deep learning applications where low precision arithmetic is used.
Innovation Solution
The implementation of a matrix operations accelerator circuit that supports 2-dimensional data structures called tiles, which are configured for matrix operations such as multiplication, addition, and transposition, using a matrix operations accelerator circuit with a fused multiply accumulate (FMA) grid and scheduler circuitry to optimize matrix operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional processors are used to handle large matrices, then general-purpose computing is maintained, but instruction intensity increases and processing efficiency deteriorates
Solution Approach 1:
The patent divides large matrices into smaller tile structures that can be processed in parallel by multiple FMA circuits. This segmentation reduces the instruction intensity on individual processing units while maintaining overall productivity through parallel execution of matrix operations across multiple tiles.
Solution Approach 2:
The patent introduces a specialized matrix operations accelerator circuit as an intermediary between the general-purpose processor and memory. This accelerator handles the computationally intensive matrix operations, reducing the instruction burden on the main processor while improving overall matrix processing efficiency through dedicated hardware support.
2Use of energy by moving object
If matrix operations are performed using conventional methods, then flexibility is maintained, but energy consumption increases for large matrix operations
Solution Approach 1:
The patent replaces conventional software-based matrix operations with hardware-accelerated FMA circuits. This substitution of mechanical/computational approach reduces energy consumption per operation while increasing computational throughput through dedicated hardware paths optimized for matrix mathematics.
Solution Approach 2:
The patent performs preliminary organization of matrix data into tile structures that are optimized for parallel processing. This pre-processing of data into appropriate formats enables more energy-efficient computation by the FMA circuits, as the data is already arranged for optimal parallel execution without requiring additional reorganization during computation.
3Speed
If general-purpose processors handle matrix operations, then adaptability is maintained, but processing speed deteriorates for large matrices
Solution Approach 1:
The matrix operations accelerator is designed to handle multiple matrix operations including multiplication, transposition, and accumulation through a unified FMA circuit architecture. This multi-functional design maintains adaptability for different matrix operations while achieving high processing speeds through specialized hardware acceleration for each operation type.
Data Source
Figure 1A~1B
Figure 2(A)~2(C)
Figure 3
AI summary
Systems, methods, and apparatuses relating to a matrix operations accelerator are described. In one embodiment, a processor includes a matrix operations accelerator circuit that includes a two-dimensional grid of fused multiply accumulate circuits; a first plurality of registers that represents an input two-dimensional matrix coupled to the matrix operations accelerator circuit; a decoder, of a core coupled to the matrix operations accelerator circuit, to decode an instruction into a decoded instruction; and an execution circuit of the core to execute the decoded instruction to cause the two-dimensional grid of fused multiply accumulate circuits to form a transpose of the input two-dimensional matrix when the matrix operations accelerator circuit is in a transpose mode.