Reconfigurable Matrix Accelerator Grid for Varying Matrix Sizes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current processor architectures face challenges in efficiently handling matrix operations, particularly in deep learning applications, due to the computational intensity of matrix-matrix multiplication and the need for specialized hardware to improve peak compute and energy efficiency.
Innovation Solution
The implementation of a matrix operations accelerator circuit that utilizes a two-dimensional grid of processing element circuits, including configuration switches to switch between modes, allowing for efficient execution of matrix operations by configuring tiles and supporting operations such as matrix multiplication, accumulation, and dot products across multiple matrices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a fixed-size matrix operations accelerator is used, then hardware simplicity is maintained, but utilization efficiency deteriorates when matrix sizes vary
Solution Approach 1:
The patent implements a dynamically reconfigurable matrix operations accelerator where the processing element grid can be reconfigured via configuration switches to accommodate different matrix sizes. This allows the hardware to adapt its structure dynamically, improving utilization efficiency without requiring multiple fixed-size accelerators, thus resolving the contradiction between productivity and device complexity.
Solution Approach 2:
The patent changes the operational parameters of the matrix accelerator by allowing configuration switches to modify the active processing elements and data flow patterns based on the specific matrix operation requirements. This parameter adjustment enables efficient handling of varying matrix sizes without increasing hardware complexity.
2Productivity
If reconfiguration is performed for different matrix sizes, then utilization efficiency improves, but processing time increases due to reconfiguration overhead
Solution Approach 1:
The patent performs configuration setup in advance by loading configuration data into the configuration switches before the actual matrix operation begins. This preliminary action ensures that the accelerator is properly configured for the specific matrix size before processing starts, minimizing reconfiguration overhead during the critical processing phase and reducing overall time loss.
3Adaptability or versatility
If zero padding is applied to match fixed matrix sizes, then compatibility with fixed-size accelerators is maintained, but computational efficiency deteriorates due to unnecessary operations
Solution Approach 1:
Instead of statically padding matrices to fixed sizes, the patent dynamically configures the processing element grid to match the actual matrix dimensions. This dynamic adaptation eliminates unnecessary computations on padded zeros while maintaining compatibility with various matrix sizes, thus resolving the contradiction between adaptability and computational efficiency.
4Speed
If specialized hardware is implemented for matrix operations, then computational speed improves, but device complexity increases
Solution Approach 1:
The patent designs a universal matrix operations accelerator that can handle multiple matrix operation types (multiplication, accumulation, dot products) and various matrix sizes through a single reconfigurable hardware structure. This multi-functionality approach achieves high computational speed for matrix operations without proportionally increasing hardware complexity, as the same hardware resources are flexibly allocated based on operation requirements.
Data Source
Figure 1A~1B
Figure 2(A)~2(C)
Figure 3
AI summary
Systems, methods, and apparatuses relating to a matrix operations accelerator are described. In one embodiment, a processor includes a matrix operations accelerator circuit that includes a two-dimensional grid of fused multiply accumulate circuits that is switchable from a first mode where a respective output of each of a first proper subset of fused multiply accumulate circuits of the two-dimensional grid is transmitted downstream to a respective input of each of a second proper subset of fused multiply accumulate circuits of the two-dimensional grid to form output values from at least one first input two-dimensional matrix and at least one second input two-dimensional matrix, and store the output values in resultant storage, to a second mode where the respective output of each of the first proper subset of fused multiply accumulate circuits of the two-dimensional grid form first output values from a first subset of the at least one first input two-dimensional matrix and the at least one second input two-dimensional matrix, and store the first output values in the resultant storage, and a respective output of each of the second proper subset of fused multiply accumulate circuits of the two-dimensional grid form second output values from a second subset of the at least one first input two-dimensional matrix and the at least one second input two-dimensional matrix, and store the second output values in the resultant storage.