Nibble Dot-Product Tiles for Large Matrices and Lower Instruction Intensity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing mainstream processors face difficulties in efficiently handling large matrices due to instruction-intensive tasks, particularly when rows of matrices are loaded into packed data registers and scattered back to memory, which is not feasible with larger matrices.
Innovation Solution
The implementation of 2-dimensional data structures called tiles, which are configured to perform matrix operations such as multiplication, addition, and dot products, utilizing hardware support in CPUs, GPUs, and accelerators, with mechanisms like tile configuration, matrix operations accelerators, and fused multiply accumulate operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If rows of matrices are loaded into packed data registers and scattered back to memory, then matrix operations can be performed, but this approach is not feasible with larger matrices due to instruction intensity
Solution Approach 1:
The patent divides large matrices into smaller 4x4 sub-matrices (tiles) that can be processed independently. Each tile is loaded into a packed data register, allowing parallel processing of multiple tiles through dot product operations. This segmentation reduces the instruction intensity for large matrices while maintaining computational efficiency.
Solution Approach 2:
The patent introduces a tile dimension to the traditional matrix operation approach. Instead of processing entire large matrices sequentially, it processes 4x4 tiles in parallel by loading multiple tiles into registers simultaneously. This dimensional approach transforms the problem from sequential row-by-row processing to parallel tile-by-tile processing.
2Area of stationary object
If traditional packed data register operations are used, then small matrices can be processed, but larger matrices become infeasible due to the scattering operations required
Solution Approach 1:
Large matrices are segmented into 4x4 sub-matrices (tiles) that fit within packed data registers. This segmentation allows large matrices to be processed through repeated application of tile-based operations without requiring scattering operations, making large matrix processing feasible.
Solution Approach 2:
The patent implements a nested structure where 4x4 tiles are nested within larger matrices. Each tile is independently processed using dot product operations, and the results are accumulated to form the final large matrix result. This nesting allows efficient processing of large matrices through hierarchical organization.
Data Source
AI summary
Disclosed embodiments relate to computing dot products of nibbles in tile operands. In one example, a processor includes decode circuitry to decode a tile dot product instruction having fields for an opcode, a destination identifier to identify a M by N destination matrix, a first source identifier to identify a M by K first source matrix, and a second source identifier to identify a K by N second source matrix, each of the matrices containing doubleword elements, and execution circuitry to execute the decoded instruction to perform a flow K times for each element (M,N) of the identified destination matrix to generate eight products by multiplying each nibble of a doubleword element (M,K) of the identified first source matrix by a corresponding nibble of a doubleword element (K,N) of the identified second source matrix, and to accumulate and saturate the eight products with previous contents of the doubleword element (M,N).