Vector Packed Matrix Multiplication Accumulation Processors
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current processors are inefficient in performing matrix operations, particularly in storing and processing matrices within vector registers, as they typically handle one-dimensional vectors rather than two-dimensional matrices, leading to suboptimal performance in applications like artificial intelligence and machine learning.
Innovation Solution
The introduction of a vector packed matrix multiplication and accumulation instruction that allows matrices to be stored and processed within single vector registers, enabling efficient matrix multiplication and accumulation operations across various data types and bit widths, including 8-bit, 16-bit, 32-bit, and floating-point formats.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If matrices are stored and processed using traditional one-dimensional vector registers, then the processor can handle vector operations, but the processing efficiency for matrix operations is low
Solution Approach 1:
The patent applies dimensionality change by transitioning from one-dimensional vector register storage to two-dimensional matrix storage within vector registers. The matrix storage format organizes data with rows and columns, allowing simultaneous access to multiple matrix elements during multiplication operations. This dimensional transformation enables the processor to perform matrix multiplication more efficiently by reducing the number of memory accesses and enabling parallel computation across multiple elements simultaneously.
2Productivity
If vector registers are used for matrix operations, then processing can be performed, but the performance in AI and machine learning applications is suboptimal
Solution Approach 1:
The patent implements universality by creating a matrix storage and processing mechanism that works across multiple data types (8-bit, 16-bit, 32-bit integers and floating-point formats) and various matrix dimensions. The same vector register infrastructure is extended to handle different matrix operation types including multiplication, accumulation, and transposition. This multi-functional approach allows AI and machine learning applications to achieve optimal performance without requiring separate specialized hardware for each operation type.
3Productivity
If traditional vector processing methods are used, then the processor architecture remains simple, but processing efficiency for matrix operations is insufficient
Solution Approach 1:
The patent applies merging by combining matrix multiplication and accumulation operations into a single unified instruction cycle. Instead of performing multiplication and then separately performing accumulation, the patent merges these operations so that multiplication results are directly accumulated into the destination matrix in one operation. This merging reduces the total number of instruction cycles required and improves processing efficiency while maintaining the existing vector register architecture.
Data Source
AI summary
Decoder circuitry to decode an instruction indicating a first vector register having a 128-bit lane to store a first matrix having two rows by K columns of data elements having a number of bits, a storage location having 128 bits to store a second matrix having K rows by two columns of data elements having the number of bits, and a second vector register having a 128-bit lane to store a third matrix having two rows by two columns of data elements having a greater number of bits. Execution circuitry is to perform operations for the instruction, including to generate and store a result matrix having two rows by two columns of result data elements having the greater number of bits in 128-bit lane of second vector register. The result matrix represents accumulation of the third matrix with product matrix generated from matrix multiplication using the first and second matrices.