Vector Processing Unit MAC Architecture for Parallel Matrix Operations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current processor architectures face inefficiencies in performing multiply-accumulate (MAC) operations, which are crucial for massive parallel computing applications like machine learning, particularly in graphics processing units (GPUs), where existing methods require tailored programming and struggle with efficient execution of MAC operations across multiple cores.
Innovation Solution
The proposed solution involves a processor architecture with multiple columns of vector processing units, each equipped with a memory port and a processing element containing a vector Arithmetic Logic Unit (ALU), a memory unit, and a vector Multiply-Add (MAD) unit. This architecture allows for efficient loading and processing of matrices, generating multiplication results, and accumulating them using multiple-input adders, enabling efficient MAC operations by distributing the workload across multiple cores.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a processor uses multiple cores for parallel execution in SIMT architecture, then productivity is improved, but device complexity increases and programming difficulty increases
Solution Approach 1:
The processor architecture is segmented into multiple independent columns, each containing a memory port and processing elements. Each column can autonomously perform MAC operations on different data, enabling parallel execution without requiring complex inter-core coordination. This segmentation resolves the contradiction by distributing workload across simple, independent units rather than complex interconnected cores.
Solution Approach 2:
Memory ports serve as intermediaries between the memory unit and processing elements, pre-computing multiplication results before data reaches the processing elements. This intermediary approach simplifies the overall architecture by offloading complex operations to dedicated hardware blocks rather than requiring general-purpose cores to handle all processing tasks.
2Productivity
If a processor implements hardware-supported MAC operations with multiple-input adders, then productivity is improved, but device complexity increases
Solution Approach 1:
Multiple adders are merged into a single multiple-input adder within each processing element, enabling parallel accumulation of multiplication results. This merging approach improves productivity by reducing the number of sequential operations needed while the shared adder resource keeps the individual processing element structure relatively simple.
Solution Approach 2:
The processing elements are designed with universal functionality to handle both data loading from memory ports and execution of MAC operations. The same processing elements perform multiple functions (data reception, multiplication, addition, result output) without requiring separate dedicated hardware for each function, thereby improving productivity without proportionally increasing complexity.
3Ease of operation
If data is loaded into buffers at memory ports before processing, then ease of operation is improved, but loss of time increases
Solution Approach 1:
Data is loaded into buffers at memory ports in advance before the processing elements need it, allowing the processing elements to receive data without waiting for the main computation cycle. This preliminary data loading action simplifies operation by automating data preparation while the asynchronous buffer mechanism minimizes time loss by overlapping data transfer with computation.
Data Source
AI summary
Processors, systems and methods are provided for hardware implemented Multiply-Accumulate (MAC) operations. An exemplary processor may include a memory unit and a plurality of columns of vector processing units coupled to the memory unit. Each column may include a memory port (MP) and a processing element (PE) having a vector ALU. The MP of each column may include an input buffer to store a first matrix loaded from the memory unit and a vector multiply-add (MAD) unit that contains a plurality of MAD units. The vector MAD unit may have a first input coupled to the input buffer and a second input coupled to the memory unit to load a second matrix from the memory unit, and may generate and output a vector of MAD results as a vector input to the vector ALU of the PE of the same column where accumulation is performed.