Streaming Multiprocessor GEMM Unit for Vector Computing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current vector computing systems face limitations in performing matrix multiply and accumulation (MMA) calculations, particularly in big data and artificial intelligence applications, due to imbalanced computation capacities between streaming multiprocessor units and the need for excessive communication between coprocessors and general-purpose registers.
Innovation Solution
Incorporating a general matrix multiply (GEMM) calculation unit within the streaming multiprocessor cluster, which reads data from and writes results directly to general-purpose registers or a local cache, allowing for efficient MMA calculations without the need for a new instruction set architecture and reducing communication overhead.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Power
If a separate coprocessor is used for MMA calculations, then the computation capacity for AI applications is improved, but the communication overhead between coprocessor and general-purpose registers increases
Solution Approach 1:
The patent merges the GEMM calculation unit directly into the streaming multiprocessor cluster, combining previously separate components (vector computing unit and matrix multiply unit) into a unified architecture. This integration eliminates the need for data transfer between separate coprocessors and general-purpose registers, thereby reducing communication overhead while maintaining enhanced computation capacity for both vector operations and MMA calculations
2Adaptability or versatility
If the vector computer is equipped with dedicated MMA calculation units, then the capability for big data and AI applications is improved, but the device complexity increases
Solution Approach 1:
The streaming multiprocessor cluster is designed with multi-functionality, serving both as a vector computing unit and a matrix multiply unit. The same physical infrastructure (streaming multiprocessors, registers, and interconnects) handles both vector operations and MMA calculations, eliminating the need for separate dedicated hardware for each function and thus avoiding increased device complexity while enhancing versatility for AI applications
3Power
If separate vector computing and MMA calculation units are used, then the specialized computation performance is improved, but the parallel execution efficiency decreases
Solution Approach 1:
The patent combines vector computing and MMA calculation capabilities within the same streaming multiprocessor cluster, allowing both types of operations to execute in parallel using shared resources. This unified architecture enables simultaneous vector operations and matrix multiply operations without requiring separate execution paths, thereby maintaining specialized computation performance while improving parallel execution efficiency
Data Source
AI summary
The invention relates to an apparatus for vector computing incorporating with matrix multiply and accumulation (MMA) calculation. The apparatus includes a streaming multiprocessor (SM), and a block selector. The register space is divided into physical blocks, each of which includes register groups, and a general matrix multiply (GEMM) calculation unit. The SM includes a general-purpose register (GPR), and the GEMM calculation unit includes an instruction queue and a arithmetic logical unit (ALU). The ALU coupled to the GPR is arranged operably to perform MMA calculation according to a GEMM instruction stored in the instruction queue, and store a calculation result in the GPR.