Matrix Multiplication Fusion with GPU Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Matrix multiplication operations in artificial neural networks, particularly on central processing units (CPUs), face significant bottlenecks due to the overhead of loading repetitive matrix elements, limiting the computational potential of even high-speed CPUs.
Innovation Solution
Implementing matrix multiplication fused with reduction (MMFR) operations using multi-core Graphics Processing Units (GPUs), which perform reduction operations in conjunction with matrix multiplication by dividing matrices into tiles and distributing computations across multiple threads, allowing for parallel processing and efficient loading of matrix elements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If matrix multiplication is performed on a single-core CPU, then the computational operations can be executed, but the loading overhead becomes the bottleneck that limits processing speed
Solution Approach 1:
The patent divides the matrix multiplication operation into multiple independent segments that can be processed in parallel. Specifically, it partitions the computation into multiple threads, each handling a portion of the matrix elements, thereby enabling simultaneous processing and reducing the sequential loading overhead that plagues single-core CPU execution.
Solution Approach 2:
The patent transitions from single-core sequential processing to multi-core parallel processing, effectively adding a dimensional aspect to the computation. By distributing matrix multiplication tasks across multiple CPU cores simultaneously, the system overcomes the linear speed limitations of single-core execution and reduces overall loading time through concurrent operations.
2Productivity
If matrix elements are loaded repeatedly for each computation, then the operations can be completed, but the loading overhead increases significantly
Solution Approach 1:
The patent implements preliminary loading of matrix elements into shared memory before the actual computation begins. By pre-loading and caching the necessary matrix data in a shared memory space accessible by multiple threads, the system eliminates repeated loading operations during computation, thereby reducing loading overhead and improving computational throughput.
3Productivity
If fusion of matrix multiplication and reduction operations is implemented, then processing efficiency improves, but the device complexity increases
Solution Approach 1:
The patent merges matrix multiplication and reduction operations into a single fused computational kernel. By combining these two operations that were traditionally executed separately into one unified thread block execution, the patent improves processing efficiency through better data reuse and reduced memory access overhead, while managing the increased operational complexity through systematic thread organization and shared memory management.
Data Source
AI summary
Apparatuses, systems, and techniques to perform matrix multiplication fused with reduction using a graphics processing unit. In at least one embodiment, one or more circuits are used to multiply two or more sub-portions of one or more matrices and generate two or more vectors therefrom using two or more parallel operations.