Matrix Multiplication Hardware with Parallel MAC Units and Double Buffering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large-matrix multiplication operations in the BLAS library face inefficiencies and latency issues, particularly when performing transpose operations, which affect computational performance in AI and HPC fields.
Innovation Solution
A hardware apparatus and method utilizing multiple parallel Multiply-and-Accumulate (MAC) units, with double buffering in X and Y buffers, and a data loader that optimizes matrix data loading modes to perform matrix multiplication operations efficiently, reducing latency by enabling simultaneous parallel operations and efficient memory access.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If transpose operations are performed in matrix multiplication based on BLAS library, then the operation can handle various matrix configurations, but latency is caused that reduces computational performance
Solution Approach 1:
The patent applies preliminary action by pre-transposing matrix data before it enters the computation pipeline. The data loader includes a transpose function that can pre-process matrix data in row-major or column-major order before loading into buffers, eliminating the need for transpose operations during the actual matrix multiplication computation and thereby reducing latency
Solution Approach 2:
The patent implements dynamics by making the data loader configurable to handle different matrix configurations dynamically. The loader can adaptively switch between row-major and column-major loading modes, and can dynamically perform transpose operations only when necessary based on the input matrix configurations, optimizing performance for different BLAS operation types
2Productivity
If multiple parallel MAC units are used, then computational performance is improved, but device complexity increases
Solution Approach 1:
The patent applies segmentation by dividing the computational workload into multiple independent MAC units that operate in parallel. Each MAC unit is a self-contained module that can independently perform multiply-accumulate operations, allowing the system to process multiple elements simultaneously while maintaining modular architecture that manages complexity
Solution Approach 2:
The patent implements universality by designing a unified data loader that serves multiple MAC units and can handle different matrix configurations (row-major, column-major, transposed). The data loader acts as a multi-functional component that adapts to different computation patterns, reducing the need for separate specialized hardware for each operation type
3Device complexity
If data is loaded sequentially into buffers, then device complexity is reduced, but productivity decreases due to insufficient parallel operation
Solution Approach 1:
The patent applies continuity of useful action by implementing double buffering where while one buffer is being read from by MAC units, another buffer is being loaded with data. This overlapping of load and compute operations ensures continuous data flow to the MAC units without idle cycles, maximizing parallel operation efficiency while maintaining a relatively simple buffer structure
Data Source
AI summary
Disclosed herein is an apparatus and method for a matrix multiplication operation. The apparatus may include memory for storing first matrix data and second matrix data, an X buffer for storing the first matrix data, a Y buffer for storing the second matrix data, multiple operation units for performing Multiply-and-Accumulate (MAC) operations in parallel on the data input from the X buffer and the Y buffer, and a data loader for storing the first matrix data and the second matrix data read from the memory in the X buffer and the Y buffer, respectively.