Parallel MAC Workload Balancing for Full Core Utilization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing techniques for parallelizing multiply-accumulate computations, such as General Matrix Multiplications (GEMMs), often result in underutilization of multi-core processors due to partially filled waves when the number of output tiles does not align with the maximum wave size, leading to inefficient resource use and degraded computational throughput.
Innovation Solution
An iteration-based parallel decomposition strategy is employed to equitably distribute MAC iterations across processor cores, ensuring each core executes either x or (x+1) iterations, thereby optimizing resource utilization and reducing idle time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the number of output tiles does not align with the maximum wave size, then the GEMM computation can be completed, but the multi-core processor becomes underutilized with degraded computational throughput
Solution Approach 1:
The patent segments the total number of MAC iterations into multiple batches, where each batch is further divided into wave-sized groups. This segmentation allows the computation to be organized into complete waves that fully utilize all processor cores, eliminating the underutilization problem caused by partial wave execution.
Solution Approach 2:
The patent dynamically adjusts the wave size parameter based on the total number of MAC iterations and processor configuration. By making the wave size adaptive rather than fixed, the system can optimize processor utilization for different computation workloads, ensuring that each wave fully utilizes available cores.
2Productivity
If multiple GEMM programs are created for different tile sizes to optimize performance, then processing performance can be optimized, but the device complexity and resource requirements increase significantly
Solution Approach 1:
The patent creates a universal GEMM program that can handle different tile sizes and computation workloads through dynamic parameter adjustment. Instead of maintaining separate specialized programs for each tile size, a single program implements adaptive wave size calculation and batched execution, making it multi-functional across different GEMM configurations.
Solution Approach 2:
The patent changes the execution parameters (wave size, number of batches, iterations per wave) based on the input matrix dimensions and processor characteristics. This parameter adaptation allows a single GEMM program to optimize performance for different tile sizes without requiring multiple hard-coded programs, reducing device complexity while maintaining high performance.
Data Source
AI summary
In various embodiments, a dispatch application performs multiply-accumulate (“MAC”) computations across parallel processing elements. In operation, the dispatch application determines a first quantity of iterations associated with a given MAC computation. The dispatch application determines a maximum number of tasks that can execute concurrently across a set of parallel processing elements. Subsequently, the dispatch application causes the maximum number of tasks to be executed concurrently across the set of parallel processing elements in order to perform the MAC computation. During execution, each task performs a substantially similar number of the first quantity of iterations. Relative to conventional tile-based approaches to performing MAC computations across parallel processing elements, the dispatch application can more evenly distribute iterations across the different parallel processing elements. Accordingly, the dispatch application can reduce the amount of parallel processing element idle time when performing MAC computations.