Forward-Style Gradient GeMM Graph Splitting for Dataflow Processors
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Optimizing compute graphs for coarse-grained reconfigurable processors remains a challenge, particularly in efficiently executing machine learning and artificial intelligence workloads that require massively parallel computations and complex dataflow graphs.
Innovation Solution
A method for improving runtime performance by receiving a compute graph for execution on a reconfigurable dataflow processor, involving tensor slicing, reshaping, and tiling, followed by general matrix multiplication operations, and appending addition nodes to form a modified compute graph, which enables parallel processing of tensor slices and resource optimization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If template-based operations are used on tensors with shared batch dimension, then the operation can be performed in a unified manner, but the compute resource utilization is poor and place and route issues arise
Solution Approach 1:
The patent applies segmentation by dividing a single template-based operation node into multiple independent GeMM operation nodes, where each node processes a specific batch element. This segmentation transforms one complex operation into B simpler operations that can be independently scheduled and executed, improving compute resource utilization and avoiding place and route congestion while maintaining the unified operation semantics through subsequent addition nodes.
2Productivity
If nodes are split into multiple parallel nodes for batch processing, then runtime performance improves, but the compute graph complexity increases
Solution Approach 1:
The patent applies merging by combining B parallel GeMM operation nodes through addition nodes that aggregate their outputs. This merging strategy maintains the parallel execution benefits for runtime performance while providing a systematic way to manage compute graph complexity through structured composition of simpler operation nodes.
3Adaptability or versatility
If traditional compilers are used for optimizing compute graphs, then general-purpose processing is maintained, but pipelining operations and data routing become inefficient
Solution Approach 1:
The patent applies parameter changes by transforming the operational parameters of tensor computations from template-based unified operations to explicit GeMM operations with specific batch dimensions. This parameter transformation enables the compute graph to be optimized for dataflow processing, improving pipelining efficiency and data routing while maintaining adaptability through the reconfigurable nature of the underlying processor.
Data Source
AI summary
A method for improving runtime performance and alleviating place and route issues in a reconfigurable computing system includes receiving a compute graph for execution on a reconfigurable dataflow processor. The compute graph includes a node specifying a template-based operation on a first and second tensor having a shared batch dimension B. The node may be split into B nodes. Each of the template-based operations on the pair of tensors may be replace with a GeMM operation on the first reduced rank tensor slice and a tile. B nodes that specify the GeMM operation may be appended with at least one first addition node that accepts input from the B nodes to produce a first modified compute graph. The first modified compute graph may be executed. The method describes a significant improvement to overall compute utilization across gradient-sections. Spatial tiling of tensors facilitates gradient calculation without the use of accumulators.