Graph Spatial Split for Parallel Tensor Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Optimizing compute graphs for coarse-grained reconfigurable processors to reduce latency and increase throughput remains a challenge.
Innovation Solution
A method for reducing latency and increasing throughput in a reconfigurable computing system involves partitioning a tensor into blocks, splitting a compute graph node into multiple nodes, and executing these nodes in parallel within a single meta-pipeline stage, followed by a separate operation to gather the tensor portions, which includes concatenation or summation operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If a single node processes the entire tensor operation sequentially, then the device complexity is low, but the latency is high and throughput is limited
Solution Approach 1:
The patent divides a single tensor operation node into multiple sub-nodes, each processing a distinct block of the tensor. This segmentation enables parallel execution of sub-nodes within a meta-pipeline stage, reducing overall latency by a factor of X (where X is the number of split nodes) while distributing the computational workload across multiple processing units.
2Productivity
If multiple nodes process tensor blocks in parallel, then throughput increases, but additional overhead is required for gathering results
Solution Approach 1:
The patent introduces a separate gathering operation node that receives results from multiple parallel sub-nodes and combines them through concatenation or summation operations. This intermediary node manages the complexity of aggregating parallel results, enabling high throughput processing while maintaining a clean interface between parallel execution and final result collection.
3Speed
If the tensor is partitioned into more blocks, then parallelism increases and latency decreases, but the overhead of additional gathering operations increases
Solution Approach 1:
The patent implements dynamic meta-pipeline stages that can adaptively configure the number of parallel sub-nodes and their corresponding gathering operations based on workload characteristics. This dynamic approach allows the system to optimize the balance between parallelism benefits and gathering overhead, adjusting the degree of node splitting to match computational requirements and minimize total execution time.
Data Source
AI summary
A method for reducing latency and increasing throughput in a reconfigurable computing system includes receiving a compute graph for execution on a reconfigurable dataflow processor comprising a grid of compute units and grid of memory units interconnected with a switching array. The compute graph includes a node specifying an operation on a tensor. The node may be split into multiple nodes that each specify the operation on a distinctive portion of the tensor to produce a first modified compute graph. The first modified compute graph may be executed. In addition, the multiple nodes may be within a single meta-pipeline stage and may be processed in parallel. Furthermore, the compute graph may further comprise a separate node for gathering the distinctive portions of the tensor into a complete tensor, to produce a second modified compute graph.