Compute Graph Node Splitting for Parallel Tensor Execution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Optimizing compute graphs for coarse-grained reconfigurable processors to reduce latency and increase throughput remains a challenge.
Innovation Solution
A method for reducing latency and increasing throughput in reconfigurable computing systems by partitioning tensor operations into multiple nodes, allowing parallel processing within a meta-pipeline stage, and adding a separate operation node to gather tensor data, which includes concatenation or summation operations to generate a complete tensor within a single meta-pipeline stage, thereby reducing latency while accounting for additional concatenation latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If a single node processes the entire tensor operation sequentially, then the device complexity is low, but the latency is high and throughput is limited
Solution Approach 1:
The patent divides a single tensor operation node into multiple sub-nodes, each processing a distinct block of the tensor. This segmentation enables parallel execution of multiple blocks within a meta-pipeline stage, reducing overall latency by a factor of X while distributing the computational workload across multiple processing units in the reconfigurable array.
2Productivity
If multiple nodes process tensor blocks in parallel, then the throughput increases, but the device complexity and data gathering requirements increase
Solution Approach 1:
The patent introduces a separate operation node that merges the results from multiple parallel tensor block processing nodes. This merging node collects the processed blocks and combines them (through concatenation or summation operations) to produce the final complete tensor, enabling parallel throughput while maintaining a unified output interface.
3Loss of time
If tensor operations are processed in a single meta-pipeline stage, then the latency is reduced, but the resource requirements and system complexity increase
Solution Approach 1:
The patent reorganizes the compute graph by introducing a new dimensional structure with multiple nodes operating in parallel within a single meta-pipeline stage. This dimensional transformation allows simultaneous processing of multiple tensor blocks without requiring additional pipeline stages, effectively reducing latency while utilizing the reconfigurable array's spatial parallelism.
Data Source
AI summary
A system generates configuration data for a reconfigurable dataflow computing system with an array of configurable units, the configuration data configured to be executed by a reconfigurable dataflow computing system comprising an array of configurable units interconnected with a switching array. The system receives a computational representation for execution on the reconfigurable dataflow computing system, the computational representation comprising a node specifying a data processing operation on associated data, transforms the node into multiple nodes that each specify the data processing operation on a distinct portion of the associated data to produce a modified computational representation. The system then generates the configuration data based at least in part on the modified computational representation, wherein the configuration data, when loaded onto an instance of the reconfigurable dataflow computing system, causes the reconfigurable dataflow computing system to implement at least the modified computational representation.