Dynamic Iteration Ports in Heterogeneous Dataflow Engines
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current dataflow frameworks struggle with expressing iteration and recursion in heterogeneous systems, leading to performance degradation, resource inefficiency, and complexity in programming GPUs and other accelerators due to the need for explicit control flow and synchronization, which limits flexibility and modularity.
Innovation Solution
The implementation of an iterative dataflow engine (IDEA) that supports arbitrary loop nesting and data-dependent termination without adding vertices to the dataflow graph, using constructs like multiport, predicated channels, and iterator ports to manage iteration and control flow, allowing for efficient scheduling across heterogeneous processors and accelerators.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If loop unrolling is used to implement iteration in DAGs, then the graph is static and constructed in advance, but the number of iterations becomes static and graph size grows with the number of iterations
Solution Approach 1:
The patent introduces dynamic iteration support by adding iteration ports and channels that allow the dataflow graph to adapt its execution behavior at runtime. The iteration port on each vertex can receive iteration counts dynamically, and the execution engine can determine actual iteration counts based on data-dependent termination criteria, making the graph size independent of iteration count.
Solution Approach 2:
The patent adds a new dimension to the dataflow graph model by introducing iteration ports and iteration channels alongside the traditional data ports and data channels. This allows iteration control information to flow independently through the graph, enabling dynamic iteration without expanding the computational vertices.
2Adaptability or versatility
If cyclic graphs are used to support iteration, then data-dependent termination can be achieved, but problem-dependent knowledge is required to schedule vertices
Solution Approach 1:
The execution engine automatically handles iteration scheduling by monitoring data-dependent termination criteria and controlling the flow of iteration tokens through the graph. The system self-manages the iteration process without requiring external problem-dependent scheduling knowledge, as the dataflow semantics naturally express when iteration should terminate.
3Reliability
If explicit control code is written for data movement between heterogeneous devices, then data synchronization can be managed, but programmer time is monopolized and bugs can occur
Solution Approach 1:
The patent introduces dataflow vertices as intermediaries that encapsulate data movement and synchronization operations between heterogeneous devices. Instead of writing explicit control code, programmers declare data dependencies in the graph, and the execution engine automatically manages the intermediate data transfer and synchronization, eliminating the need for manual synchronization code while improving reliability.
4Adaptability or versatility
If additional vertices are added to represent iteration constructs, then iteration can be expressed explicitly, but performance degrades and memory resources are consumed
Solution Approach 1:
The patent makes existing dataflow vertices universal by adding iteration ports to them, allowing the same vertices to participate in both data computation and iteration control. This multi-functionality enables iteration to be expressed without adding dedicated iteration construct vertices, thereby avoiding the performance degradation and memory overhead associated with additional vertices.
Data Source
AI summary
Various embodiments provide techniques and constructs to improve execution speed of distributed iterative computation using heterogeneous specialized resources including, for example, processors and accelerators. Iteration over an arbitrary sub-graph without loop unrolling including for algorithms with data-dependent loop termination and large iteration counts, including as a result of nested iteration, are supported in a resource-efficient manner without adding vertices to a dataflow graph to represent iteration constructs. Instead, some or all of the existing vertices within the sub-graph that is to be iterated upon based on having additional and/or modified ports and channels associated with them.