Computational Graph Node Fusion for Parallel Branch Execution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing node fusion modes for computational graphs in neural networks have limitations, such as conservative search methods and constraints that prevent the identification of all parallelizable branches, leading to inefficient use of computing resources.
Innovation Solution
A method that considers additional parallelizable branch combinations beyond conventional rules, allowing for the fusion of nodes that support parallel execution, thereby expanding the range of nodes that can be fused.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If topological sorting is used to determine execution sequence, then execution dependency relationship is established, but execution time increases due to implicit sequencing of data-independent tasks
Solution Approach 1:
The patent segments the computational graph into independent branches based on data dependency analysis. By identifying branches that do not share data dependencies, the system can execute nodes in parallel across different branches, eliminating the sequential execution constraint imposed by topological sorting while maintaining correct execution dependencies within each branch.
Solution Approach 2:
The patent introduces a new dimension of execution organization by transitioning from single-sequence topological ordering to multi-branch parallel execution. This dimensional change allows simultaneous execution of independent branches, transforming the execution model from one-dimensional sequence to multi-dimensional parallel structure.
2Productivity
If existing node fusion modes (horizontal fusion, operator-level parallelism) are used, then some nodes can be fused, but many parallelizable nodes are missed due to restrictive constraints
Solution Approach 1:
The patent changes the parameters of node fusion by relaxing the strict constraints of existing modes. Instead of requiring nodes to have identical data layouts or come from the same parent node, the system identifies fusion candidates based on branch independence and computational characteristics, allowing heterogeneous nodes to be fused when they support parallel execution.
Solution Approach 2:
The patent creates a universal node fusion mechanism that can handle multiple fusion scenarios beyond the limited cases of horizontal fusion and operator-level parallelism. The system identifies parallelizable branches and applies fusion rules that work across different node types and structures, making the fusion process universally applicable to various computational graph configurations.
3Reliability
If conservative search methods are used for node fusion, then search stability is maintained, but the range of fusible nodes is limited
Solution Approach 1:
The patent introduces dynamic search criteria that adapt to different computational graph structures. Instead of using fixed conservative rules, the system dynamically identifies parallelizable branches and fusion candidates based on the specific characteristics of each graph, allowing the search to adjust its behavior to maximize fusion opportunities while maintaining stability through structured search algorithms.
Data Source
Figure 1~2
Figure 3(a)~3(b)
Figure 4
AI summary
Embodiments of this application disclose a node fusion method for a computational graph and a device, which may be used in the artificial intelligence field, and may be specifically used in a deep learning framework. The method includes: converting a neural network into a computational graph; extracting one or more parallelizable branch groups from the computational graph based on a dependency relationship between nodes in the computational graph, where the dependency relationship indicates at least one of the following relationships: the parallelizable branch group has a common parent node, the parallelizable branch group has a common child node, the parallelizable branch group has no parent node, and the parallelizable branch group has no child node; and finally, fusing a plurality of nodes in any parallelizable branch group that respectively belong to different sub-branches to obtain a new computational graph. In this application, possibilities of some other parallelizable branches are considered, so that parallelizable branch combinations different from those defined by a rule in the conventional technology are found, and nodes in the branch combinations may be fused during node fusion of a computational graph. This extends a range of nodes that support fusion and that can be obtained.