Bipartite Graph Construction for Deep Neural Network Visualization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for visualizing large deep neural networks during parallel training are inadequate, as tools like TensorBoard fail to clearly display the structure, making it difficult to locate and understand the communication nodes, which are crucial for optimizing parallel policy design and reducing training duration.
Innovation Solution
A method and apparatus for constructing and displaying a bipartite graph that extracts communication nodes to a top layer, allowing for clear visualization and quick location of communication nodes, by cutting cross-communication edges and aggregating computation nodes, thereby facilitating optimal parallel policy design and reducing training duration.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional computational graph visualization tools are used to display large model structures, then the complete graph structure can be shown, but the communication nodes cannot be clearly located and the model structure cannot be clearly displayed
Solution Approach 1:
The patent segments the computational graph into two distinct parts: communication nodes and computation nodes. By separating these elements and displaying them in different visual layers, the patent enables clear location of communication nodes while maintaining the overall graph structure. This segmentation resolves the contradiction by making communication nodes easily identifiable without losing the completeness of the model structure.
Solution Approach 2:
The patent extracts communication nodes from the complex computational graph and places them in a separate top layer for prominent display. This extraction allows users to quickly locate and analyze communication nodes without being overwhelmed by the entire graph structure, while the underlying computation nodes remain visible for context.
2Productivity
If parallel policy is designed without clear communication node identification, then the model can be trained in parallel, but redundant communication nodes are introduced causing performance bottlenecks
Solution Approach 1:
The patent provides visual feedback about communication node locations and connections through the bipartite graph display. This feedback enables researchers to identify redundant communication patterns and optimize parallel policies accordingly, preventing the introduction of unnecessary communication overhead that would slow down training.
Solution Approach 2:
The patent enables preliminary analysis of communication nodes before parallel training execution. By visualizing the computational graph structure and identifying communication nodes in advance, researchers can design optimized parallel policies that minimize communication overhead, preventing performance bottlenecks before they occur during training.
3Measurement precision
If layer-by-layer expansion is used to locate communication nodes, then all nodes can be examined, but the process becomes complex and cumbersome
Solution Approach 1:
The patent introduces a new visual dimension by placing communication nodes in a top layer separate from the main computational graph. This dimensional change allows communication nodes to be displayed prominently without requiring layer-by-layer expansion, making them easily locatable while maintaining the ability to examine all nodes if needed.
Data Source
AI summary
This application discloses a construction method and apparatus for a bipartite graph, and a display method and apparatus for a bipartite graph. The construction method includes: searching a computational graph for at least one cross-communication edge corresponding to a first communication node, where the first communication node is one of M communication nodes included in the computational graph, the first communication node corresponds to P predecessor nodes and Q successor nodes, each of the at least one cross-communication edge indicates a communication path between one of the P predecessor nodes and one of the Q successor nodes, and no cross-communication edge passes through the M communication nodes; and cutting cross-communication edges respectively corresponding to the M communication nodes, and performing an aggregation operation to obtain the bipartite graph, where any two of the M communication nodes are connected without an edge in the bipartite graph.