Distributed Model Compilation via Graph Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In distributed model compilation systems, uneven loads between master and slave nodes lead to low resource utilization and excessive load on the master node, resulting in inefficiencies in deep learning model compilation.
Innovation Solution
A distributed model compilation system where the master node determines a logic calculation graph, divides it into sub-graphs, and transmits these sub-graphs to slave nodes for compilation, allowing each slave node to allocate local resources and compile independently, with completion information returned to the master node to determine overall compilation status.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If model compilation is performed centrally on the master node, then compilation control is simplified, but resource utilization decreases and master node load increases
Solution Approach 1:
The patent divides the logic calculation graph into multiple sub-graphs and distributes them to different slave nodes for parallel compilation. This segmentation allows the compilation workload to be distributed across multiple nodes, improving resource utilization while maintaining centralized control through the master node's coordination of the division and distribution process.
Solution Approach 2:
The patent transitions from a single-node compilation architecture to a multi-node distributed architecture, adding a spatial dimension to the compilation process. By utilizing multiple slave nodes simultaneously, the system improves resource utilization and reduces the master node's compilation load while maintaining control through coordinated sub-graph distribution.
2Device complexity
If model compilation is performed centrally on the master node, then system architecture is simplified, but master node load becomes excessive
Solution Approach 1:
The patent segments the compilation tasks by dividing the logic calculation graph into sub-graphs and assigning them to different slave nodes. This reduces the computational load on the master node while maintaining a relatively simple architecture where the master node retains control over task distribution and coordination.
Solution Approach 2:
The master node acts as an intermediary that coordinates between the compilation request and the slave nodes. It divides the work into sub-graphs and distributes them, reducing its direct computational load while maintaining architectural simplicity through centralized coordination.
3Productivity
If compilation tasks are distributed to slave nodes, then resource utilization improves, but system complexity increases
Solution Approach 1:
The patent improves resource utilization by segmenting the compilation workload into sub-graphs that can be executed in parallel on multiple slave nodes. The system complexity is managed by keeping the segmentation logic centralized on the master node, which coordinates the distribution without requiring complex peer-to-peer communication between nodes.
Data Source
AI summary
The present disclosure discloses a distributed model compilation system. A master node of the system determines the logic calculation graph of the model based on model information, divides the logic calculation graph into multiple logic calculation sub-graphs, generates a distributing message for each logic calculation sub-graph, and then transmits the distributing message to a slave node. Each of the slave nodes allocates a local computing resource to compile the logic calculation sub-graph based on the received distributing message, and transmits compilation completion information to the master node. The master node determines the completion of model compilation based on the compilation completion information returned by each slave node, and executes the target work based on the compiled model.