Machine Learning Computation Graph Partitioning for Resource Decoupling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep learning tasks face reduced running efficiency due to a mismatch between general-purpose computational resources and dedicated computational resources, such as GPUs, leading to insufficient pre-processing data and inefficient hardware utilization.
Innovation Solution
A method and platform for computational optimization of machine learning that decouples data processing and training by dividing the computation graph into data worker and training worker subgraphs, allowing dynamic allocation of CPU resources based on efficiency and scheduling across clusters to improve hardware utilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If data processing and training are compiled together and run in the same machine, then code simplicity is maintained, but hardware resource allocation cannot meet varying task requirements
Solution Approach 1:
The computation graph is divided into two independent subgraphs: data worker subgraph (upstream nodes) and training worker subgraph (stateful nodes and downstream nodes), separated by a partitioned edge. This segmentation allows independent resource allocation for each subgraph, enabling flexible hardware resource configuration that adapts to varying task requirements while maintaining manageable system complexity through modular architecture.
2Productivity
If general-purpose computational resources are configured, then data processing capability is provided, but insufficient data is supplied to dedicated computational resources causing reduced running efficiency
Solution Approach 1:
The system dynamically adjusts the number of data worker units based on the actual data consumption rate of training worker units. When data consumption increases, additional data worker units are spawned to process more data in parallel, ensuring that sufficient data is continuously supplied to dedicated computational resources to maintain high running efficiency without over-provisioning resources.
Solution Approach 2:
The system implements feedback mechanisms where training worker units signal data consumption status to the scheduler, which then adjusts data worker unit allocation accordingly. This feedback loop ensures that data supply quantity dynamically matches actual demand, preventing both data shortages that reduce efficiency and resource waste.
3Power
If dedicated computational resources improve computational capability, then training performance increases, but general-purpose resources cannot provide sufficient data causing bottleneck
Solution Approach 1:
The data worker subgraph performs preliminary data processing actions independently before data is needed for training. By pre-processing and preparing data in advance through the data worker units, the system eliminates data processing bottlenecks that would otherwise limit the computational capability of dedicated resources, ensuring data is ready when training needs it.
Data Source
AI summary
Disclosed are a method and a platform for computational optimization of machine learning. The method includes: partitioning, through a partitioned edge, a machine learning computation graph into a data worker subgraph constituted by upstream nodes of stateful nodes and a training worker subgraph constituted by the stateful nodes and downstream nodes of the stateful nodes; and on each side of the partitioned edge, adding a data sending node to the data worker subgraph and adding a data receiving node to the training worker subgraph, respectively. By dividing the computation graph on the basis of whether a node is stateful or stateless and inserting communication nodes, a data worker and a training worker in a same task can be decoupled, so that general-purpose computational resources involved in the data worker at runtime can be dynamically allocated. Furthermore, through combining with a scheduler, the general-purpose computational resources can be scheduled across clusters.