Master Node Task Segmentation for Distributed Computing Clusters
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In distributed deep learning model training, the master computing device experiences high load and low efficiency due to the need to compile and distribute task code to multiple computing devices, leading to inefficient task execution.
Innovation Solution
A method where the master computing device divides tasks into subtasks, determines required operators, and distributes them to computing nodes in a cluster system, allowing each node to generate executable subgraphs and execute tasks independently, thereby reducing the master node's workload and improving efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the master computing device compiles and distributes task code to multiple computing devices, then the task can be executed in a distributed manner, but the load on the master computing device increases and execution efficiency decreases
Solution Approach 1:
The patent segments the centralized compilation and distribution task into smaller subtasks that can be independently executed by different computing devices. Each computing device receives only the necessary subgraph information for its specific computation nodes, rather than the entire task code being processed centrally by the master device. This segmentation reduces the master device's workload while maintaining distributed execution capabilities.
Solution Approach 2:
The patent performs preliminary analysis of the task code to identify computation nodes and their dependencies before distribution. The master computing device pre-processes the task code to generate computation node information and data transmission relationships, which are then distributed to appropriate computing devices. This preliminary action reduces the real-time processing burden on the master device during task execution.
2Reliability
If the master computing device compiles the complete task code, then all computation nodes can be executed, but the compilation time and processing load increase significantly
Solution Approach 1:
The task code is segmented into computation nodes based on their functional dependencies. Each computing device receives only the subgraph information necessary for its assigned computation nodes, rather than the complete task code. This segmentation maintains execution completeness while significantly reducing compilation time for each device.
Solution Approach 2:
Each computing device receives customized subgraph information tailored to its specific computation nodes and data transmission requirements. The compilation process generates location-specific and device-specific execution plans, ensuring that each device has exactly the information it needs for its local computation tasks, reducing unnecessary processing overhead.
Data Source
AI summary
The present disclosure discloses a method for executing a task. The method includes: a master computing device node in a computing cluster system receives a task code of a to-be-executed task; the master computing device node divides the to-be-executed task into subtasks, and for each of the subtasks, the master computing device node determines operators required to execute the subtask based on the task code; the master computing device node respectively distributes the subtasks to computing nodes in the computing cluster system, such that for each of the computing nodes, the computing node generates an executable task subgraph for the computing node based on the operators required to execute the subtask distributed to the computing node and data transmission relationships between the operators required to execute the subtask distributed to the computing node, and runs the executable task subgraph to execute the to-be-executed task.