ML Workload Scheduling Using NUMA-Aware Task Placement
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing distributed computing systems face performance bottlenecks due to non-local memory access and data communications between resources, which are bandwidth intensive and inefficient.
Innovation Solution
Implement NUMA-aware scheduling and resource allocation techniques that assign tasks to resource groups within a distributed system based on a non-uniform memory access (NUMA) topology, leveraging resource locality to reduce non-local memory access and data communications.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If tasks are assigned to distributed resources without considering memory topology, then resource utilization is improved, but memory access time and bandwidth consumption increase
Solution Approach 1:
The patent applies local quality by assigning tasks to specific NUMA nodes based on their memory access characteristics. Each NUMA node has local memory with faster access times, and the scheduler identifies which nodes have local memory resources to assign tasks, thereby reducing memory access time while maintaining high resource utilization through targeted task placement.
2Adaptability or versatility
If tasks are assigned to remote resources in a distributed system, then system scalability is improved, but data communication bandwidth consumption increases
Solution Approach 1:
The patent segments the distributed system into multiple NUMA nodes, each with its own local memory resources. The scheduler divides tasks and assigns them to specific NUMA nodes based on local resource availability. This segmentation allows the system to scale by adding more NUMA nodes while minimizing cross-node communication and bandwidth consumption by keeping task execution localized.
3Adaptability or versatility
If non-local memory access operations are performed, then resource flexibility is improved, but performance bottlenecks increase
Solution Approach 1:
The patent performs preliminary action by pre-identifying NUMA node topology and local memory resources before task assignment. The scheduler proactively maps tasks to NUMA nodes that have local memory resources available, preventing non-local memory access operations before they occur. This preliminary topology-aware assignment maintains resource flexibility while avoiding performance bottlenecks associated with remote memory access.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, are described for scheduling tasks of ML workloads. A system receives requests to perform the workloads and determines, based on the requests, resource requirements to perform the workloads. The system includes multiple hosts and each host includes multiple accelerators. The system determines a quantity of hosts assigned to execute tasks of the workload based on the resource requirement and the accelerators for each host. For each host in the quantity of hosts, the system generates a task specification based on a memory access topology of the host. The specification specifies the task to be executed at the host using resources of the host that include the multiple accelerators. The system provides the task specifications to the hosts and performs the workloads when each host executes assigned tasks specified in the task specifications for the host.