NUMA Thread Scheduling via Memory Proximity Lists
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Non-uniform memory access (NUMA) architectures face challenges in achieving load balance due to varying memory access times across processing cores, leading to inefficiencies and delays in thread execution.
Innovation Solution
A system that determines the topology of a NUMA architecture, generates a proximity list ranking cores by memory latency, and selectively transfers threads from over-utilized to under-utilized cores, prioritizing new nested threads over suspended threads to minimize memory latency and optimize core utilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If threads are transferred between cores in NUMA architecture without considering memory proximity, then load balancing is achieved, but memory access latency increases
Solution Approach 1:
The patent applies local quality by creating proximity lists that rank source cores based on their memory access latency to the target core. When performing load balancing, the system preferentially selects source cores with lower latency (better local memory access) over those with higher latency, thus optimizing memory access performance while maintaining load balancing.
Solution Approach 2:
The patent implements preliminary action by pre-computing proximity lists for each core that rank other cores based on memory access latency. These proximity lists are calculated in advance and stored, so when load balancing is needed, the system can quickly reference the pre-computed lists to select optimal source cores without performing real-time latency calculations.
2Productivity
If threads with large state information are transferred, then load balancing is improved, but transfer time and overhead increase
Solution Approach 1:
The patent applies partial action by implementing a preference for transferring new nested threads over suspended threads. New nested threads typically have smaller or no state information compared to suspended threads that have been executing. By partially transferring only the necessary thread context (and preferring threads with less state), the system achieves load balancing while minimizing transfer overhead and time.
3Ease of manufacture
If random core selection is used for thread transfer, then implementation simplicity is maintained, but memory access performance deteriorates
Solution Approach 1:
The patent implements preliminary action by pre-computing proximity lists that rank source cores based on memory access latency. These lists are calculated in advance using the NUMA topology and memory controller information, storing the ranked order of cores for each target core. This pre-computation maintains implementation simplicity while eliminating random selection, as the system can directly reference the pre-sorted lists during load balancing operations.
Solution Approach 2:
The patent applies local quality by making the core selection process aware of memory access characteristics. Instead of uniform or random selection, the proximity lists encode local memory access quality information, allowing the scheduler to preferentially select source cores with better (lower latency) memory access paths to the target core.
Data Source
AI summary
A topology reader may determine a topology of a Non-Uniform Memory Access (NUMA) architecture including a number of, and connections between, a plurality of sockets, each socket including one or more cores and at least one memory configured to execute a plurality of threads of a software application. A core list generator may generate, for each designated core of the NUMA architecture, and based on the topology, a proximity list listing non-designated cores in an order corresponding to a proximity of the non-designated cores to the designated core. A core selector may determine, at a target core and during the execution of the plurality of threads, that the target core is executing an insufficient number of the plurality of threads, and may select a source core at the target core, according to the proximity list associated therewith, for subsequent transfer of a transferred thread from the selected source core to the target core for execution thereon..