NUMA-aware memory allocation for multi-processor systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional multi-processor computing systems face bottlenecks due to shared memory controllers, limited memory capacity, and high latency, especially as the number of CPUs increases, leading to inefficiencies in memory access and resource contention in Non-Uniform Memory Access (NUMA) architectures.
Innovation Solution
Implementing a NUMA-aware memory allocation system with dedicated allocators per logical core, using the libnuma library to set NUMA policies for virtual memory addresses, and employing a global freelist partitioned by nodes to optimize memory allocation and reduce remote access penalties, thereby avoiding expensive synchronization mechanisms and kernel space calls.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a single memory controller is shared by all CPUs, then device complexity is reduced, but memory access latency increases and memory capacity is limited
Solution Approach 1:
The patent divides the single memory controller into multiple distributed memory controllers, one per NUMA node. Each memory controller manages local memory resources independently, eliminating the bottleneck of a shared controller and reducing memory access latency for local operations.
Solution Approach 2:
The patent transitions from a centralized memory controller architecture to a distributed architecture across multiple NUMA nodes. This dimensional change allows memory resources to be spatially distributed, enabling parallel access paths and reducing latency through local memory operations.
2Device complexity
If a single memory controller manages all DIMMs, then device complexity is simplified, but the number of supported DIMMs is limited
Solution Approach 1:
The patent segments the memory management function across multiple memory controllers, each handling a subset of DIMMs on its NUMA node. This segmentation allows the system to support a larger total number of DIMMs by distributing the management load across multiple controllers rather than overloading a single controller.
Solution Approach 2:
The patent combines multiple memory controllers and their managed DIMMs into a unified NUMA architecture. While each controller manages local resources independently, the overall system achieves enhanced memory capacity by aggregating resources across multiple nodes, effectively merging their capabilities.
3Device complexity
If multiple CPUs access DRAM simultaneously through a single memory controller, then resource sharing is simplified, but resource contention increases
Solution Approach 1:
The patent segments memory access paths by creating separate access channels through multiple memory controllers distributed across NUMA nodes. Each CPU can access local memory through its nearest memory controller, segmenting the access paths and eliminating contention at a single controller bottleneck.
Solution Approach 2:
The patent implements local memory access optimization where each CPU preferentially accesses memory on its local NUMA node with lower latency. This local quality approach reduces contention by keeping memory traffic local rather than forcing all access through a centralized controller, thereby improving overall memory access efficiency.
4Quantity of substance
If remote memory access is allowed in NUMA architecture, then memory capacity is increased, but access latency increases
Solution Approach 1:
The patent uses NUMA-aware memory allocators to preliminarily allocate memory on the local NUMA node before remote access is needed. By pre-positioning data locally and using memory pinning techniques, the system reduces the need for subsequent remote memory accesses, thereby minimizing latency penalties while still providing access to remote memory when necessary.
Data Source
AI summary
A system includes a first node including a first processor and a first memory, and a second node including a second processor and a second memory. A worker thread of the first processor calls an allocation API to allocate a portion of the first memory, and calls a first allocator of a first memory manager associated with the first node to allocate a specified size of the first memory. The first memory manager calls an operating system function to reserve a memory segment of the specified size, sets a first Non-Uniform Memory Access policy of the first node to preferred, binds the reserved memory segment to the first node, adds the memory segment to a global freelist, and returns an address pointer of the reserved memory segment to the worker thread.