NUMAlloc Memory Allocator for NUMA Architecture
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing memory allocators for NUMA architecture computers fail to effectively manage memory allocation and deallocation, leading to performance bottlenecks due to remote accesses, cache contention, node imbalance, and interconnect congestion.
Innovation Solution
The NUMAlloc memory allocator employs fine-grained memory management, binding-based memory binding, origin-aware memory allocation, incremental sharing of transparent huge pages, and an interleaved heap to minimize performance bottlenecks and improve locality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If general-purpose memory allocators are used in NUMA architecture, then memory allocation is simple and universal, but performance deteriorates due to remote accesses and cache contention
Solution Approach 1:
The patent implements per-node memory allocators that are tailored to specific NUMA nodes, allowing each node to manage its own memory locally. This resolves the contradiction by making the memory allocation system adaptive to local NUMA characteristics while maintaining good performance through localized memory access, avoiding remote accesses that would slow down execution.
Solution Approach 2:
The patent divides the global memory space into per-node memory spaces, with each NUMA node having its own dedicated memory allocator and memory pool. This segmentation allows each node to independently manage its memory resources, improving execution speed by eliminating cross-node memory accesses while maintaining versatility through a systematic approach that can be applied across different NUMA configurations.
2Quantity of substance
If memory is allocated from remote nodes in NUMA architecture, then memory capacity is increased, but access latency increases and performance decreases
Solution Approach 1:
The patent pre-allocates memory pools to each NUMA node before runtime, establishing local memory resources in advance. This preliminary action ensures that each node has sufficient memory capacity allocated to it locally, eliminating the need for remote memory accesses during execution and thereby reducing access latency while maintaining adequate memory capacity.
Solution Approach 2:
The patent introduces per-node memory allocators as intermediaries between the application and the physical memory. These allocators act as local buffers that manage memory requests within each NUMA node, preventing direct remote accesses to memory and reducing latency by handling memory operations locally before any potential remote transfers are needed.
3Productivity
If multiple threads share the same memory pool in NUMA architecture, then memory utilization is improved, but cache contention and node imbalance increase
Solution Approach 1:
The patent segments the shared memory pool into per-node memory spaces, with each NUMA node having its own dedicated memory allocator. This segmentation allows multiple threads to share memory within their local node without causing cross-node contention, maintaining high memory utilization while ensuring performance stability by keeping memory access patterns localized to each node.
Solution Approach 2:
The patent implements local memory management where each NUMA node has its own memory pool and allocator tailored to that node's characteristics. This local quality approach allows threads to efficiently share memory resources within the same node while avoiding the performance instability caused by remote accesses and cross-node cache contention, thereby maintaining both high utilization and stable performance.
4Productivity
If thread migration is allowed in NUMA architecture, then load balancing is improved, but remote accesses and cache invalidation increase
Solution Approach 1:
The patent binds threads to specific NUMA nodes in advance, performing this allocation as a preliminary action before thread execution begins. This preliminary binding prevents thread migration during runtime, thereby avoiding the increased memory management complexity that would arise from tracking thread locations, invalidating caches, and managing remote memory accesses associated with dynamic migration.
Solution Approach 2:
Instead of allowing threads to migrate freely and managing the consequences, the patent inverts the approach by preventing migration through static binding. This inversion simplifies memory management by eliminating the need to handle thread mobility, cache coherence across nodes, and remote memory access tracking, thereby reducing device complexity while still achieving load balancing through the initial binding configuration.
Data Source
AI summary
A system and method for memory allocation and management in non-uniform memory access (“NUMA”) architecture computing environments is disclosed. The system and method contemplates both hardware heterogeneity and allocation/deallocation attributes, with fine-grained memory management. NUMAlloc is centered on a binding-based memory management. On top of it, NUMAlloc proposes an “origin-aware memory management” to ensure the locality of memory allocations and deallocations, as well as a method called “incremental sharing” to balance the performance benefits and memory overhead of using transparent huge pages. It further introduced an interleaved heap to reduce the load imbalance among different nodes and an efficient mechanism for object movement. The system and method provides a scalable and increased performance alternative over other prior art memory allocators.