NUMA Cache Replacement Policy Optimizing Memory Access Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In NUMA architectures, the conventional LRU cache replacement algorithm fails to optimize system performance due to its inability to consider the varying access overheads across different NUMA nodes, leading to inefficient memory access and increased latency.
Innovation Solution
A new cache replacement policy that allocates replacement priorities based on the distance between the home NUMA node of a data block and the current NUMA node, as well as other access attributes like access latency and frequency, to store data blocks for longer periods in cache spaces closer to the processor, thereby reducing remote memory access overheads.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the conventional LRU cache replacement algorithm is used, then the cache management is simple, but the system performance deteriorates due to inability to consider varying access overheads across different NUMA nodes
Solution Approach 1:
The patent applies local quality by making the cache replacement policy location-aware. Different cache lines are treated differently based on their NUMA node location. The replacement priority is adjusted according to the distance between the home NUMA node and current NUMA node, creating non-uniform replacement behavior that optimizes for local memory access performance while managing remote memory access overheads.
Solution Approach 2:
The patent changes the replacement parameter from simple LRU (least recently used) to a multi-parameter policy that considers access latency, access frequency, and NUMA node distance. This transforms the single-dimensional LRU algorithm into a multi-dimensional replacement strategy that dynamically adjusts replacement priorities based on multiple factors including the distance between NUMA nodes.
2Loss of time
If data blocks are stored in cache spaces closer to the processor, then memory access latency is reduced, but cache management complexity increases due to need to track home NUMA nodes and access attributes
Solution Approach 1:
The patent applies preliminary action by pre-calculating and storing the home NUMA node information when data blocks are first allocated or migrated. This home NUMA node identifier is maintained in the cache line metadata, allowing the replacement policy to quickly determine replacement priorities without performing complex real-time calculations. The access attributes are tracked and updated in advance to facilitate efficient replacement decisions.
3Productivity
If remote memory access overheads are reduced through strategic cache management, then system efficiency improves, but the cache replacement policy becomes more complex requiring tracking of multiple access attributes
Solution Approach 1:
The patent implements feedback mechanisms by continuously monitoring access attributes such as access frequency and access latency for each cache line. This feedback information is used to dynamically adjust replacement priorities, creating a closed-loop control system that adapts to actual memory access patterns. The home NUMA node information serves as feedback about the optimal location for data blocks, guiding replacement decisions to minimize remote access overheads.
Data Source
AI summary
A data management method is applied to a computing system. The computing system includes a plurality of NUMA nodes, each NUMA node includes a processor and a memory, and each memory is used to store a data block. In the method, a processor in a NUMA node receives an operation request for a data block, and the processor processes the data block, and allocates a replacement priority of the data block in cache space of the NUMA node based on an access attribute of the data block, where the access attribute of the data block includes a distance between a home NUMA node of the data block and the NUMA node.