Memory Tiering Near Far Swap Buffer OS Visibility
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The high cost of DDR SDRAM used as near memory in datacenter servers, which is typically around 50% of the total costs, and the fact that near memory is not visible or accessible to the Operating System, limiting its utilization.
Innovation Solution
Implementing memory multi-tiering by using near memory as a swap buffer for far memory instead of dedicated cache memory, allowing the CPU to continue caching data while exposing near and far memory to the OS as addressable system memory.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If near memory is used as dedicated cache memory for CPU, then CPU access speed is improved, but OS accessibility and memory utilization are worsened
Solution Approach 1:
The near memory is designed to serve multiple functions simultaneously: it acts as cache memory for the CPU to maintain high access speed, while also being exposed as system memory to the OS for general-purpose storage and swapping operations. This multi-functionality resolves the contradiction by allowing the same memory resource to benefit both CPU performance and OS accessibility.
Solution Approach 2:
The memory management system dynamically adjusts the role and usage of near memory based on system conditions. The memory can switch between being primarily CPU cache, OS-accessible system memory, or swap buffer depending on workload requirements, allowing optimal performance characteristics to be maintained while providing OS accessibility when needed.
2Productivity
If near memory is used as dedicated cache memory, then CPU performance is improved, but capital investment costs are worsened
Solution Approach 1:
By making near memory accessible to both CPU and OS simultaneously, the same memory capacity serves dual purposes: fast caching for CPU performance and expandable system memory for capacity needs. This eliminates the need for separate memory allocations, reducing total capital investment while maintaining performance benefits.
Solution Approach 2:
The invention merges the traditionally separate cache memory and system memory into a unified memory space that is accessible by both CPU and OS. This consolidation reduces the total amount of memory hardware needed, as the near memory fulfills multiple roles that previously required distinct memory components.
3Quantity of substance
If near memory is exposed to OS as system memory, then memory capacity and accessibility are improved, but CPU access speed may be worsened
Solution Approach 1:
The memory system is segmented into different accessible regions: a portion of near memory is optimized for CPU access as cache, while another portion is exposed to OS as system memory. This segmentation allows CPU performance-critical operations to use the fast cache portion, while the OS can utilize other portions for storage and swapping, maintaining speed for CPU operations while providing expanded capacity to the OS.
Solution Approach 2:
Different regions of the near memory have different quality characteristics optimized for different purposes. The memory architecture provides locally optimized access paths: direct fast access for CPU cache operations, and controlled access through the memory controller for OS system memory operations. This local quality differentiation maintains high speed for CPU operations while providing expanded capacity to the OS.
4Adaptability or versatility
If near memory is used as swap buffer for far memory, then memory utilization and OS accessibility are improved, but device complexity is worsened
Solution Approach 1:
The memory management system implements self-service mechanisms where the memory controller automatically manages the swap buffer operations between near and far memory. The system monitors memory usage patterns and performs swapping operations autonomously without requiring complex external management, reducing the apparent device complexity while maintaining high utilization and accessibility.
Data Source
AI summary
Techniques of memory tiering in computing devices are disclosed herein. One example technique includes retrieving, from a first tier in a first memory, data from a data portion and metadata from a metadata portion of the first tier upon receiving a request to read data corresponding to a system memory section. The method can then include analyzing the data location information to determine whether the first tier currently contains data corresponding to the system memory section in the received request. In response to determining that the first tier currently contains data corresponding to the system memory section in the received request, transmitting the retrieved data from the data portion of the first memory to the processor in response to the received request. Otherwise, the method can include identifying a memory location in the first or far memory that contains data corresponding to the system memory section and retrieving the data from the identified memory location.