Memory Tiering with Near-Memory Swap Buffering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
High-speed DDR SDRAM used as near memory in computing devices is expensive and not accessible to the Operating System, contributing significantly to datacenter server costs, as it is exclusively controlled by the CPU and used as dedicated cache memory.
Innovation Solution
Implementing memory multi-tiering where near memory is used as a swap buffer for far memory, allowing both to be exposed to the Operating System as addressable system memory, with a hardware memory controller managing swapping operations at a cacheline granularity, enabling the near memory to continue caching data while providing allocatable system memory.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If DDR SDRAM is used as near memory for CPU, then access speed is improved, but cost increases significantly
Solution Approach 1:
The near memory is designed to serve dual functions: acting as cache memory for the CPU and as accessible system memory for the operating system. This multi-functionality allows the same hardware resource to provide both high-speed caching benefits and expandable system memory capacity, thereby reducing the need for additional expensive components while maintaining performance benefits.
2Speed
If DDR SDRAM is used as near memory for CPU, then access speed is improved, but memory availability to OS deteriorates
Solution Approach 1:
The system dynamically manages the near memory by allowing the operating system to allocate and deallocate memory regions as needed. The near memory can be flexibly configured between CPU-cache mode and OS-accessible mode, enabling adaptive resource management that responds to changing system requirements and workload characteristics.
Solution Approach 2:
The near memory is designed to serve dual functions: acting as cache memory for the CPU and as accessible system memory for the operating system. This multi-functionality allows the same hardware resource to provide both high-speed caching benefits and expandable system memory capacity, thereby reducing the need for additional expensive components while maintaining performance benefits.
3Productivity
If near memory is used as dedicated cache memory, then CPU caching performance is improved, but system memory capacity deteriorates
Solution Approach 1:
The near memory is designed to serve dual functions: acting as cache memory for the CPU and as accessible system memory for the operating system. This multi-functionality allows the same hardware resource to provide both high-speed caching benefits and expandable system memory capacity, thereby reducing the need for additional expensive components while maintaining performance benefits.
Data Source
AI summary
Techniques of memory tiering in computing devices are disclosed herein. One example technique includes retrieving, from a first tier in a first memory, data from a data portion and metadata from a metadata portion of the first tier upon receiving a request to read data corresponding to a system memory section. The method can then include analyzing the data location information to determine whether the first tier currently contains data corresponding to the system memory section in the received request. In response to determining that the first tier currently contains data corresponding to the system memory section in the received request, transmitting the retrieved data from the data portion of the first memory to the processor in response to the received request. Otherwise, the method can include identifying a memory location in the first or second memory that contains data corresponding to the system memory section and retrieving the data from the identified memory location.