Unified Near Far Memory Architecture for Data Center Resource Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current enterprise/cloud computer systems face inefficient resource utilization and higher costs due to over-allocation of resources to meet performance requirements, leading to suboptimal use of rack resources in data centers.
Innovation Solution
A unified hardware and software two-level memory scheme is introduced, combining 'near' and 'far' memory in a transparent manner to application software, where near memory uses volatile memory like DDR4/5 DRAM and far memory uses non-volatile storage-class memory (SCM) accessed via a low-latency fabric, allowing dynamic composition of memory resources based on workload demands.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If over-allocation of memory resources is implemented to meet performance requirements, then system reliability is improved, but device complexity and cost increase
Solution Approach 1:
The memory system is segmented into two distinct levels: near memory (volatile DDR4/5 DRAM) and far memory (non-volatile storage-class memory). This segmentation allows each memory type to be optimized for its specific function while being managed as a unified resource, resolving the contradiction by providing reliability through diversified memory architecture rather than over-allocation of a single memory type.
Solution Approach 2:
The patent introduces a new dimension to the memory hierarchy by adding far memory accessed through a low-latency fabric, creating a two-level memory architecture. This dimensional expansion allows the system to achieve reliability through architectural diversity rather than simply increasing the quantity of a single memory type, thereby avoiding the complexity and cost of over-allocation.
2Productivity
If over-allocation of compute resources is implemented to meet performance requirements, then productivity is improved, but loss of substance increases
Solution Approach 1:
The unified two-level memory system serves multiple functions: it provides high-speed access for active data (near memory) and persistent storage for inactive data (far memory), while being managed as a single unified resource. This multi-functionality allows the same memory infrastructure to support both high productivity through fast access and high resource utilization through dynamic composition, eliminating the need for over-allocation.
Solution Approach 2:
The patent implements dynamic composition of memory resources where the system can adaptively allocate memory capacity between near and far levels based on workload demands. This dynamic behavior allows the system to maintain high productivity by providing fast access when needed while maximizing resource utilization by using far memory for less frequently accessed data, avoiding the waste associated with static over-allocation.
3Adaptability or versatility
If unified two-level memory is implemented, then resource utilization is improved, but device complexity increases
Solution Approach 1:
The patent introduces a low-latency fabric as an intermediary component that connects near memory and far memory, enabling unified access to both memory levels. This intermediary abstraction simplifies the management complexity by providing a standardized interface for memory access, allowing the system to achieve high adaptability in resource allocation without proportionally increasing overall system complexity.
Data Source
AI summary
Unified hardware and software two-level memory mechanisms and associated methods, systems, and software. Data is stored on near and far memory devices, wherein an access latency for a near memory device is less than an access latency for a far memory device. The near memory devices store data in data units having addresses in a near memory virtual address space, while the far memory devices store data in data units having addresses in a far memory address space, with a portion of the data being stored on both near and far memory devices. In response to memory read access requests, a determination is made to where data corresponding to the request is located on a near memory device, and if so the data is read from the near memory device; otherwise, the data is read from a far memory device. Memory access patterns are observed, and portions of far memory that are frequently accessed are copied to near memory to reduce access latency for subsequent accesses.