Unified On-Package Memory Architecture for Bandwidth and Capacity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current memory systems, even with stacked memory dies coupled via an interposer, are often memory bandwidth limited and fail to provide sufficient memory capacity for compute-intensive applications like AI and machine learning, necessitating a solution that enhances both bandwidth and capacity.
Innovation Solution
A combined on-package and off-package memory system with a custom base-layer that includes memory management logic, allowing for unified access to both on-package and off-package memories, providing greater bandwidth and capacity by managing memory access requests and migrating data between different memory pools.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If stacked memory is coupled to a processor using an interposer, then memory bandwidth is improved, but memory capacity remains insufficient for compute-intensive applications
Solution Approach 1:
The patent combines on-package HBM memory and off-package DDR memory into a unified memory system accessible by the processor. The memory management logic integrates both memory types into a single address space, allowing the processor to access both high-bandwidth HBM memory and high-capacity DDR memory through the same interface, thereby simultaneously achieving high bandwidth and large capacity.
Solution Approach 2:
The memory management logic serves multiple functions: it manages both on-package and off-package memory, performs address translation for unified access, and dynamically allocates memory resources. This multi-functional component enables the system to provide both high-speed access and large capacity without requiring separate memory controllers for each memory type.
2Quantity of substance
If more memory capacity is provided using off-package memory, then memory capacity is improved, but access speed decreases compared to on-package memory
Solution Approach 1:
The unified memory space is segmented into different memory pools corresponding to different physical memory locations (on-package HBM and off-package DDR). The memory management logic routes access requests to appropriate memory pools based on address mappings, allowing frequently accessed data to reside in high-speed HBM memory while less frequently accessed data resides in high-capacity DDR memory, thus achieving both high speed and large capacity.
3Ease of operation
If a unified memory interface is provided for both on-package and off-package memory, then ease of operation is improved, but device complexity increases due to memory management logic
Solution Approach 1:
The memory management logic acts as an intermediary between the processor and the heterogeneous memory system. It handles address translation, memory allocation, and access routing, shielding the processor from the complexity of managing multiple memory types. The processor sees a simple unified memory interface while the memory management logic manages the underlying complexity of coordinating HBM and DDR memory access.
Data Source
AI summary
A combined on-package and off-package memory system uses a custom base-layer within which are fabricated one or more dedicated interfaces to off-package memories. An on-package processor and on-package memories are also directly coupled to the custom base-layer. The custom base-layer includes memory management logic between the processor and memories (both off and on package) to steer requests. The memories are exposed as a combined memory space having greater bandwidth and capacity compared with either the off-package memories or the on-package memories alone. The memory management logic services requests while maintaining quality of service (QoS) to satisfy bandwidth requirements for each allocation. An allocation may include any combination of the on and/or off package memories. The memory management logic also manages data migration between the on and off package memories.