Shared HBM-LPDDR Memory Mapping for Multi-Processor Bandwidth
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing memory solutions, such as DRAM, are costly in terms of power and money while offering lower capacity and bandwidth, necessitating a more efficient and cost-effective approach for memory capacity and bandwidth expansion in computing devices with compute capability.
Innovation Solution
A computing device with two or more independent compute units, including GPUs and PIMs, is integrated with vertically stacked HBM and LPDDR memory, allowing for flexible usage modes and shared memory allocation across these units to enhance memory capacity and bandwidth.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If existing memory solutions such as DRAM are used, then memory capacity is provided, but power consumption is high and cost is high while bandwidth is limited
Solution Approach 1:
The memory system is segmented into multiple independent memory devices (e.g., HBM stack and LPDDR memory) that can be independently accessed by different processing devices. This segmentation allows selective activation of memory resources based on computational needs, reducing overall power consumption while providing expanded memory capacity.
Solution Approach 2:
The patent introduces a multi-dimensional memory address space where logical memory addresses are mapped to physical memory locations across vertically stacked HBM memory devices and LPDDR memory. This dimensional expansion enables efficient memory capacity scaling without proportionally increasing power consumption, as not all memory layers need to be actively powered simultaneously.
2Quantity of substance
If existing memory solutions such as DRAM are used, then memory capacity is provided, but bandwidth is limited and cost is high
Solution Approach 1:
The memory system provides universal access capabilities where multiple processing devices (GPUs and PIMs) can access the same physical memory space through different logical memory spaces. This multi-functional architecture enables bandwidth expansion without requiring separate dedicated memory for each processing unit, thus avoiding the cost and complexity of dual GPU server configurations.
Solution Approach 2:
The patent introduces a memory management intermediary that handles address translation and memory allocation between processing devices and physical memory. This intermediary enables efficient bandwidth utilization by intelligently routing memory access requests across HBM and LPDDR memory devices, achieving high throughput without the need for expensive dedicated high-bandwidth memory for each processing unit.
3Productivity
If multiple independent processing devices are used to increase compute capability, then processing power is improved, but memory cost and complexity increase
Solution Approach 1:
The patent merges the memory spaces of multiple processing devices into a unified physical memory infrastructure. By combining HBM memory devices with LPDDR memory and creating a shared address space, the system reduces memory allocation complexity compared to maintaining separate memory systems for each processing device, as seen in dual GPU server configurations.
Solution Approach 2:
The memory allocation system is designed to be dynamic, allowing the memory management intermediary to adaptively allocate and map logical memory spaces to physical memory locations based on runtime requirements. This dynamic approach simplifies memory management for multiple processing devices compared to static memory allocation schemes that would require complex pre-configuration.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Systems and methods for extended memory are disclosed. An apparatus (100) for extended memory may include a first processing device (102); a second processing device (200); a first memory device (106, 108); and a second memory device (104). A first logical memory space and a second logical memory space are allocated to respectively the first processing device (102) and the second processing device (200). The first logical memory space and the second logical memory space are further mapped to a first physical memory space of one of the first memory device (106, 108) and the second memory device (104).