HBM Channel Allocation for Flexible Memory and Bandwidth Sharing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current systems and methods for using high-bandwidth memories (HBMs) are inefficient in allocating memory capacity and bandwidth between multiple workloads, often leading to overprovisioning or underprovisioning based on varying workload requirements.
Innovation Solution
A system and method for configuring high-bandwidth memories with selectively allocatable memory capacity and bandwidth, utilizing a plurality of data channels and shareable memory, controlled by a base die, to assign workloads to channels based on their attributes, allowing independent allocation of memory and bandwidth.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If memory capacity and bandwidth are provisioned for multiple workloads using current HBM configurations, then workloads can be supported, but overprovisioning occurs leading to resource waste and increased cost
Solution Approach 1:
The HBM die is segmented into multiple independent memory channels (e.g., 8 channels), each capable of being independently configured and allocated to different workloads. This segmentation allows precise control over memory capacity and bandwidth allocation per workload, eliminating the need to overprovision entire HBM stacks for peak demands of individual workloads.
Solution Approach 2:
The system implements dynamic allocation of memory channels to workloads based on real-time workload requirements. The base die and HBM die can reconfigure channel assignments and memory capacity allocations as workloads change, allowing the system to adapt memory resource distribution dynamically rather than being fixed at manufacturing.
2Ease of manufacture
If fixed memory capacity is allocated to each data channel, then allocation is simple, but efficiency decreases when workload requirements vary significantly
Solution Approach 1:
Memory capacity allocation is made dynamic through the ability to selectively enable or disable portions of shareable memory for each data channel based on workload needs. The system can adjust the amount of memory capacity assigned to each channel at runtime, balancing manufacturing simplicity with operational efficiency.
Solution Approach 2:
The system changes the parameter of memory capacity allocation from fixed to variable by introducing shareable memory regions that can be dynamically assigned to different data channels. This allows the same physical memory to serve different workloads with different capacity requirements without requiring complex manufacturing variations.
3Productivity
If bandwidth is allocated independently of memory capacity, then bandwidth can be optimized for each workload, but control complexity increases
Solution Approach 1:
The independent bandwidth allocation is achieved by segmenting the memory interface into separate bandwidth control mechanisms for each data channel. The base die contains independent bandwidth controllers that can adjust bandwidth parameters for each channel without affecting memory capacity allocation, managing complexity through modular control architecture.
Solution Approach 2:
The base die acts as an intermediary between the HBM die and the workloads, mediating both memory capacity and bandwidth allocations. This intermediary layer abstracts the complexity of independent bandwidth control from the HBM die itself, allowing bandwidth optimization while keeping the overall system manageable through centralized control logic.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The technology is directed to systems and methods of high-bandwidth memory allocation. High-bandwidth memory may include a plurality of data channels and shareable memory that can be selectively allocated to particular data channels. In addition, bandwidth may be selectively allocated to the data channels independent of the shareable memory. The allocation of memory and bandwidth to particular data channels may be based on identified attributes of workloads that are to be associated with each data channel.