HBM Channel Allocation With Shareable Memory for Mixed Workloads
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current systems and methods for using high-bandwidth memories (HBMs) are inefficient in allocating memory capacity and bandwidth between multiple workloads, often leading to overprovisioning or underprovisioning based on varying workload requirements.
Innovation Solution
A system and method for configuring high-bandwidth memories with selectively allocatable memory capacity and bandwidth, utilizing a plurality of data channels, shareable memory, and through-silicon vias (TSVs) to dynamically assign resources based on workload attributes, allowing independent allocation of memory and bandwidth to specific data channels.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If memory capacity and bandwidth are allocated to multiple workloads using fixed HBM configuration, then workloads can be supported simultaneously, but overprovisioning occurs leading to resource waste and increased cost
Solution Approach 1:
The patent implements dynamic memory allocation by allowing shareable memory regions to be selectively assigned to different data channels based on workload requirements. The base die controls the allocation of shareable memory to data channels, enabling the system to adapt memory capacity and bandwidth provisioning to actual workload needs rather than using fixed allocations, thereby eliminating overprovisioning and improving resource efficiency
Solution Approach 2:
The HBM is divided into multiple data channels with associated memory cells and shareable memory regions. Each data channel can be independently configured and allocated to different workloads, allowing granular control over memory resource distribution. This segmentation enables precise matching of memory capacity and bandwidth to individual workload requirements
2Productivity
If fixed HBM configuration is used for multiple workloads, then system simplicity is maintained, but inefficient resource utilization occurs
Solution Approach 1:
A base die is introduced as an intermediary component that controls the allocation of shareable memory to data channels. The base die receives allocation requests and configures the memory resources accordingly, providing a centralized control mechanism that enables efficient resource utilization without requiring complex reconfiguration of the entire HBM system. This intermediary layer simplifies the allocation control while improving productivity
3Adaptability or versatility
If HBM overprovisions memory capacity for workloads with varying requirements, then all workload types can be supported, but the number of HBM dies increases leading to higher cost
Solution Approach 1:
Shareable memory regions are designed to serve multiple data channels and different workload types. The same physical memory cells can be allocated to different data channels based on which workload requires them at any given time, making the memory resources universal rather than dedicated. This multi-functionality reduces the total memory capacity needed across all HBM dies while maintaining support for diverse workload requirements
Data Source
AI summary
The technology is directed to systems and methods of high-bandwidth memory allocation. High-bandwidth memory may include a plurality of data channels and shareable memory that can be selectively allocated to particular data channels. In addition, bandwidth may be selectively allocated to the data channels independent of the shareable memory. The allocation of memory and bandwidth to particular data channels may be based on identified attributes of workloads that are to be associated with each data channel.