HBM+ Memory System Fabric Interconnect Scaling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current high bandwidth memory (HBM) systems are limited by the number of DRAM stacks and pin connections, which restricts memory and bandwidth capacity, leading to performance bottlenecks in applications like deep learning and high-performance computing.
Innovation Solution
The proposed HBM+ system consists of multiple HBM+ cubes with logic and memory dies, stacked three-dimensionally, and interconnected via buffer-based or peer-to-peer communication links, allowing for increased memory capacity and bandwidth through a scalable architecture that includes multiple cards linked via fabric connections, enabling parallel operations and efficient data processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If multiple GPUs are used to expand memory capacity and bandwidth, then memory capacity and bandwidth increase, but device complexity and system cost increase
Solution Approach 1:
The patent merges multiple HBM stacks into a unified HBM+ memory system with shared control logic and interconnect fabric, allowing multiple memory stacks to function as a single integrated resource rather than separate GPU memory units. This reduces system complexity while maintaining expanded memory capacity.
Solution Approach 2:
The HBM+ card controller and fabric interconnect provide universal functionality that serves multiple HBM stacks and multiple cards simultaneously, enabling a single infrastructure to support expanded memory capacity across the entire system rather than requiring dedicated control paths for each GPU.
2Quantity of substance
If more HBM stacks are added per card, then memory capacity increases, but pin connection limitations are reached
Solution Approach 1:
The fabric interconnect acts as an intermediary that enables communication between HBM stacks on different cards, allowing memory capacity to expand beyond the pin connection limits of a single card. The fabric provides additional communication pathways that bypass the PCIe pin bottleneck.
Solution Approach 2:
The patent transitions from a single-card two-dimensional memory expansion to a multi-card three-dimensional memory system, where the fabric interconnect adds a new dimension for memory addressing and data access that is not constrained by individual card pin limitations.
3Extent of automation
If data is moved between storage and processing, then computational functionality is provided, but bus bandwidth is consumed and congestion occurs
Solution Approach 1:
The patent segments the memory system into multiple independent HBM stacks that can perform compute operations locally within each stack, reducing the need for data movement across the PCIe bus. Each stack can process data independently before returning results, minimizing bus bandwidth consumption.
4Productivity
If HBM bandwidth is increased for parallel accelerators, then high performance computing is facilitated, but the number of cards is limited by pin connections
Solution Approach 1:
The fabric interconnect is pre-configured with routing capabilities and buffering mechanisms that enable high-bandwidth communication between cards before data transfer operations begin. This preliminary setup allows multiple cards to operate in parallel at high speeds without being constrained by pin connection limitations.
Data Source
AI summary
A high bandwidth memory (HBM) system includes a first HBM+ card. The first HBM+ card includes a plurality of HBM+ cubes. Each HBM+ cube has a logic die and a memory die. The first HBM+ card also includes a HBM+ card controller coupled to each of the plurality of HBM+ cubes and configured to interface with a host, a pin connection configured to connect to the host, and a fabric connection configured to connect to at least one HBM+ card.