Multi-HBM Cube Interconnect for Higher Bandwidth in Small Footprints
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Semiconductor devices face challenges in increasing storage capacity and bandwidth while maintaining a small footprint, as methods like DRAM cell scaling or increasing core die size are hindered by cost, power, thermal, and speed concerns, leading to difficulties in expanding HBM stack capacity and bandwidth without negatively affecting performance.
Innovation Solution
Connecting multiple high bandwidth memory (HBM) cubes to a graphics processing unit (GPU) either in series or parallel via IO buses, allowing the GPU to communicate with primary and satellite cubes, thereby increasing memory capacity and bandwidth without altering the physical dimensions of the HBM cubes, using physical and secondary communication circuits to manage addresses and data transfer.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If DRAM cell scaling or increasing core die size is used to increase storage capacity and bandwidth, then memory capacity and bandwidth are improved, but thermal footprint increases and performance deteriorates
Solution Approach 1:
The memory system is divided into multiple independent HBM cubes, each with its own memory array and I/O circuitry. These cubes are connected in a daisy-chain configuration where each cube (except the last) connects to the next via dedicated inter-cube interfaces. This segmentation allows the system to achieve higher total capacity through parallel operation of multiple smaller units rather than one large unit, thereby distributing thermal load across multiple discrete heat sources rather than concentrating it in a single large memory array.
2Productivity
If HBM stack capacity is increased to expand memory bandwidth, then bandwidth is improved, but device footprint increases
Solution Approach 1:
Instead of expanding memory capacity in the vertical dimension (increasing stack height), the patent transitions to a horizontal expansion model by connecting multiple HBM cubes in a daisy-chain configuration across the substrate. Each cube maintains a compact footprint, and the overall system capacity scales by adding more cubes laterally rather than vertically. This dimensional shift from vertical stacking to horizontal chaining preserves the small footprint of individual cubes while achieving system-level capacity and bandwidth expansion through parallel access to multiple cubes.
3Quantity of substance
If multiple HBM cubes are connected in series or parallel, then memory capacity and bandwidth are increased, but system complexity increases
Solution Approach 1:
Each HBM cube is designed with universal, identical interface circuitry that can function both as a master interface (connecting to the GPU) and a slave interface (connecting to the next cube in the chain). The same basic I/O block structure is replicated across all cubes, with each cube capable of initiating and completing transactions independently. This universality simplifies the overall system architecture because the same design template is reused throughout, reducing the need for specialized control logic for each cube and making the system more manageable despite having multiple units.
Data Source
AI summary
An apparatus including a high bandwidth memory circuit and associated systems and methods are disclosed herein. The apparatus may include multiple HBM cubes connected to a processor, such as a GPU. The HBM cubes may be connected in series or in parallel. One or more of the HBM cubes can include a secondary communication circuit configured to facilitate the expanded connection between the multiple cubes.