Optical HBM Links With Tiered Cooling for Package Constraints
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
High performance computing systems face limitations in integrating multiple high bandwidth memory (HBM) dies due to space, power, and thermal constraints, which restricts bandwidth and capacity, particularly in datacenter environments where components have different design requirements.
Innovation Solution
The implementation of an optical interface that connects HBM packages to processor packages via optical fibers, allowing for flexible cooling solutions and component placement without sacrificing high performance computing, enabling efficient datacenter scaling and upgrading.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If more HBM dies are integrated in a package to increase bandwidth and capacity, then memory performance is improved, but space constraints, power constraints, and thermal constraints are worsened
Solution Approach 1:
The patent transitions from horizontal integration of HBM dies within a single package to vertical stacking of HBM assemblies in multiple tiers. This dimensional change allows multiple high-bandwidth memory modules to be stacked vertically (2-tier, 3-tier, or 4-tier configurations) rather than constrained to a single planar package, effectively increasing memory capacity and bandwidth while managing space utilization in the vertical dimension.
Solution Approach 2:
The patent divides the memory system into separate modular HBM assemblies that can be independently stacked and cooled. Each HBM assembly is a discrete unit that can be combined with others to form larger memory systems. This segmentation allows flexible configuration of memory capacity and enables independent thermal management of each assembly through intermediate cooling components.
2Productivity
If more HBM dies are integrated in a package to increase bandwidth and capacity, then memory performance is improved, but power constraints are worsened
Solution Approach 1:
The patent divides the memory system into separate modular HBM assemblies with independent power and thermal management. Each assembly can be powered and cooled independently, allowing for scalable power distribution based on actual memory capacity and performance requirements. This modular approach enables more efficient power utilization compared to a monolithic package where all dies must be supported at full power.
Solution Approach 2:
The patent enables dynamic configuration of memory capacity and power consumption through selective activation of HBM assemblies. Systems can be configured with 2-tier, 3-tier, or 4-tier stacks depending on performance requirements, allowing power consumption to scale with actual memory bandwidth needs rather than providing maximum power capacity for all possible configurations.
3Productivity
If more HBM dies are integrated in a package to increase bandwidth and capacity, then memory performance is improved, but thermal constraints are worsened
Solution Approach 1:
The patent divides the memory system into separate HBM assemblies with intermediate cooling components between tiers. This segmentation creates multiple thermal zones that can be managed independently, preventing heat accumulation from propagating through the entire stack. The intermediate cooling components actively remove heat from each tier, maintaining lower operating temperatures across the system.
Solution Approach 2:
The patent introduces intermediate cooling components as mediators between HBM die tiers. These cooling components act as thermal barriers and active heat removal points between stacked memory modules, preventing thermal coupling between tiers and enabling higher density stacking without proportional increases in temperature. The intermediaries facilitate thermal management in high-density vertical configurations.
4Adaptability or versatility
If optical interface is used to connect HBM packages to processor packages, then flexibility in component placement is improved, but device complexity is worsened
Solution Approach 1:
The patent employs standardized optical interface modules that can be universally applied across different HBM assembly configurations (2-tier, 3-tier, 4-tier) and processor package types. This universal optical interface design allows the same connection technology to serve multiple functions: connecting different numbers of HBM tiers to processors, supporting various form factors, and enabling future scalability without requiring custom interface designs for each configuration.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This solution enhances memory bandwidth and flexibility in component placement, facilitating continuous datacenter upgrades to meet increasing computational demands while maintaining high performance computing capabilities, particularly beneficial for memory-intensive applications like large language models.
Implementation Method 1
an optical interface configured to connect the HBM assemblies to one or more optical fibers that form an optical link with one or more processor packages
Implementation Method 2
the one or more cooling components may include a cold-plate that is configured to contact the first set of components of the plurality of HBM assemblies
Data Source
AI summary
The technology generally relates to high bandwidth memory (HBM) packages and processor packages that have optical connectivity. Disclosed systems and methods herein allow for HBM dies that are interconnected with an optical interface in a manner that allows for compact, high-performance computing. An HBM package can be cooled using a cooling unit that is distinct from the processor package. In addition, the cooling unit can be configured so as to provide thermal contact with a subset of high-power components within the HBM package.