Optical HBM Disaggregation for ASIC Package Space and Thermal Limits

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

High performance computing requires multiple high bandwidth memory (HBM) dies, but integration in ASIC packages is limited by space, power, and thermal constraints, leading to costly and wasteful replacement of entire packages due to faulty HBM dies.

Innovation Solution

Disaggregating HBM dies from ASIC packages by optically connecting them through HBM optics module packages, which include HBM chiplets and optical chiplets, allowing for increased flexibility and reduced repair costs by enabling additional HBM dies to be connected without size or thermal constraints.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If HBM dies are integrated within ASIC packages, then bandwidth and capacity are improved, but space constraints, power constraints, and thermal constraints of the ASIC package limit the number of HBM dies that can be integrated

Engineering Contradiction:

ImprovebandwidthVSAvoidASIC package footprint

Core Design Contradiction:

ProductivityVSArea of stationary object

Solution Approach 1:

The system divides the HBM memory system into separate modular components (HBM dies, HBM chiplets, optical chiplets) that can be independently packaged and connected to the ASIC package via optical interfaces, allowing the memory subsystem to be segmented from the processing unit

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent transitions from traditional electrical interconnects to optical interconnects, adding a new dimension of communication capability that enables extended reach and reduced constraints on HBM die integration by using light instead of electrical signals for data transmission

Inventive Principle:

Principle #17Another dimension (Dimensionality change)

2Productivity

If additional HBM dies are integrated in an ASIC package, then bandwidth and capacity increase, but power constraints and thermal constraints of the ASIC package are exceeded

Engineering Contradiction:

ImprovebandwidthVSAvoidASIC package power consumption

Core Design Contradiction:

ProductivityVSUse of energy by stationary object

Solution Approach 1:

By segmenting the HBM dies into separate packages connected via optical interfaces, each HBM package can be independently powered and thermally managed, distributing the power and thermal load away from the ASIC package

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

Optical chiplets serve as intermediary components between HBM dies and the ASIC package, enabling data transmission without requiring direct electrical connection, thereby reducing power consumption and thermal generation within the ASIC package boundaries

Inventive Principle:

Principle #24Intermediary (Mediator)

3Productivity

If HBM dies are integrated within ASIC packages, then performance is improved, but a defect in any HBM die makes the entire ASIC package inoperable requiring costly replacement

Engineering Contradiction:

ImproveperformanceVSAvoidrepair cost

Core Design Contradiction:

ProductivityVSEase of repair

Solution Approach 1:

The system architecture segments the HBM memory subsystem into independently replaceable modules, allowing faulty HBM dies or chiplets to be replaced without affecting the ASIC package or other HBM components, thereby reducing repair costs and improving maintainability

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The modular design enables selective replacement of only the defective HBM component while retaining and recovering the functional ASIC package and other HBM dies, maximizing resource utilization and minimizing waste

Inventive Principle:

Principle #34Discarding and recovering

Applied Scientific Principles

This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.

Function Achieved in This Case

This solution increases the number of HBM dies that can be connected to ASIC packages, reduces repair costs by allowing only faulty components to be replaced, and maintains performance by leveraging the longer channel reach of optical links.

Implementation Method 1

Optical chiplets may be configured to optically connect the HBM optics module package to the ASIC package

Methodology Applied

Scientific EffectOptical communication: Optical Fibre

Data Source

PatentUS20230343768A1Optical Communication for Memory Disaggregation in High Performance Computing

Publication Date: 2023.10.26 GOOGLE LLC

AI summary

The technology generally relates to disaggregating memory from an application specific integrated circuit (“ASIC”) package. For example, a high-bandwidth memory (“HBM”) optics module package may be connected to an ASIC package via one or more optical links. The HBM optics module package may include HBM dies(s), HBM chiplet(s) and an optical chiplet. The optical chiplet may be configured to connect the HBM optics module to one or more optical fibers that form an optical link with one or more other components of the ASIC package. By including an optical chiplet in the HBM optics module package, the HBM optics module package may be disaggregated from an ASIC package.