Electro-Photonic Chip Networks for Memory Bandwidth and Power Limits
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current electronic processing systems face challenges with memory latency, bandwidth constraints, and power inefficiency, particularly in high bandwidth memory (HBM) systems, which require memory to be mounted close to the client device, imposing signal-integrity and thermal constraints, and limiting the integration of memory capacity.
Innovation Solution
Implementing a hybrid electronic-photonic network-on-chip (NoC) with bidirectional photonic channels connecting circuit packages, using photonic integrated circuits (PICs) for inter-chip communication, and leveraging photonic channels for data movement to reduce energy losses and increase processing speed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If memory is mounted close to the client device using HBM architecture, then memory bandwidth is improved, but signal integrity constraints and thermal constraints worsen
Solution Approach 1:
The patent introduces a silicon interposer as an intermediary component between the HBM memory stacks and the client device. The interposer provides a controlled impedance environment and proper signal routing, acting as a mediator that enables high-bandwidth memory connection while maintaining signal integrity. The interposer's specialized trace routing and grounding structures resolve the signal integrity constraints that would otherwise prevent close mounting of memory to the device.
Solution Approach 2:
The patent transitions from planar 2D packaging to 3D stacked architecture using HBM memory stacks mounted vertically on the interposer. This dimensional change allows memory to be positioned extremely close to the client device in the vertical dimension while the interposer manages the horizontal signal routing, thereby achieving high bandwidth without compromising signal integrity through proper spatial separation and routing.
2Loss of time
If HBM stacks are placed close to the client device, then memory latency is reduced, but thermal constraints worsen
Solution Approach 1:
The patent utilizes vertical stacking in the third dimension to place HBM memory stacks close to the client device, reducing latency through shorter data paths. The interposer provides thermal management structures and spacing in the horizontal plane, allowing the vertical proximity needed for low latency while managing heat dissipation through lateral thermal pathways and proper grounding.
Solution Approach 2:
The silicon interposer acts as a thermal intermediary, providing thermal vias and grounding structures that conduct heat away from the densely packed memory stacks. The interposer's specialized layout with ground planes and thermal pathways mediates between the heat-generating memory components and the device housing, managing thermal constraints while enabling close memory placement for low latency.
3Quantity of substance
If more HBM stacks are integrated into the system, then memory capacity increases, but packaging complexity worsens
Solution Approach 1:
The patent divides the memory system into separate HBM stacks that are independently manufactured and then integrated onto a single interposer. This segmentation allows each stack to be optimized independently and simplifies the packaging process, as the interposer provides a standardized interface for connecting multiple stacks. The modular approach enables increased memory capacity through additive stacking rather than complex monolithic integration.
Solution Approach 2:
The silicon interposer serves multiple functions simultaneously: it provides electrical interconnection between memory stacks and the client device, manages signal integrity through controlled impedance routing, provides thermal management pathways, and enables mechanical support for multiple stacks. This multi-functionality reduces overall packaging complexity by consolidating multiple subsystems into a single component.
4Speed
If SerDes blocks are used for chip-to-chip communication, then data transmission is enabled, but power consumption increases
Solution Approach 1:
The patent replaces the electrical SerDes transmission system with optical communication using photonic integrated circuits. Instead of converting parallel messages to serial bit streams through electrical SerDes blocks, the system uses optical channels to transmit data directly between chips. This substitution eliminates the power-intensive electrical-to-optical conversion processes while maintaining high data transmission speeds through parallel optical waveguide channels.
Solution Approach 2:
The patent introduces photonic integrated circuits as intermediary components that interface with the HBM memory stacks. These PICs provide optical transmission capabilities, acting as mediators that convert electrical signals from the memory to optical signals for low-power long-distance transmission, and then convert back to electrical signals at the destination. This intermediary approach enables high-speed communication with significantly reduced power consumption compared to direct electrical SerDes links.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The solution achieves reduced power consumption and increased processing speed by minimizing data movement within and between circuit packages, while maintaining high bandwidth and reducing latency, particularly suitable for artificial intelligence computing tasks.
Implementation Method 1
The first and second circuit packages can be connected to each other by one or more inter-chip bidirectional photonic channels, e.g., implemented with optical fiber
Data Source
AI summary
Multi-chip electro-photonic networks for interconnecting multiple circuit packages via photonic channels. Examples of a computing system include a first circuit package, a second circuit package, and one or more inter-chip bidirectional photonic channels interconnecting the first and second circuit packages. The first circuit package can include one or more memory nodes. The second circuit package can include multiple compute nodes and intra-chip bidirectional photonic channels interconnecting the compute nodes.