Monolithic In-Package Optical I/O for Pooled Memory Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current compute systems face limitations in accessing larger memory pools due to the limited capacity of in-package memory stacks, necessitating a new technology for high-bandwidth, low-latency memory access that exceeds the constraints of existing Peripheral Component Interconnect express (PCIe) buses or switches to DRAM.
Innovation Solution
Implementing Monolithic In-Package Optical Input and Output (MIPO) chiplets for optical data communication between semiconductor chips and external memory pools, enabling conversion of electrical data signals to optical signals for transmission and vice versa, thereby extending memory capacity and improving bandwidth-density, latency, and energy-cost.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If in-package memory stacks are used to provide high-bandwidth memory access, then bandwidth and latency are improved, but memory capacity is limited to just under 100 GB
Solution Approach 1:
The memory system is segmented into in-package HBM stacks for high-performance access and off-package DRAM pools for capacity extension. The electro-optical chip acts as an intermediary that segments the interconnect architecture into electrical and optical domains, allowing each domain to serve its optimal function.
Solution Approach 2:
An electro-optical chip is introduced as an intermediary component between the integrated circuit chip and off-package memory. This intermediary converts electrical signals to optical signals for transmission over optical links, enabling high-speed communication with off-package memory while maintaining the performance characteristics of in-package memory.
2Quantity of substance
If PCIe bus or switch is used to connect to off-package DRAM, then memory capacity is increased, but bandwidth-density and latency deteriorate
Solution Approach 1:
The patent replaces the electrical signal transmission mechanism (PCIe bus) with an optical signal transmission mechanism. The electro-optical chip converts electrical data signals to optical signals that travel through optical links to off-package memory, achieving both capacity extension and performance maintenance.
Solution Approach 2:
The transmission medium parameter is changed from electrical signals to optical signals. This parameter change enables the system to achieve the bandwidth-density and latency performance of in-package memory while accessing off-package memory capacity, as optical transmission has lower latency and higher bandwidth than electrical PCIe buses.
3Quantity of substance
If more HBM stacks and interfaces are integrated in-package, then memory capacity increases, but device complexity and package real-estate requirements increase
Solution Approach 1:
The patent extracts additional memory capacity from the in-package constraint by connecting to off-package DRAM pools through optical links. This removes the need to pack more HBM stacks into the limited in-package real-estate, simplifying the package architecture while increasing total memory capacity.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The MIPO I/O chiplets enable access to larger memory capacities at improved bandwidth-density, latency, and energy-cost, effectively overcoming the limitations of in-package memory constraints by leveraging optical interconnects for off-package memory access.
Implementation Method 1
Each of the at least one optical macro is configured to convert outgoing electrical data signals received through the electrical interface from the high-bandwidth interface into outgoing optical data signals
Implementation Method 2
Each of the at least one optical macro is configured to convert incoming optical data signals received through the photonic interface from the optical link into incoming electrical data signals
Data Source
AI summary
A computer memory system includes an electro-optical chip, an electrical fanout chip electrically connected to an electrical interface of the electro-optical chip, and at least one dual in-line memory module (DIMM) slot electrically connected to the electrical fanout chip. A photonic interface of the electro-optical chip is optically connected to an optical link. The electro-optical chip includes at least one optical macro that converts outgoing electrical data signals into outgoing optical data signals for transmission through the optical link. The optical macro also converts incoming optical data signals from the optical link into incoming electrical data signals and transmits the incoming electrical data signals to the electrical fanout chip. The electrical fanout chip directs bi-directional electrical data communication between the electro-optical chip and a dynamic random access memory (DRAM) DIMM corresponding to the at least one DIMM slot.