Electro-Photonic Multi-Chip Interconnects for Memory Bandwidth
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current electronic processing systems face challenges with memory latency, bandwidth constraints, and power inefficiency in performing multiply-accumulate operations, particularly in AI computing tasks, due to limitations in signal integrity and complexity of packaging high bandwidth memory systems.
Innovation Solution
Implementing a hybrid electronic-photonic network-on-chip (NoC) with bidirectional photonic channels connecting circuit packages, utilizing photonic channels for data movement between processing elements and memory subsystems, and employing a novel clocking scheme and dot product engine to reduce power consumption and increase processing speed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If high bandwidth memory (HBM) is mounted on a silicon interposer with pins running over electrical wires at over 3 GHz, then memory bandwidth is improved, but signal integrity constraints and packaging complexity increase significantly
Solution Approach 1:
The patent replaces electrical signal transmission with optical signal transmission using photonic integrated circuits. Light replaces electrical wires as the transmission medium, eliminating the need for high-speed electrical interconnects and their associated signal integrity constraints. This substitution fundamentally changes the physical domain from electrical to optical, resolving the packaging complexity issue while maintaining high bandwidth.
Solution Approach 2:
The patent changes the operating frequency domain from electrical (3 GHz and above) to optical frequencies. By transmitting data as light signals rather than electrical signals, the system achieves much higher effective bandwidth without being constrained by electrical signal integrity limits. This parameter change in the transmission medium's fundamental property enables higher speeds without proportional increases in packaging complexity.
2Speed
If memory elements are placed close to client devices to improve memory latency, then access speed is improved, but the number and arrangement of HBM stacks are highly constrained
Solution Approach 1:
The patent uses optical transmission to replace electrical interconnects, enabling memory modules to be positioned at greater distances from processing elements without penalty. Optical signals have higher bandwidth and are less susceptible to signal degradation over distance, allowing flexible spatial arrangement of memory and processing components while maintaining high access speeds. This eliminates the strict placement constraints of electrical HBM systems.
3Productivity
If serializer/deserializer blocks are used to convert parallel messages into serial bit streams for inter-chip communication, then data transmission between chips is enabled, but significant energy is expended in moving data to and from SerDes
Solution Approach 1:
The patent replaces electrical serialization/deserialization with direct optical transmission. By encoding data directly onto light signals and transmitting optically between chips, the system eliminates the energy-intensive SerDes conversion process. Optical transmission maintains parallel data paths without requiring serial conversion, dramatically reducing the energy expended in data movement while preserving inter-chip communication capability.
4Productivity
If conventional electronic processing systems are used for AI computing, then processing capacity is increased, but power efficiency in performing multiply-accumulate operations deteriorates
Solution Approach 1:
The patent replaces electronic processing with photonic processing for AI computing operations. By performing multiply-accumulate operations using optical components such as modulators, waveguides, and photodetectors, the system achieves high processing capacity with significantly lower power consumption. Optical fields can perform parallel computations without the resistive losses inherent in electronic circuits, improving power efficiency while maintaining or enhancing processing capacity.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The solution achieves reduced power consumption and increased processing speed by minimizing data movement and energy losses, leveraging photonic channels for long-distance data transfer and optimizing MAC operations, particularly suitable for AI computing tasks like neural networks.
Implementation Method 1
a second circuit package includes an electronic integrated circuit comprising multiple processing elements that are connected by bidirectional photonic channels (e.g., implemented in a photonic integrated circuit in a separate layer or chip of the package)
Implementation Method 2
In the receive direction, bits arrive on an optical fiber or electrical interconnect
Data Source
AI summary
Multi-chip electro-photonic networks for interconnecting multiple circuit packages via photonic channels. Examples of a computing system include a first circuit package, a second circuit package, and one or more inter-chip bidirectional photonic channels interconnecting the first and second circuit packages. The first circuit package can include one or more memory nodes. The second circuit package can include multiple compute nodes and intra-chip bidirectional photonic channels interconnecting the compute nodes.