Systolic Memory Access via WDM Photonic Interconnects
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Modern processing units face challenges in orchestrating efficient data movement between processing elements and memory due to limitations in existing memory access technologies, particularly in supporting high-bandwidth and low-latency communication patterns, especially with photonic interconnects that do not support duplex communication over a single waveguide.
Innovation Solution
The implementation of systolic memory access using high-bandwidth, wavelength division multiplexing (WDM) based photonic interconnects, where data flows between memory sub-systems through a processor sub-system in predetermined directions for clock cycles, avoiding the need for duplex communication by using separate optical fibers for each direction, and utilizing optical interface circuits with buffering and microring resonators for efficient data transmission and reception.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Power
If photonic interconnects are used for high-bandwidth communication, then data transmission bandwidth is improved, but communication complexity increases due to lack of duplex support over single waveguide
Solution Approach 1:
The patent segments the communication channel by using separate optical fibers for transmit and receive directions, eliminating the need for duplex capability over a single waveguide. This segmentation simplifies the photonic interconnect architecture while maintaining high bandwidth performance.
Solution Approach 2:
The patent introduces an intermediary approach by using a memory sub-system as a buffer between processing elements, allowing data to be stored and accessed in predictable patterns. This mediator enables efficient data movement without requiring complex duplex communication protocols.
2Productivity
If traditional memory access patterns are used, then hardware simplicity is maintained, but data movement efficiency deteriorates for high-performance computing applications
Solution Approach 1:
The patent implements dynamic memory access patterns tailored to specific computational workloads, particularly for deep learning inference. The memory controller adapts access patterns based on the computational graph and data dependencies, optimizing data movement efficiency for each application scenario.
Solution Approach 2:
The patent ensures continuous useful action by maintaining a pipeline of data movements where data is continuously flowing between memory sub-systems and processing elements. The systolic array architecture enables overlapping compute and data access operations, eliminating idle time and maximizing hardware utilization.
3Reliability
If separate optical fibers are used for each communication direction, then communication reliability is improved, but hardware cost increases
Solution Approach 1:
The patent merges the communication infrastructure by using a shared optical fiber infrastructure combined with wavelength division multiplexing. This allows multiple data streams to share the same physical medium while maintaining reliable bidirectional communication through wavelength separation, reducing hardware costs compared to dedicated fibers for each direction.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach enables efficient data movement with minimal latency and hardware cost, suitable for applications with predictable read/write patterns, such as deep learning inference, by effectively utilizing the high bandwidth offered by WDM photonic interconnects and optimizing data flow through a compiler-managed static memory mapping scheme.
Implementation Method 1
high-bandwidth, wavelength division multiplexing (WDM) based photonic interconnects
Implementation Method 2
optical interface circuits with buffering and microring resonators for efficient data transmission and reception
Data Source
AI summary
Techniques to access memory in a systolic pattern. For example, a processor sub-system is connected between a first memory sub-system and a second memory sub-system. In response to a first clock signal, a communication direction of a first connection between the processor sub-system and the first memory sub-system is configured to receive first data in the processor sub-system from the first memory sub-system; and a communication direction of a second connection between the processor sub-system and the second memory sub-system is configured to transmit second data from the processor sub-system to the second memory sub-system. In response to a second clock signal, the communication direction of the first connection and the communication direction of the second connection are reversed.