Chiplet CPU-Accelerator SoCs With Unified Memory Tunneling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Modern server systems face limitations with Peripheral Component Interconnect Express (PCIe) connections, such as limited shared address space between CPUs and accelerators, high bandwidth and low latency requirements, and restricted port and lane counts that hinder efficient CPU-accelerator balancing.
Innovation Solution
Implementing a system-on-chip (SoC) with a central processing unit (CPU) and accelerators connected via die-to-die interconnects, utilizing a uniform memory access tunneling system to allow accelerators direct access to shared memory through a high-level interconnect protocol tunneled into a die-to-die protocol, eliminating the need for accelerator memory during processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If PCIe is used to connect accelerators to CPU, then accelerators can be attached to CPU, but shared address space between CPU and accelerators is not allowed and latency increases
Solution Approach 1:
The system divides the interconnect architecture into multiple die-to-die interconnects, each serving specific functional blocks. This segmentation allows the CPU and accelerators to have direct access to shared memory through dedicated interconnect paths, eliminating PCIe's indirect access mechanism and enabling shared address space while reducing latency.
Solution Approach 2:
The patent introduces a memory controller as an intermediary component that mediates between the CPU and accelerators. This memory controller manages the shared address space and provides unified memory access, allowing both CPU and accelerators to access memory simultaneously without PCIe's protocol overhead.
2Device complexity
If limited number of PCIe ports and lanes are used, then system complexity is reduced, but number of accelerators that can be attached is limited and CPU-accelerator ratio balancing becomes difficult
Solution Approach 1:
The system segments the interconnect architecture into multiple die-to-die interfaces, each capable of supporting multiple accelerators. This allows the system to attach more accelerators without increasing the number of PCIe ports, as each die-to-die interconnect can serve multiple accelerator functions through protocol tunneling.
Solution Approach 2:
The die-to-die interconnect protocol is designed to be universal and can be tunneled through different protocols (PCIe, CXL). This multi-functionality allows a single interconnect to serve multiple purposes and support various accelerator types, increasing the number of attachable accelerators without proportionally increasing system complexity.
3Quantity of substance
If external I/O connections like PCIe are used, then accelerators can be connected, but aggregate bandwidth is limited and latency increases
Solution Approach 1:
The patent transitions from external I/O connections to on-chip die-to-die interconnects, effectively moving the communication path from a higher dimensional external interface to a lower dimensional internal connection. This dimensional change enables higher bandwidth and lower latency by eliminating the external interface overhead and protocol translation layers.
Data Source
AI summary
In some embodiments, a system-on-chip, includes a central processing unit (CPU); an accelerator coupled to the CPU via a first die-to-die interconnect; and uniform memory coupled to the CPU via a second die-to-die interconnect. In some embodiments, in order to prevent use of accelerator memory for processing operations by the accelerator, the accelerator utilizes a uniform memory access tunneling system located in the accelerator to tunnel a high-level interconnect protocol associated with the second die-to-die interconnect to a die-to-die interconnect protocol associated with the first die-to-die interconnect, the uniform memory access tunneling system being configured to allow access to the uniform memory using a shared address space.