Hourglass Fat-Tree NoC for Uniform High-Bandwidth Memory Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current network topologies in System on Chip (SoC) designs are limited by low throughput, high latency, and unequal bandwidth distribution, failing to support the high performance capabilities of Digital Signal Processors (DSPs) and high bandwidth memory (HBM), particularly in connecting cache and system memory.
Innovation Solution
A network on chip design utilizing a combination of fat-tree topologies in an hourglass configuration, ensuring equal or greater bandwidth at target nodes compared to initiator nodes, with equidistant connectivity and uniform access to system memory, allowing for any-to-any connections and striding capabilities.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If current network topologies are used in SoC designs, then device complexity is reduced, but throughput is limited and cannot support high bandwidth capabilities
Solution Approach 1:
The network topology is segmented into two separate fat-tree structures: an initiator fat-tree connecting initiator nodes to initiator core switches, and a target fat-tree connecting target nodes to target core switches. This segmentation allows each fat-tree to be optimized independently for high throughput while managing complexity through modular design.
Solution Approach 2:
The patent introduces a new dimensional aspect by creating bidirectional any-to-any connectivity between initiator core switches and target core switches. This adds a new layer of connectivity dimension that enables high bandwidth communication patterns without increasing the physical footprint or basic topology complexity.
2Adaptability or versatility
If network topology is simplified, then device complexity is reduced, but any-to-any connectivity and equidistant connectivity are compromised
Solution Approach 1:
The fat-tree topology provides universal connectivity where any initiator node can communicate with any target node through any available path. The core switches are designed with multi-functionality to handle various communication patterns including any-to-any connectivity and equidistant connectivity to memory channels, making the network adaptable to different access patterns without requiring topology changes.
3Productivity
If high bandwidth memory and DSP capabilities are supported, then productivity is improved, but current network topologies cannot maintain low latency
Solution Approach 1:
The patent changes key network parameters by implementing a fat-tree topology with increased bandwidth capacity on critical paths. The network link widths and switch port capacities are adjusted to match the high bandwidth capabilities of HBM and DSPs, while the hierarchical structure maintains short path lengths to preserve low latency characteristics.
4Adaptability or versatility
If uniform access to system memory is implemented, then adaptability is improved, but network topology complexity increases
Solution Approach 1:
The patent achieves equipotentiality in the network by designing the fat-tree topology such that all initiator nodes have equal access capability to all target memory channels. Each initiator can reach any memory channel through any available path with the same number of hops and similar latency characteristics, creating a uniform access environment despite the complex underlying topology.
Data Source
AI summary
A network on chip for high performance computing includes a first fat-tree network topology that connects each of a number of initiator nodes to one of a number of initiator core switches through one or more levels of initiator edge switches. A second fat-tree network topology connects each of the target nodes to one of a number of target core switches through one or more levels of target edge switches, and the first fat-tree network topology is joined to the second fat-tree network topology through connections between the initiator core switches and the target core switches.