Layered Interleaving Network for High-Bandwidth Memory Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The increasing bandwidth requirements for memory access in AI-specific chips, driven by improved GPU hashrate, pose challenges for system-on-chip (SoC) design due to limitations in frequency and bus width, particularly with multi-channel and interleaving memory technologies, which complicate layout, network-on-chip performance, and access module integration.
Innovation Solution
A data transmission apparatus employing a multi-channel and layered interleaving network with adjustable sub-networks, where each layer includes at least one interleaving sub-network, configured to provide a total bandwidth greater than or equal to the sum of individual ports, allowing flexible distribution and reduced hardware overhead, enabling efficient data transmission and scalable memory access.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If multi-channel and interleaving memory technology is used to increase bandwidth, then memory access bandwidth is improved, but layout complexity and physical implementation difficulty increase significantly
Solution Approach 1:
The patent divides the interleaving network into multiple layers, where each layer contains a subset of channels. This segmentation allows channels to be distributed across different physical locations (both sides of SoC) while maintaining logical interleaving functionality, thus reducing layout complexity while preserving high bandwidth capability
Solution Approach 2:
The patent introduces a layered dimension to the traditional flat interleaving structure. By organizing channels into multiple layers with inter-layer coupling, the system achieves high bandwidth without requiring all channels to be placed in a single plane, thereby solving the layout distribution problem
2Productivity
If more than 8 channels are used to meet bandwidth requirements, then total bandwidth is improved, but network on chip performance and system stability deteriorate
Solution Approach 1:
The patent segments the large-scale interleaving network into multiple smaller sub-networks organized in layers. Each layer handles a manageable subset of channels, preventing any single network segment from becoming too complex. This segmentation maintains NOC performance while achieving high total bandwidth through the combined capacity of all layers
Solution Approach 2:
The patent implements dynamic channel allocation and routing within the layered structure, allowing the system to adapt traffic patterns and load distribution across different layers and channels. This dynamic management prevents network congestion and maintains stability even with more than 8 channels
3Productivity
If frequency and bus width are increased to meet bandwidth requirements, then data transmission speed is improved, but process limitations and timing constraints prevent further increases
Solution Approach 1:
Instead of continuing to increase frequency and bus width in the traditional single-dimension approach, the patent transitions to a multi-dimensional solution by introducing layered interleaving. This allows bandwidth expansion through additional spatial dimensions (layers) rather than pushing the limits of frequency and bit width, avoiding process and timing constraints
Data Source
Figure 1
Figure 2~3
Figure 4~5
AI summary
According to embodiments of the present disclosure, there is provided a data transmission apparatus. The data transmission apparatus includes a plurality of first ports, a plurality of second ports, and a plurality of data channels. The plurality of first ports are coupled to a processing unit. The plurality of second ports are coupled to a plurality of memories. The plurality of data channels are disposed among the first ports and the second ports to form an interleaving network having a plurality of layers, and configured to transmit data among the processing unit and the plurality of memories, such that each layer in the interleaving network includes at least one interleaving subnetwork.