Stacked Processor-DRAM Ring Topology for Parallel ML Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current stacked device systems face limitations in efficiently interconnecting multiple processing units and memory devices for parallel machine learning tasks, particularly in implementing distributed and parallel algorithms like Ring-AllReduce, due to constraints in data and control signal communication through shared through-silicon via connections.
Innovation Solution
The system employs a ring topology interconnection configuration using through-silicon via (TSV) connections between DRAM and processor dies, enabling efficient data and control signal communication through a handshake protocol, allowing multiple device stacks to operate in parallel on different neural network models or compute problems by configuring separate ring topologies and using interconnect selectors to include or exclude the host.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If multiple device stacks are interconnected using shared through-silicon via connections, then data and control signal communication is enabled, but communication efficiency and scalability are limited
Solution Approach 1:
The system segments the interconnection architecture into multiple independent ring topologies, where each ring can operate separately. This allows different device stacks to be organized into distinct communication groups, enabling parallel operations without interference. The segmentation of shared TSV connections into dedicated ring paths resolves the bottleneck of shared bandwidth while maintaining scalability.
Solution Approach 2:
The patent transitions from traditional two-dimensional planar interconnections to three-dimensional stacked architectures with vertical TSV connections. By adding the vertical dimension, the system achieves higher density and enables multiple rings to operate simultaneously at different vertical levels, dramatically improving communication efficiency without increasing lateral complexity.
2Productivity
If device stacks operate in parallel on different neural network models, then processing efficiency improves, but coordination and data transfer between stacks become more complex
Solution Approach 1:
The host device serves as an intermediary that coordinates operations across multiple device stacks. It manages the distribution of neural network models, collects results, and orchestrates the parallel training processes. This centralized coordination simplifies the complexity of inter-stack communication while maintaining high processing efficiency through parallel operations.
Solution Approach 2:
The system performs preliminary actions by pre-configuring ring topologies and pre-distributing neural network models to appropriate device stacks before parallel processing begins. This preparation phase establishes the communication pathways and data distribution patterns needed for efficient parallel execution, reducing coordination overhead during actual processing.
3Adaptability or versatility
If separate ring topologies are configured for different compute problems, then parallel processing capability increases, but system configuration complexity increases
Solution Approach 1:
The ring topology configuration is made dynamic and reconfigurable, allowing the system to adapt its interconnection patterns based on the specific compute problems being solved. Interconnect selectors enable runtime reconfiguration of ring topologies, so the system can optimize its architecture for different workloads without permanent hardware changes, balancing versatility with manageable complexity.
Data Source
AI summary
Multiple device stacks are interconnected in a ring topology. The inter-device stack communication may utilize a handshake protocol. This ring topology may include the host so that the host may initialize and load the device stacks with data and/or commands (e.g., software, algorithms, etc.). The inter-device stack interconnections may also be configured to include/remove the host and/or to implement varying numbers of separate ring topologies. By configuring the system with more than one ring topology, and assigning different problems to different rings, multiple, possibly unrelated, machine learning tasks may be performed in parallel by the device stack system.