Stacked Processor-Memory Rings for Parallel ML Communication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current stacked device systems face limitations in efficiently interconnecting multiple processing units and memory devices for parallel machine learning tasks, particularly in implementing distributed and parallel algorithms like Ring-AllReduce, due to constraints in data and control signal communication through shared through-silicon via connections.
Innovation Solution
The system employs a ring topology interconnection configuration using through-silicon via connections between DRAM and processor dies, enabling efficient communication and parallel processing by configuring separate ring topologies for different machine learning tasks, allowing concurrent training of neural network models across device stacks using a handshake protocol.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Volume of moving object
If shared through-silicon via connections are used for interconnecting device stacks, then vertical integration and compact form factor are achieved, but communication efficiency and scalability for parallel processing are limited
Solution Approach 1:
The system segments the monolithic interconnection into multiple independent ring topologies, where each ring can operate autonomously for different machine learning tasks. This allows parallel processing across device stacks without contention on shared via connections, resolving the contradiction between compact vertical integration and parallel processing efficiency.
Solution Approach 2:
The patent transitions from purely vertical through-silicon via connections to a hybrid architecture that incorporates horizontal ring topology interconnections. This dimensional change enables data to flow laterally across device stacks, bypassing the bottleneck of shared vertical vias and enabling scalable parallel processing while maintaining the compact stacked form factor.
2Productivity
If multiple device stacks are interconnected for distributed machine learning, then processing capability is improved, but system complexity and interconnection management become problematic
Solution Approach 1:
The ring topology interconnection is designed as a universal communication fabric that can handle multiple machine learning workloads simultaneously. Each device stack can participate in multiple rings and perform different functions (training, inference, data movement) through the same physical infrastructure, reducing interconnection management complexity while scaling processing capability.
Solution Approach 2:
The system dynamically configures ring topologies and assigns device stacks to different rings based on workload requirements. This dynamic reconfiguration capability allows the system to adapt to varying processing demands without requiring complex static interconnection designs, thereby improving processing capability while managing complexity through software-controlled flexibility.
3Productivity
If concurrent training of multiple neural network models is implemented, then system utilization is improved, but communication protocol complexity increases
Solution Approach 1:
The patent introduces a protocol abstraction layer that mediates communication between device stacks performing concurrent neural network training. This intermediary protocol manages data flow, synchronization, and coordination across multiple concurrent training tasks, enabling high system utilization while shielding the underlying complexity of simultaneous multi-model training communications.
Data Source
AI summary
Multiple device stacks are interconnected in a ring topology. The inter-device stack communication may utilize a handshake protocol. This ring topology may include the host so that the host may initialize and load the device stacks with data and/or commands (e.g., software, algorithms, etc.). The inter-device stack interconnections may also be configured to include/remove the host and/or to implement varying numbers of separate ring topologies. By configuring the system with more than one ring topology, and assigning different problems to different rings, multiple, possibly unrelated, machine learning tasks may be performed in parallel by the device stack system.