Embedded Rings in Networked Processors for Allreduce Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current computer networks used for machine learning applications face inefficiencies in data exchange between processing nodes, leading to underutilization of communication links and increased power consumption due to idle active links, particularly in broadcast and Allreduce collectives.
Innovation Solution
A network topology with interconnected processing nodes arranged in specific configurations, such as ladder and multi-face prism structures, enables efficient implementation of Allreduce collectives by utilizing bidirectional links to form embedded rings, allowing simultaneous data transmission and reducing link idle periods.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional broadcast and Allreduce collectives are implemented in current computer networks, then data exchange between processing nodes is achieved, but link utilization is poor and power consumption increases due to idle active links
Solution Approach 1:
The patent implements dynamic link configuration where processing nodes can be dynamically connected or disconnected based on operational requirements. During Allreduce operations, nodes are connected in a ring topology to maximize link utilization. During other operations, nodes can be disconnected to minimize power consumption, thus adapting the network topology to current workloads dynamically
Solution Approach 2:
The system employs periodic Allreduce operations synchronized with computational workloads. Links are activated only during these periodic Allreduce operations rather than remaining continuously active, enabling the system to achieve necessary data exchange when needed while minimizing power consumption during idle periods
2Productivity
If processing nodes are connected in traditional network topologies, then data exchange is enabled, but link idle periods increase reducing overall efficiency
Solution Approach 1:
The system performs preliminary actions by pre-establishing ring topologies and preparing communication paths before Allreduce operations begin. Processing nodes are pre-configured in optimal positions within the ring structure, and data is buffered and prepared in advance, allowing continuous data exchange during operations without idle periods
Solution Approach 2:
The patent ensures continuous useful action by implementing pipelined communication where data exchange operations overlap with computational operations. Multiple data streams are transmitted simultaneously in different directions around the ring, and buffer memory is used to maintain continuous data flow, eliminating idle periods and maximizing link utilization throughout the Allreduce operation
Data Source
AI summary
A network comprising interconnected first and second processors, each processor comprising one or more of: multiple processing units arranged on a chip configured to execute program code; an on-chip interconnect comprising groups of exchange paths connected to receive data from corresponding groups of the processing units; external interfaces configured to communicate data off-chip as packets, each having a destination address, external interfaces of the first and second processors being connected by an external link; multiple exchange blocks, each connected to groups of the exchange paths; a routing bus configured to route packets between the exchange blocks and the external interfaces. Processing units of the first processor generate off-chip packets such that the group of processing units serviced by the first exchange block on the first processor address off-chip packets to the group of processing units on the second processor serviced by the corresponding first exchange block of the second processor.