Collective Communication Routing With Bit Strings to Avoid Path Contention
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing information processing systems face challenges in flexibly adjusting the number of servers participating in collective communication, leading to path contention issues in communication networks, especially in full mesh systems where all-reduce communication is performed.
Innovation Solution
The system employs a configuration where information processing devices connected to switches store and use bit strings to determine communication destinations, allowing selective participation of servers in collective communication, avoiding path contention through the generation of bit string sets using Galois fields or greedy algorithms, and generating communication tables to facilitate efficient data sharing among participating servers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If all servers participate in collective communication in a full mesh system, then communication coverage is maximized, but path contention increases and efficiency decreases
Solution Approach 1:
The system segments the set of all servers into multiple groups, where only servers within the same group participate in collective communication. This segmentation reduces path contention by limiting the number of simultaneous communication paths while maintaining adaptability through configurable group memberships.
Solution Approach 2:
The system dynamically adjusts server participation in collective communication based on group assignments. Servers can be flexibly assigned to different groups to optimize communication patterns, allowing the system to adapt to varying workload requirements while controlling path contention.
2Productivity
If the network topology is changed to reduce path contention, then communication efficiency improves, but network complexity increases
Solution Approach 1:
The system introduces a logical grouping dimension overlaying the physical network topology. By organizing servers into groups with specific communication rules, the system achieves efficient communication patterns without modifying the underlying physical network structure, thus avoiding increased hardware complexity.
3Ease of manufacture
If bit string selection is random, then implementation simplicity is maintained, but linear independence cannot be guaranteed
Solution Approach 1:
The system pre-generates sets of bit strings that are guaranteed to have linear independence properties. These pre-validated bit string sets are stored and reused, eliminating the need for complex runtime validation while ensuring reliability of the communication destination determination.
Data Source
AI summary
An information processing device coupled to a first switch among a plurality of switches and included in a plurality of information processing devices includes: a memory; and a processor coupled to the memory and configured to: store, in the memory, communication destination information based on a plurality of bit strings related to communication destinations of collective communication; and communicate with an information processing device connected to a second switch among the plurality of switches on a basis of the communication destination information. Some information processing devices that include the information processing device among the plurality of information processing devices participate in the collective communication, and the plurality of bit strings is selected from a bit string set related to the communication destinations of the plurality of information processing devices on a basis of the number of the some information processing devices.