Multi-threaded Communication for Infiniband Clusters
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In large-scale deep learning training tasks, communication between devices is a bottleneck that slows down training speed, despite the use of Infiniband architecture for acceleration, as existing methods do not effectively manage thread states and message processing efficiently.
Innovation Solution
A communication method and system that utilize multi-threading by creating separate threads for each target device, including a communication thread, message sending thread, and message receiving thread, with a task pool for caching messages and state identification bits to manage thread states and asynchronous operations, leveraging the Infiniband interface for efficient data transmission.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If multi-threading is implemented with separate communication threads for each target device, then data transmission speed and bandwidth utilization are improved, but device complexity and thread management overhead increase
Solution Approach 1:
The system divides communication operations into separate threads for each target device, with dedicated communication threads, message sending threads, and message receiving threads. This segmentation allows parallel processing of multiple communication operations simultaneously, improving data transmission speed while organizing complexity through structured thread division
Solution Approach 2:
A task pool is introduced as an intermediary data structure to manage messages between threads. The task pool caches messages that need to be processed, allowing threads to efficiently retrieve and handle messages without direct complex inter-thread coordination, thus reducing management overhead while maintaining high-speed parallel communication
2Productivity
If Infiniband interface is used for direct hardware-level communication, then communication efficiency is improved, but protocol stack complexity and hardware dependency increase
Solution Approach 1:
The patent extracts and utilizes the Infiniband interface capabilities directly at the hardware level, bypassing traditional complex protocol stacks. By directly calling IB interface functions from the communication threads, the system achieves high communication efficiency while simplifying the software protocol layer, as the hardware interface handles low-level protocol details
Solution Approach 2:
The Infiniband interface is leveraged to perform multiple communication functions (data transmission, synchronization, memory sharing) through a single hardware interface. This multi-functionality improves overall communication efficiency by consolidating multiple operations into one standardized interface, reducing the need for separate protocol stacks for different communication needs
Data Source
AI summary
Embodiments of the present disclosure disclose communication methods and systems, electronic devices, and computer clusters. The method includes: separately creating a corresponding thread for at least one of a plurality of target devices, where the created thread corresponding to the target device includes a communication thread and a message processing thread, and the message processing thread includes a message sending thread and/or a message receiving thread; and communicating with a corresponding target device on the basis of the corresponding created thread.