MPI Triplet Generation for Distributed Data Flow Graph Communication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In distributed computing systems, especially those using machine learning platforms like TensorFlow, the dynamic and random communication time sequences make it difficult for nodes to determine peer ends for data transmission, leading to increased negotiation load and reduced data transmission efficiency when using MPI technology.
Innovation Solution
The solution involves generating and using triplets that include a message tag, message size, and process sequence numbers to facilitate MPI sending and receiving primitives without prior negotiation between computing nodes, allowing for independent generation and use of these parameters by each node, thus simplifying the communication process and improving efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If MPI library is introduced to support data communication in distributed computing system, then data communication capability is improved, but negotiation process increases data communication load and affects transmission efficiency
Solution Approach 1:
The system performs preliminary actions by pre-establishing communication routes and pre-negotiating communication parameters between computing nodes before actual data transmission occurs. The route establishment module creates communication routes in advance, and the parameter negotiation module negotiates parameters beforehand, so that when data needs to be transmitted, the path and parameters are already determined, eliminating real-time negotiation delays and improving transmission efficiency
2Adaptability or versatility
If dynamic and random communication time sequence is used in computing platform, then flexibility of computation is improved, but difficulty in determining peer end increases
Solution Approach 1:
The system implements feedback mechanisms where computing nodes continuously report their computation status, data readiness, and communication capabilities to a central route establishment module. This feedback allows the system to dynamically adjust communication routes and timing while maintaining the ability to determine peer ends, as the central module has real-time visibility into the state of all nodes and can coordinate communications accordingly
Data Source
Figure 1
Figure 2
Figure 3
AI summary
This application discloses a distributed computing system. Both a first computing node and a second computing node in the distributed computing system store information about a name, a size, and a communication peer identifier of a first data flow graph parameter in a data flow graph, the first computing node stores the first data flow graph parameter, the first computing node and the second computing node generate respective triplets by using respective same interface parameter generation algorithms and information about the first data flow graph parameter that are stored in the nodes. The triplet is used as an interface parameter of a message passing interface MPI primitive that is used to transmit the first data flow graph parameter between the first computing node and the second computing node. In this way, the first computing node and the second computing node may transmit a data flow graph parameter by using an MPI primitive without negotiation. This can improve efficiency of data communication between computing nodes, to improve efficiency of computing a data flow graph by the distributed computing system.