Switch-Node Data Aggregation for Distributed Collective Communication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current collective communication methods in distributed computing systems face inefficiencies due to a large amount of data to be communicated and a high quantity of times of data synchronization, which affect the efficiency of distributed computing systems.
Innovation Solution
Implement a data processing method and switch node system that reduces the amount of data communicated and the number of synchronization times by aggregating data in a switch node before sending the result to the computing node, allowing the switch node to be used for other tasks during non-aggregation periods.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data aggregation is performed through traditional collective communication methods between computing nodes, then data exchange can be achieved, but the amount of data to be communicated increases and the number of synchronization times increases
Solution Approach 1:
The patent introduces a switch node as an intermediary between computing nodes to perform data aggregation. Instead of computing nodes directly exchanging large amounts of data through multiple synchronization steps, the switch node receives data from computing nodes, performs aggregation locally, and returns the aggregated result. This intermediary approach significantly reduces the communication volume and synchronization times between computing nodes.
Solution Approach 2:
The patent changes the dimension of data aggregation from the computing node layer to the network infrastructure layer (switch node). By moving the aggregation function to the switch node, the system transforms a distributed computation problem into a network-assisted operation, reducing the communication burden on computing nodes and improving overall system efficiency.
2Productivity
If data aggregation is performed through traditional collective communication methods, then data synchronization can be achieved, but the quantity of times of synchronization increases
Solution Approach 1:
The switch node acts as a mediator that consolidates multiple synchronization operations into a single aggregation operation. Computing nodes send data to the switch node once, and the switch node performs all necessary aggregations locally, eliminating the need for multiple rounds of synchronization between computing nodes.
Solution Approach 2:
The switch node performs data aggregation in advance before the computing nodes need the aggregated results. By pre-aggregating data at the network level, the system reduces the number of synchronization steps required in the main computing workflow, thereby reducing time loss.
3Reliability
If switch node resources are continuously occupied for data aggregation, then aggregation jobs can be completed, but the switch node cannot process other jobs
Solution Approach 1:
The switch node performs data aggregation in periodic intervals rather than continuously occupying resources. The aggregation operation is triggered only when data from computing nodes is available, allowing the switch node to process other jobs during non-aggregation periods. This periodic action pattern ensures both reliable aggregation job completion and high resource utilization.
Solution Approach 2:
The switch node dynamically adjusts its resource allocation between aggregation jobs and other processing tasks. Instead of being statically dedicated to aggregation, the switch node can flexibly switch between aggregation operations and other network processing functions based on real-time workload conditions, maximizing overall system productivity.
Data Source
Figure 1~2
Figure 3~4
Figure 5
AI summary
This application provides a data processing method, a switch node, and a related system, and is applied to a distributed computing system including a plurality of computing nodes and at least one first switch node. Each first switch node is connected to at least one computing node in the plurality of computing nodes. After receiving control information of the at least one computing node connected to the first switch node, the first switch node sends a data read request to the at least one computing node based on the control information. After obtaining a computation result obtained by computing data read based on the data read request, the first switch node sends the computation result to the at least one computing node. In the distributed computing system, when the plurality of computing nodes need to perform collective communication, a switch node obtains, from the computing nodes, data that needs to be aggregated, performs data aggregation, and then sends an aggregation result to the computing nodes, to reduce an amount of data that needs to be communicated and a quantity of times of synchronization that is performed when the data aggregation is performed through collective communication, thereby improving computing efficiency.