Switch Node Multicast Job Control for HPC Congestion Relief
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The challenge in high-performance computing (HPC) scenarios is the inefficient implementation of multicast communication between multiple nodes, particularly due to the large data amounts involved, which leads to congestion and resource bottlenecks.
Innovation Solution
A data processing method and switch node system that simplifies the control plane procedure by allowing computing nodes to dynamically create multicast jobs, with switch nodes managing data transmission and buffer capacity to avoid congestion.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If multicast data is transmitted directly from source to multiple destinations, then data transmission speed is improved, but network congestion and buffer overflow occur due to large data volumes
Solution Approach 1:
The patent segments the data transmission process into two distinct phases: a control plane phase for exchanging control information and a data plane phase for actual data transmission. This segmentation allows the system to manage large data volumes by first establishing transmission parameters and then executing controlled data transfer, preventing congestion while maintaining high speed.
Solution Approach 2:
The patent implements preliminary action by having computing nodes send control information to switch nodes before actual data transmission occurs. The switch node creates a multicast job based on this control information, preparing the transmission path and parameters in advance. This preliminary setup enables efficient data transmission while avoiding buffer overflow by controlling the transmission process before it begins.
2Reliability
If switch node buffer capacity is increased to handle large multicast data, then data transmission reliability is improved, but device complexity and resource consumption increase
Solution Approach 1:
The patent applies dynamics by making the buffer management adaptive rather than static. The switch node dynamically manages buffer resources based on the actual multicast data volume and transmission requirements. By using control information to pre-establish transmission parameters, the system can allocate buffer resources efficiently, ensuring reliability without requiring excessive fixed buffer capacity, thus reducing device complexity.
3Productivity
If control plane procedure is simplified to improve multicast efficiency, then system response time is reduced, but control information management complexity increases
Solution Approach 1:
The patent extracts the control information management function from the data transmission process and separates it into a distinct control plane. Computing nodes send control information to switch nodes, which then create multicast jobs independently. This extraction simplifies the overall procedure by allowing control and data planes to operate independently, improving multicast efficiency while making control information management more systematic and manageable.
Data Source
Figure 1~2
Figure 3~4
Figure 5
AI summary
This application provides a data processing method, a switch node, and a related system, and is applied to a distributed computing system including a plurality of computing nodes and at least one first switch node. Each first switch node is connected to at least one of the plurality of computing nodes. After receiving control information from a computing node, the first switch node creates a multicast job based on the received control information. The multicast job includes multicast destination information. Then, the first switch node obtains multicast data based on the multicast job, and sends the multicast data to the computing node based on the multicast job. In the distributed computing system, when the plurality of computing nodes execute a service that requires multicast, there is no need to configure related multicast information in advance. Instead, when the multicast is required, a computing node that participates in the multicast sends control information to a switch node, and the switch node creates a multicast job based on the control information to implement the multicast. In this way, a control plane procedure can be simplified when the multicast is implemented, and multicast efficiency can be improved.