SDN Controller Path Selection for Balanced AI Training Traffic
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data center networks face traffic imbalance due to high traffic throughput in AI training tasks, leading to reduced network throughput and poor AI cluster service performance, particularly in scenarios where the number of data flows is small but traffic is high.
Innovation Solution
An SDN controller is used to select forwarding paths based on network topology and task information, configuring flow tables to ensure balanced data flow routing, which includes acquiring task information, selecting paths with low network load, and configuring multicast flow tables for point-to-multipoint or multipoint-to-multipoint relationships.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If existing traffic balance mechanism is used to forward data flows, then forwarding is simple, but traffic imbalance occurs on links reducing network throughput
Solution Approach 1:
An SDN controller is introduced as an intermediary between data flows and forwarding nodes. The controller collects task information from computing resource scheduling platforms or servers, determines optimal forwarding paths based on network topology and communication models, and configures forwarding flow tables on forwarding nodes. This mediator enables global traffic balance while maintaining simple local forwarding operations.
Solution Approach 2:
The SDN controller performs preliminary path selection and flow table configuration before data flows are forwarded. By acquiring task information in advance, selecting optimal paths based on topology and communication models, and pre-configuring forwarding rules, the system ensures traffic balance is established before high-throughput AI training data flows begin, preventing traffic imbalance issues.
2Reliability
If SDN controller selects forwarding paths based on topology and task information, then traffic balance is achieved, but system complexity increases
Solution Approach 1:
The SDN controller centralizes complex path selection and flow table configuration functions, isolating control plane complexity from the data plane forwarding nodes. Forwarding nodes only need to execute simple table-lookup forwarding based on pre-configured rules, while the controller handles the complex tasks of collecting task information, analyzing communication models, and determining optimal paths based on network topology.
Solution Approach 2:
The system automatically acquires task information from computing resource scheduling platforms or servers, performs path selection based on collected topology information and communication models, and dynamically configures forwarding flow tables without manual intervention. This automation reduces operational complexity while maintaining traffic balance.
3Measurement precision
If flow tables are configured to each forwarding node, then precise path control is achieved, but configuration overhead increases
Solution Approach 1:
The SDN controller performs preliminary path selection by analyzing network topology and communication models before configuring flow tables. By determining optimal forwarding paths in advance and batch-configuring flow tables to multiple forwarding nodes simultaneously, the system achieves precise path control for AI training data flows while minimizing configuration time and overhead.
Data Source
AI summary
Examples of the present disclosure provide a method and a device for forwarding a data flow, an SDN controller and a storage medium. The method is applied to the SDN controller, and includes: acquiring task information of an AI training task, wherein the task information comprises a communication model of the AI training task, an address of a source task node and an address of a destination task node, and the communication model is to indicate an AI training data transferring relationship between the source task node and the destination task node; selecting a forwarding path between the source task node and the destination task node based on topology information of a network and the communication model, wherein the topology information includes a topology structure, a link state and a utilization rate of link bandwidth; configuring a forwarding flow table to each forwarding node on the forwarding path, causing the each forwarding node to forward a data flow of the AI training task from the source task node to the destination task node along the forwarding path based on the forwarding flow table. This solution can realize a traffic balance on whole network links, improve the network throughput and improve performance of AI cluster service.