A flow scheduling method, device and related equipment

Abstract

This application relates to the field of network communication technology, and in particular to a traffic scheduling method, apparatus, and related equipment. The method is applied to a controller; the method includes: acquiring network topology information, wherein the network topology information includes port connection relationships between computing nodes and access layer network devices, and between access layer network devices and core layer network devices; configuring the equivalent multipath forwarding policy of the access layer network devices, configuring its hash factor as the source port number of the data packets; acquiring the source port identifiers of multiple uplink ports connected to the core layer network devices by each access layer network device, to determine the source port identifier list corresponding to each access layer network device; sending the source port identifier list corresponding to the access layer network devices accessed by each computing node, so that when forwarding data packets, the computing node sequentially allocates each source port identifier included in the source port identifier list to each data packet, thereby achieving packet-by-packet traffic scheduling from the computing node side.

Description

Technical Field

[0001] This application relates to the field of network communication technology, and in particular to a traffic scheduling method, apparatus and related equipment. Background Technology

[0002] As the scale of intelligent computing clusters continues to expand, the amount of data transmitted between servers within the cluster is growing exponentially. Each data stream has unique characteristics, including content purpose, data size, bandwidth requirements, and latency requirements. These differentiated needs often lead to mutual interference between data streams, causing local network congestion, which in turn results in performance degradation or even data loss. Although device network bandwidth and server network card bandwidth continue to increase, simply expanding bandwidth can no longer meet the needs of explosive business growth. Network congestion problems are becoming increasingly prominent in intelligent computing networks, especially in large-scale model training scenarios.

[0003] In a Layer 2 box-to-box network in an intelligent computing center, to avoid network congestion, the uplink / downlink bandwidth convergence ratio of Leaf nodes is typically set to 1:1, and each Leaf node is fully interconnected with all Spine nodes. For clients with a limited number of servers, multiple lines are connected between the Spine and Leaf nodes to make better use of switch ports. For example, with each device equipped with 128 ports, there are two links between each Spine and each Leaf node. In a full-scale network, up to 4096 GPU cards can be connected.

[0004] Training large models in intelligent computing networks is characterized by complex traffic structures and high bandwidth requirements per stream. Traditional multipath load balancing techniques (such as hash algorithms) often fail in this scenario, leading to a significant decrease in network utilization, increasing the risk of congestion, and causing packet loss. Notably, in RDMA transmission scenarios, a packet loss rate of 0.1% can result in a 50% performance degradation; therefore, optimizing network load balancing is crucial.

[0005] Existing network load balancing schemes rely on IP pairs or IP pairs + QP pairs for path selection. Even with balanced path planning, the coarse-grained routing due to varying bandwidth requirements of each flow makes it difficult to balance flows with different bandwidth needs. This can lead to congestion on some links and idleness on others, failing to fully utilize the network's bandwidth. While existing intelligent operation and maintenance systems (such as path navigation) can perform global path calculations, they lack effective means to control terminal traffic and inject it into specified paths with fine granularity, resulting in gaps in end-to-end network coordination. Therefore, in a typical Layer 2 network architecture with multiple physical links between Spine-Leaf nodes, achieving seamless coordination between fine-grained packet control at the end-side (NIC) and deterministic path guidance at the network side is crucial to overcoming existing bottlenecks. Summary of the Invention

[0006] This application provides a traffic scheduling method, apparatus, and related equipment.

[0007] In a first aspect, this application provides a traffic scheduling method applied to a controller; the method includes: Obtain network topology information of the network, wherein the network topology information includes the port connection relationship between computing nodes and access layer network devices, and the port connection relationship between access layer network devices and core layer network devices; Configure the equal-cost multipath forwarding policy of the access layer network device, and configure its hash factor as the source port number of the data packet; Obtain the source port identifiers of multiple uplink ports connected to the core layer network device by each access layer network device, in order to determine the source port identifier list corresponding to each access layer network device; Each computing node sends a list of source port identifiers corresponding to the access layer network devices it accesses, so that when forwarding data packets, the computing node can sequentially distribute each source port identifier included in the list to each data packet, thereby achieving packet-by-packet traffic scheduling from the computing node side.

[0008] Optionally, the method further includes: Configurations are issued to each core layer network device in the network to configure multiple downlink ports of each core layer network device connected to the same access layer network device into a load balancing group, and a load balancing group identifier is assigned to the load balancing group. The load balancing group maintained by a core layer network device includes the mapping relationship between the downlink port of the core layer network device and one or more uplink ports. When forwarding a data packet, the core layer network device determines the uplink port to forward the data packet based on the downlink port that receives the data packet and the mapping relationship.

[0009] Optionally, the load balancing group is an LBN group, and the load balancing identifier value is an LBN group identifier.

[0010] Optionally, the method is applied to the controller of the intelligent computing center; the network is a Leaf-Spine network, the core layer network device is a Spine device, and the access layer network device is a Leaf device.

[0011] Secondly, this application provides a traffic scheduling device applied to a controller; the device includes: The acquisition unit is used to acquire network topology information of the network, wherein the network topology information includes the port connection relationship between computing nodes and access layer network devices, and the port connection relationship between access layer network devices and core layer network devices; The configuration unit is used to configure the equal-cost multipath forwarding strategy of the access layer network device and configure its hash factor as the source port number of the data packet; The acquisition unit is further configured to acquire the source port identifiers of multiple uplink ports of each access layer network device connected to the core layer network device, so as to determine the source port identifier list corresponding to each access layer network device; The sending unit is used to send a list of source port identifiers corresponding to the access layer network devices accessed by each computing node, so that when the computing node forwards data packets, it will sequentially distribute each source port identifier included in the list of source port identifiers to each data packet, thereby achieving packet-by-packet traffic scheduling from the computing node side.

[0012] Optionally, the configuration unit is further configured to: Configurations are issued to each core layer network device in the network to configure multiple downlink ports of each core layer network device connected to the same access layer network device into a load balancing group, and a load balancing group identifier is assigned to the load balancing group. The load balancing group maintained by a core layer network device includes the mapping relationship between the downlink port of the core layer network device and one or more uplink ports. When forwarding a data packet, the core layer network device determines the uplink port to forward the data packet based on the downlink port that receives the data packet and the mapping relationship.

[0013] Optionally, the load balancing group is an LBN group, and the load balancing identifier value is an LBN group identifier.

[0014] Optionally, the device is applied to the controller of the intelligent computing center; the network is a Leaf-Spine network, the core layer network device is a Spine device, and the access layer network device is a Leaf device.

[0015] Thirdly, embodiments of this application provide a traffic scheduling device, which includes: Memory, used to store program instructions; A processor is configured to invoke program instructions stored in the memory and execute the steps of the method as described in any one of the first aspects above, according to the obtained program instructions.

[0016] Fourthly, embodiments of this application also provide a computer-readable storage medium storing computer-executable instructions for causing a computer to perform the steps of the method as described in any of the first aspects above.

[0017] In summary, the traffic scheduling method provided in this application is applied to a controller. The method includes: acquiring network topology information, wherein the network topology information includes port connection relationships between computing nodes and access layer network devices, and port connection relationships between access layer network devices and core layer network devices; configuring the equivalent multipath forwarding policy of the access layer network devices, and configuring its hash factor as the source port number of the data packet; acquiring the source port identifiers of multiple uplink ports connected to the core layer network devices by each access layer network device, to determine the source port identifier list corresponding to each access layer network device; sending the source port identifier list corresponding to the access layer network devices accessed by each computing node, so that when the computing node forwards data packets, it sequentially allocates each source port identifier included in the source port identifier list to each data packet, thereby achieving packet-by-packet traffic scheduling from the computing node side.

[0018] The traffic scheduling method provided in this application realizes packet-by-packet scheduling of network traffic in the Layer 2 multi-link network of the intelligent computing center through the coordinated control between the controller, network devices and computing nodes in the network, which significantly improves network performance and resource utilization. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments of this application or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings of the embodiments of this application.

[0020] Figure 1 A detailed flowchart of a traffic scheduling method provided in an embodiment of this application; Figure 2 This is a schematic diagram of the configuration of an LBN group provided in an embodiment of this application; Figure 3 This is a schematic diagram of the structure of a traffic scheduling device provided in an embodiment of this application; Figure 4 This is a schematic diagram of the hardware architecture of a traffic scheduling device provided in an embodiment of this application. Detailed Implementation

[0021] The terminology used in the embodiments of this application is for the purpose of describing particular embodiments only and is not intended to limit the application. The singular forms “a,” “the,” and “the” as used in this application and claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to any and all possible combinations comprising one or more of the associated listed items.

[0022] It should be understood that although the terms first, second, third, etc., may be used to describe various information in embodiments of this application, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" may also be interpreted as "when," "when," or "in response to a determination."

[0023] For example, see Figure 1 The diagram shown is a detailed flowchart of a traffic scheduling method provided in an embodiment of this application. The method is applied to a controller and includes the following steps: Step 100: Obtain network topology information, wherein the network topology information includes the port connection relationship between computing nodes and access layer network devices, and the port connection relationship between access layer network devices and core layer network devices.

[0024] In this embodiment, the network is a smart computing center network. That is, the method is applied to the controller of the smart computing center (e.g., an AD-DC controller). The network architecture of the network managed by the controller is a Leaf-Spine network. Therefore, the corresponding core layer network device is a Spine device, the access layer network device is a Leaf device, and the computing nodes access the network through the Leaf device.

[0025] In practical applications, the AD-DC controller dynamically collects and stores the network topology information of the intelligent computing center, specifically including: the port connection relationship between all network devices (Leaf / Spine) included in the network, the physical location of the server nodes (computing nodes) and the network access points (accessed Leaf devices).

[0026] Step 110: Configure the equal-cost multipath forwarding policy of the access layer network device, and configure its hash factor as the source port number of the data packet.

[0027] Specifically, the controller issues configurations to each access layer network device (Leaf device) to configure the equal-cost multipath forwarding policy of the access layer network device. The purpose is to enable the access layer network device to perform hash forwarding based on the source port of the data packet when forwarding data packets.

[0028] In other words, after the access network device completes the above configuration, if it receives a data packet, it obtains the source port identifier (source port number) of the data packet, performs hash processing based on the source port number, and selects a route (selection of the outgoing port) based on the hash result.

[0029] Step 120: Obtain the source port identifiers of multiple uplink ports of each access layer network device connected to the core layer network device, so as to determine the source port identifier list corresponding to each access layer network device.

[0030] In this embodiment, the controller interacts with each Leaf device to obtain the port identifier of the uplink port connected to each Spine device (each uplink port has a unique mapped source port identifier), and constructs a list of source port identifiers corresponding to the Leaf device.

[0031] For example, for Leaf 1, interact with Leaf 1 to obtain the port identifiers of the uplink ports that connect Leaf 1 to each Spine device, and form the source port identifier list 1 corresponding to Leaf 1.

[0032] For Leaf 2, interact with Leaf 2 to obtain the port identifiers of the uplink ports connecting Leaf 2 to each Spine device, and form a source port identifier list 2 corresponding to Leaf 2.

[0033] Step 130: Send a list of source port identifiers corresponding to the access layer network devices it accesses to each computing node, so that when the computing node forwards data packets, it will sequentially distribute each source port identifier included in the list to each data packet, thereby achieving packet-by-packet traffic scheduling from the computing node side.

[0034] Specifically, if compute node 1 accesses the network through Leaf 1, it sends the source port identifier list corresponding to Leaf 1 to compute node 1. This allows compute node 1 to sequentially assign each source port identifier included in source port identifier list 1 to each data packet to be sent when forwarding data packets for the training task. For example, assuming source port identifier list 1 includes source port identifier 11, source port identifier 12, source port identifier 13, and source port identifier 14, then when the compute node sends the first data packet, it sets the source port number of the data packet to source port identifier 11; when sending the second data packet, it sets the source port number of the data packet to source port identifier 12; when sending the third data packet, it sets the source port number of the data packet to source port identifier 13; when sending the fourth data packet, it sets the source port number of the data packet to source port identifier 14; when sending the fifth data packet, it sets the source port number of the data packet to source port identifier 11, and so on.

[0035] In this embodiment of the application, assuming that computing node 1 accesses Leaf 1 through network card 1, when sending the source port identifier list corresponding to the access layer network device it accesses to computing node 1, the correspondence between source port identifier list 1 and network card 1 is determined, and the mapping relationship is sent to the agent deployed on computing node 1. When the network card 1 of computing node 1 sends a data packet for the training task, the source port number of the data packet is set to source port identifier 11. When sending the second data packet, the source port number of the data packet is set to source port identifier 12. When sending the third data packet, the source port number of the data packet is set to source port identifier 13. When sending the fourth data packet, the source port number of the data packet is set to source port identifier 14. When sending the fifth data packet, the source port number of the data packet is set to source port identifier 11, and so on.

[0036] This achieves packet-by-packet spraying on the Leaf device, thereby utilizing the entire network bandwidth of the intelligent computing center.

[0037] Furthermore, in this embodiment of the application, the controller can also issue configurations to each core layer network device in the network to configure multiple downlink ports of each core layer network device connected to the same access layer network device as a load balancing group, and assign a load balancing group identifier to the load balancing group. The load balancing group maintained by a core layer network device includes the mapping relationship between the downlink port of the core layer network device and one or more uplink ports. When the core layer network device forwards a data packet, it determines the uplink port for forwarding the data packet based on the downlink port that receives the data packet and the mapping relationship.

[0038] Specifically, the load balancing group is an LBN group, and the load balancing identifier value is an LBN group identifier.

[0039] For example, see Figure 2 The diagram shown is a configuration schematic of an LBN group provided in an embodiment of this application. On each Spine device, the Leaf device and the downlink port connected to each Leaf device are added to an LBN group. For example, the downlink port connected to Leaf 1 (L1) of the Spine device is configured as LBN group 1; the downlink port connected to Leaf 2 (L2) of the Spine device is configured as LBN group 2; ...; the downlink port connected to Leaf 64 (L64) of the Spine device is configured as LBN group 64.

[0040] In practical applications, each LBN group includes a mapping relationship between the downlink port connected to the corresponding Leaf device and at least one uplink port locally.

[0041] Based on the same inventive concept as the above-described embodiments, see, for example, the following: Figure 3 The diagram shown is a structural schematic of a traffic scheduling device provided in an embodiment of this application. This device is applied to a controller; the device includes: The acquisition unit 30 is used to acquire network topology information of the network, wherein the network topology information includes the port connection relationship between computing nodes and access layer network devices, and the port connection relationship between access layer network devices and core layer network devices; Configuration unit 31 is used to configure the equal-cost multipath forwarding strategy of the access layer network device and configure its hash factor as the source port number of the data packet; The acquisition unit 30 is further configured to acquire the source port identifiers of multiple uplink ports connected to the core layer network device by each access layer network device, so as to determine the source port identifier list corresponding to each access layer network device; The sending unit 32 is used to send a list of source port identifiers corresponding to the access layer network devices accessed by each computing node, so that when the computing node forwards data packets, it will sequentially distribute each source port identifier included in the list of source port identifiers to each data packet, thereby achieving packet-by-packet traffic scheduling from the computing node side.

[0042] Optionally, the configuration unit 31 is further configured to: Configurations are issued to each core layer network device in the network to configure multiple downlink ports of each core layer network device connected to the same access layer network device into a load balancing group, and a load balancing group identifier is assigned to the load balancing group. The load balancing group maintained by a core layer network device includes the mapping relationship between the downlink port of the core layer network device and one or more uplink ports. When forwarding a data packet, the core layer network device determines the uplink port to forward the data packet based on the downlink port that receives the data packet and the mapping relationship.

[0043] Optionally, the load balancing group is an LBN group, and the load balancing identifier value is an LBN group identifier.

[0044] Optionally, the device is applied to the controller of the intelligent computing center; the network is a Leaf-Spine network, the core layer network device is a Spine device, and the access layer network device is a Leaf device.

[0045] These units can be one or more integrated circuits configured to implement the above methods, such as one or more Application Specific Integrated Circuits (ASICs), one or more digital signal processors (DSPs), or one or more Field Programmable Gate Arrays (FPGAs). Alternatively, when one of these units is implemented using processing element scheduler code, the processing element can be a general-purpose processor, such as a Central Processing Unit (CPU) or other processor capable of calling program code. Furthermore, these units can be integrated together to form a system-on-a-chip (SOC).

[0046] Furthermore, regarding the traffic scheduling device provided in this application embodiment, from a hardware perspective, the hardware architecture diagram of the traffic scheduling device can be found in [reference needed]. Figure 4 As shown, the traffic scheduling device may include: a memory 40 and a processor 41. The memory 40 is used to store program instructions; the processor 41 calls the program instructions stored in the memory 40 and executes the above method embodiment according to the obtained program instructions. The specific implementation method and technical effect are similar, and will not be described again here.

[0047] Optionally, this application also provides a controller, including at least one processing element (or chip) for performing the above method embodiments.

[0048] Optionally, this application also provides a program product, such as a computer-readable storage medium storing computer-executable instructions for causing the computer to perform the above-described method embodiments.

[0049] Here, a machine-readable storage medium can be any electronic, magnetic, optical, or other physical storage device that can contain or store information, such as executable instructions, data, etc. For example, a machine-readable storage medium can be: RAM (Random Access Memory), volatile memory, non-volatile memory, flash memory, storage drives (such as hard disk drives), solid-state drives, any type of storage disk (such as optical discs, DVDs, etc.), or similar storage media, or combinations thereof.

[0050] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, which can take the form of a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email sending and receiving device, game console, tablet computer, wearable device, or any combination of these devices.

[0051] For ease of description, the above devices are described separately by function as various units. Of course, in implementing this application, the functions of each unit can be implemented in one or more software and / or hardware.

[0052] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, embodiments of this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0053] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0054] Furthermore, these computer program instructions can also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in the process. Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0055] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0056] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A traffic scheduling method, characterized in that, Applied to a controller; the method includes: Obtain network topology information of the network, wherein the network topology information includes the port connection relationship between computing nodes and access layer network devices, and the port connection relationship between access layer network devices and core layer network devices; Configure the equal-cost multipath forwarding policy of the access layer network device, and configure its hash factor as the source port number of the data packet; Obtain the source port identifiers of multiple uplink ports connected to the core layer network device by each access layer network device, in order to determine the source port identifier list corresponding to each access layer network device; Each computing node sends a list of source port identifiers corresponding to the access layer network devices it accesses, so that when forwarding data packets, the computing node can sequentially distribute each source port identifier included in the list to each data packet, thereby achieving packet-by-packet traffic scheduling from the computing node side.

2. The method as described in claim 1, characterized in that, The method further includes: Configurations are issued to each core layer network device in the network to configure multiple downlink ports of each core layer network device connected to the same access layer network device into a load balancing group, and a load balancing group identifier is assigned to the load balancing group. The load balancing group maintained by a core layer network device includes the mapping relationship between the downlink port of the core layer network device and one or more uplink ports. When forwarding a data packet, the core layer network device determines the uplink port to forward the data packet based on the downlink port that receives the data packet and the mapping relationship.

3. The method as described in claim 2, characterized in that, The load balancing group is an LBN group, and the load balancing identifier value is the LBN group identifier.

4. The method according to any one of claims 1-3, characterized in that, The method is applied to the controller of the intelligent computing center; the network is a Leaf-Spine network, the core layer network device is a Spine device, and the access layer network device is a Leaf device.

5. A flow scheduling device, characterized in that, Applied to a controller; the device includes: The acquisition unit is used to acquire network topology information of the network, wherein the network topology information includes the port connection relationship between computing nodes and access layer network devices, and the port connection relationship between access layer network devices and core layer network devices; The configuration unit is used to configure the equal-cost multipath forwarding strategy of the access layer network device and configure its hash factor as the source port number of the data packet; The acquisition unit is further configured to acquire the source port identifiers of multiple uplink ports of each access layer network device connected to the core layer network device, so as to determine the source port identifier list corresponding to each access layer network device; The sending unit is used to send a list of source port identifiers corresponding to the access layer network devices accessed by each computing node, so that when the computing node forwards data packets, it will sequentially distribute each source port identifier included in the list of source port identifiers to each data packet, thereby achieving packet-by-packet traffic scheduling from the computing node side.

6. The apparatus as claimed in claim 5, characterized in that, The configuration unit is also used for: Configurations are issued to each core layer network device in the network to configure multiple downlink ports of each core layer network device connected to the same access layer network device into a load balancing group, and a load balancing group identifier is assigned to the load balancing group. The load balancing group maintained by a core layer network device includes the mapping relationship between the downlink port of the core layer network device and one or more uplink ports. When forwarding a data packet, the core layer network device determines the uplink port to forward the data packet based on the downlink port that receives the data packet and the mapping relationship.

7. The apparatus as claimed in claim 6, characterized in that, The load balancing group is an LBN group, and the load balancing identifier value is the LBN group identifier.

8. The apparatus according to any one of claims 5-7, characterized in that, The device is used in the controller of the intelligent computing center; the network is a Leaf-Spine network, the core layer network device is a Spine device, and the access layer network device is a Leaf device.

9. A flow scheduling device, characterized in that, The traffic scheduling device includes: Memory, used to store program instructions; A processor is configured to invoke program instructions stored in the memory and execute the steps of the method as described in any one of claims 1-4 according to the obtained program instructions.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions for causing the computer to perform the steps of the method as described in any one of claims 1-4.

Images