Traffic forwarding scheduling method and device

Abstract

The invention provides a traffic forwarding scheduling method and device. The method comprises the following steps: notifying a switch of to-be-transmitted data stream metadata; based on the data stream metadata, determining first communication data required by data forwarding of the data stream metadata; wherein the first communication data is used for representing the communication duration and the communication bandwidth required by the communication session; based on the first communication data and second communication data, target forwarding paths between the switches are determined hop by hop when metadata of the same data stream is forwarded, and the second communication data is used for representing resource occupation conditions of equivalent forwarding paths between the switches.

Description

Technical Field

[0001] The present application relates to the technical field of traffic scheduling, and in particular to a traffic forwarding scheduling method and device. Background Art

[0002] In large-scale RoCE network scenarios, there are multiple equal-cost paths from end to end, and AI applications make actual path selection based on the switch's equal-cost path load algorithm. The existing RoCE network solution does not have a targeted equal-cost multi-path forwarding scheduling design for AI application network characteristics. Limited by the inherent defects of the existing path selection mechanism of the switch, when facing AI application communications with elephant flow characteristics, there will be an equal-cost path load imbalance problem (such as a single path is congested, and other paths are idle), resulting in severe congestion of the entire RoCE network when it is far from reaching the designed capacity. The larger the network scale, the more equal-cost links, and the greater the probability of congestion and link imbalance. Summary of the invention

[0003] The purpose of the embodiments of the present application is to provide a traffic forwarding scheduling method and device for solving the problem of network congestion caused by failure to perform effective path selection during data transmission in a large-scale RoCE network scenario.

[0004] The embodiment of the present application adopts the following technical solution: a traffic forwarding scheduling method, comprising:

[0005] Notify the switch of metadata of the data stream to be transmitted;

[0006] Based on the data stream metadata, determine first communication data required for the data stream metadata to perform data forwarding; wherein the first communication data is used to characterize the communication duration and communication bandwidth requirement required for the communication session;

[0007] Based on the first communication data and the second communication data, a target forwarding path between the switches when forwarding the same data flow metadata is determined hop by hop, wherein the second communication data is used to characterize resource occupancy of an equivalent forwarding path between the switches.

[0008] In some embodiments, notifying the switch of metadata of the data stream to be transmitted includes:

[0009] Constructing data stream metadata, the data stream metadata including source IP, destination IP, protocol type, source port, destination port, data communication algorithm, communication data block size, and number of application task communication members; wherein the data stream metadata includes multiple data stream metadata, each of which corresponds to one application;

[0010] Generating a plurality of extended messages for the data stream metadata through a link layer discovery protocol;

[0011] The plurality of extended messages are sent to the switch simultaneously and in parallel.

[0012] In some embodiments, determining first communication data required for data forwarding based on the data stream metadata includes:

[0013] Determine the communication duration required for the communication session based on the data communication algorithm, the communication data block size, and the number of application task communication members in the data stream metadata;

[0014] When the data communication algorithm is the Ring algorithm, the communication duration h is:

[0015] h = 2 × (r-1) × s ÷ (r × B);

[0016] Where r is the number of communication members of the application task, s is the size of the communication data block, and B is the physical link carrying ratio.

[0017] In some embodiments, determining first communication data required for data forwarding by the data stream metadata based on the data stream metadata further includes:

[0018] Determine the communication bandwidth requirement of the communication session based on the data communication algorithm, the communication data block size and the number of application task communication members in the data stream metadata;

[0019] The communication bandwidth requirement w is:

[0020] w=2×(r-1)×s×R÷r;

[0021] Where R is the scaling factor.

[0022] In some embodiments, the second communication data includes network bandwidth occupancy and duration occupancy;

[0023] Determining, hop by hop, a target forwarding path between the switches when forwarding the same data flow metadata based on the first communication data and the second communication data, includes:

[0024] According to the extended message, from a plurality of equal-cost forwarding paths between the two switches, a forwarding path that meets the communication duration and communication bandwidth requirements of the current data stream metadata communication session and has the smallest network bandwidth occupancy and duration occupancy is determined as the target forwarding path;

[0025] In the case that there are multiple data stream metadata, each target forwarding path corresponding to the data stream metadata is determined simultaneously and in parallel.

[0026] In some embodiments, the method further comprises:

[0027] When the target forwarding path is determined, a notification signal is sent to the AI ​​application corresponding to the source port.

[0028] In some embodiments, the method further comprises:

[0029] The data stream to be forwarded at the source port is forwarded to the target port through the overall target forwarding path, wherein the overall target forwarding path includes the target forwarding path.

[0030] The embodiment of the present application also provides a traffic forwarding scheduling device, including:

[0031] a notification module configured to notify the switch of metadata of the data stream to be transmitted;

[0032] A first determination module is configured to determine first communication data required for data forwarding of the data stream metadata based on the data stream metadata; wherein the first communication data is used to characterize the communication duration and communication bandwidth requirement required for the communication session;

[0033] The second determination module is configured to determine, hop by hop, a target forwarding path between each of the switches when forwarding the same data flow metadata based on the first communication data and the second communication data, wherein the second communication data is used to characterize resource occupancy of an equivalent forwarding path between each of the switches.

[0034] In some embodiments, the device further includes a flow management module and a distributed flow management module, wherein the flow management module is arranged in a communication library of the AI ​​application, and the distributed flow management module is arranged in each switch.

[0035] In some embodiments, the flow management module and the distributed flow management module use link layer discovery protocol extension to perform data flow metadata learning, and the flow management module and the distributed flow management module are not aware of each other. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0037] Figure 1 This is a flow chart of the traffic forwarding scheduling method of this application;

[0038] Figure 2 This is a schematic diagram of the structure of the AI ​​application communication library and flow management module of this application;

[0039] Figure 3 This is a schematic diagram of the structure of the RoCE switch and distributed flow management module of this application;

[0040] Figure 4 For this application Figure 1 A flowchart of an embodiment of step S10;

[0041] Figure 5 This is the structural diagram of the LLDP extended message for this application;

[0042] Figure 6 This is a flowchart of an embodiment of step S20 in FIG. 1 of the present application;

[0043] Figure 7 This is a flowchart of an embodiment of step S30 in FIG. 1 of the present application;

[0044] Figure 8 This is a structural block diagram of the scheduling forwarding device of this application. DETAILED DESCRIPTION

[0045] Various aspects and features of the present application are described herein with reference to the accompanying drawings.

[0046] It should be understood that various modifications may be made to the embodiments of the present application. Therefore, the above description should not be considered as limiting, but only as an example of an embodiment. Other modifications within the scope and spirit of the present application will occur to those skilled in the art.

[0047] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the present application and, together with the general description of the present application given above and the detailed description of the embodiments given below, serve to explain the principles of the present application.

[0048] These and other characteristics of the present application will become apparent from the following description of a preferred form of embodiment given as a non-limiting example with reference to the accompanying drawings.

[0049] It should also be understood that although the present application has been described with reference to some specific examples, those skilled in the art will be able to readily implement many other equivalent forms of the present application.

[0050] The above and other aspects, features and advantages of the present application will become more apparent in view of the following detailed description when taken in conjunction with the accompanying drawings.

[0051] Specific embodiments of the present application are described hereinafter with reference to the accompanying drawings; however, it should be understood that the embodiments applied for are merely examples of the present application, which may be implemented in a variety of ways. Well-known and / or repeated functions and structures are not described in detail to avoid unnecessary or redundant details that obscure the present application. Therefore, the specific structural and functional details applied for herein are not intended to be limiting, but merely serve as a basis and representative basis for the claims to teach those skilled in the art to use the present application in a variety of ways with substantially any suitable detailed structure.

[0052] This specification may use the phrases "in one embodiment," "in another embodiment," "in yet another embodiment," or "in other embodiments," all of which may refer to one or more of the same or different embodiments according to the present application.

[0053] RoCE (RDMA over Converged Ethernet): A network protocol that allows remote direct memory access over Ethernet. It is the application of RDMA (Remote Direct Memory Access) on Ethernet, which means remote direct data access. It is created to solve the delay of server-side data processing in network transmission. This protocol is widely used in scenarios such as high-performance computing, distributed storage, and AI artificial intelligence to reduce CPU processing and latency and improve application performance. RoCE switches select the next hop of an equal-cost path for data streams based on a five-tuple or seven-tuple of source MAC, destination MAC, source IP, destination IP, protocol type, source port, and destination port. RoCE switches have an inherent defect that when there are few data streams, the number of data streams on different equal-cost paths varies greatly (such as multiple streams on one path and other paths are idle), resulting in unbalanced traffic.

[0054] LLDP (Link Layer Discovery Protocol): provides a standard link layer discovery protocol that enables network nodes (servers, switches, etc.) to discover each other and exchange their configuration information in the network. LLDP supports custom extensions based on TLV (triplet encoding), and messages can be encapsulated in LLDPDU (LinkLayer Discovery Protocol Data Unit) to publish data to directly connected network nodes.

[0055] AI application network communication characteristics: few data flows, large data volume of a single flow, that is, a typical elephant flow. Elephant flow refers to a large amount of continuous data transmission through a network link. A five-tuple is usually used to identify a flow. A flow with a high rate (occupying a large bandwidth) and a long time is called an elephant flow, while a flow with a low rate and a short time is called a mouse flow. For example, sending emails, browsing the web, chatting on WeChat, etc. are all mouse flows. In the case of an elephant flow, if the throughput of a single flow is too large, for example, when a large file is transmitted between two laptops via the SMB protocol, the single flow rate reaches Gbps or hundreds of Mbps, which will cause the central processing unit (CPU) of the security engine to be extremely high in utilization, even up to 100%. This is because each flow is randomly assigned to a different CPU core through a hash. If a single flow is too large, the data packets of the flow will only be assigned to one CPU core for processing, resulting in extremely high utilization of the core, while the utilization of other cores is very low.

[0056] After a brief introduction to the basic technology, the embodiment of the present application provides a traffic forwarding scheduling method to solve the problems in the background technology.

[0057] Combination Figure 1 , the method comprising:

[0058] S10, notifying the switch of metadata of the data stream to be transmitted.

[0059] Exemplarily, the metadata of the data flow to be transmitted by the AI ​​application is notified to the switch to assist the switch in scheduling the traffic path forwarding. Figure 2 and Figure 3 , you can add a flow management module (CL-Flow module) to the communication library of the AI ​​application and a distributed flow management module (SW-Flow module) to the RoCE switch. Before the collective communication, the flow management module in the communication library is used to notify the RoCE switch of the metadata of the data flow to be transmitted, so as to assist the distributed flow management module of the switch in scheduling the traffic path forwarding.

[0060] Collective communication refers to a communication method in which multiple processes (usually greater than 2) exchange data according to certain rules, involving data exchange between multiple processes. There is usually a set of predefined communication primitives (such as AllReduce) for users to use. For example, AllReduce: After adding the input data of all nodes (or other operations), the result is sent to the output of all nodes. This is one of the most commonly used collective communication primitives in distributed training, used to synchronize data such as model gradients.

[0061] S20, based on the data stream metadata, determining first communication data required for data forwarding based on the data stream metadata; wherein the first communication data is used to characterize the communication duration and communication bandwidth requirement required for the communication session.

[0062] Exemplarily, the data stream metadata is data of a certain size, which requires a certain communication duration and communication bandwidth requirement when forwarding data. Different data stream metadata may have different sizes, and the communication duration and communication bandwidth requirement required for transmission are also different. Based on the data stream metadata, the first communication data required for the data stream metadata to forward data is determined, and the first communication data is used to characterize the communication duration and communication bandwidth requirement required for the communication session.

[0063] For example, in combination with the above embodiment, the distributed flow management module of the RoCE switch can calculate the communication duration and communication bandwidth required for the AI ​​application communication session based on the data flow metadata notified by the flow management module in the communication library. For example, the communication duration can be calculated based on the algorithm and communication primitives used by the AI ​​communication library, such as RingAllReduce, the size of a single communication data block, the network transmission bandwidth, the transmission delay, and the link congestion level.

[0064] Based on the communication algorithm, operation primitives (such as Allreduce), single communication data block size, link bandwidth, and link latency of AI applications, the amount and duration of bandwidth resources occupied by AI application data flows are calculated. Combined with the data flow carrying capacity of equivalent physical links, the optimal communication forwarding path is scheduled and selected.

[0065] S30, based on the first communication data and the second communication data, determining hop by hop a target forwarding path between the switches when forwarding the same data flow metadata, wherein the second communication data is used to characterize resource occupancy of an equivalent forwarding path between the switches.

[0066] Exemplarily, the first communication data can be used to characterize the communication time and communication bandwidth requirements for the data stream metadata to be transmitted. When selecting and determining the target forwarding path, the forwarding path that does not match the first communication data cannot achieve the transmission of the data stream metadata to be transmitted. Here, it is necessary to obtain the resource occupancy of each forwarding path based on the second communication data, and determine whether the forwarding path matches the first communication data corresponding to the data stream metadata to be transmitted based on the resource occupancy of each forwarding path.

[0067] For example, the first communication data represents that the communication time required for the number of data stream elements to be transmitted is 1 second, and the communication bandwidth requirement is 0.8Mbps (megabits per second). When the data is transmitted from one switch to another switch for the next hop, there are 6 equivalent forwarding paths between the two switches, and the delays of 5 of the equivalent forwarding paths are far more than 1 second, or the link congestion is serious, then these 5 equivalent forwarding paths cannot meet the transmission of the number of data stream elements to be transmitted. Among the 6 equivalent forwarding paths, only the delay and link congestion of 1 equivalent forwarding path can meet the transmission requirements of the number of data stream elements to be transmitted, then it can be determined that the equivalent forwarding path is the target forwarding path when the data stream metadata to be transmitted is transmitted from one switch to another interactive machine for the next hop data transmission. Of course, it can be understood that the number of equivalent forwarding paths between switches can also be other numbers, which is only used as an example and does not constitute a limitation on the scope of protection of the claims.

[0068] By scheduling AI application communication paths hop by hop, the theoretically optimal end-to-end communication path selection can be achieved.

[0069] In some embodiments, in combination Figure 4 , the step of notifying the switch of metadata of the data stream to be transmitted includes:

[0070] S101, constructing data stream metadata, the data stream metadata including source IP, destination IP, protocol type, source port, destination port, data communication algorithm, communication data block size, and number of application task communication members; wherein the data stream metadata includes multiple data stream metadata, each of which corresponds to an application.

[0071] Exemplarily, before collective communication, source ports are randomly assigned to all RoCE communication sessions, and data flow metadata is constructed. The data flow metadata may include source IP, destination IP, protocol type, source port, destination port, data communication algorithm, communication data block size, and number of application task communication members. Source IP, destination IP, source port, destination port, and protocol type can identify a communication session or data flow. The communication library of an AI application may include multiple AI applications, and the number of data flow metadata may correspond to the number of AI applications. Each AI application that needs to conduct a communication session may have a corresponding number of data flow metadata constructed. The data flow metadata may uniquely characterize its corresponding AI application.

[0072] The construction of all data stream metadata can be completed through the stream management module in the AI ​​application communication library.

[0073] S102: Generate multiple extended messages for the data stream metadata through a link layer discovery protocol.

[0074] Exemplarily, the LLDP protocol is provided with a Flow TLV extension (Type: Flow, Value: flow metadata, CL-Flow monitoring address and port), which is used by the CL-Flow module to notify the data flow metadata to the RoCE switch network. That is, Flow TLV support can be added to the existing LLDP protocol extension to reduce the implementation cost.

[0075] The CL-Flow module in the AI ​​application communication library and the SW-Flow module in the communication switch use the LLDP protocol extension to learn the metadata of the service data flow. There is no need to establish a long connection between the modules, and they do not perceive or record each other's existence. The SW-Flow module runs distributedly on all switches, does not rely on other members, and independently designs the forwarding scheduling of data flows.

[0076] Each AI application performs routing scheduling independently and in parallel. The LLDP source data message load is very low, the network bandwidth is almost not occupied, and there is no need to wait for other AI applications. Almost no time is consumed during communication.

[0077] For example, the LLDP extended message can be encapsulated as follows: Figure 5 The format shown includes switch ID, interface ID, time, system TLVs and Flow TLV (communication quintuple and CL-Flow).

[0078] S103, sending the plurality of extended messages to the switch simultaneously and in parallel.

[0079] Exemplarily, the data flow metadata and CL-Flow module information are sent to the directly connected Leaf switch by expanding the LLDP message. The Leaf switch is the end switching device in the Spine-Leaf architecture in building the data center network, and is also an important device connecting the server and the network. The Leaf switch is responsible for connecting the server and other network devices to achieve data transmission and flow control. The Spine switch is mainly used to connect multiple Leaf switches and provide horizontal high-speed data transmission and forwarding functions. The Spine switch is also responsible for achieving high availability and fault tolerance of the data center network, and achieving load balancing and redundancy through a multi-path network architecture. The communication path between any two Leaf switches is consistent, and only needs to pass through one Spine switch and one Leaf switch, thereby reducing latency and congestion. The Spine switch is responsible for forwarding data traffic between Leaf switches at high speed to ensure the efficient operation of the data center network. The Leaf switch is the next-level switch connected to the Spine switch, serving as the entry point for connecting servers and other network devices.

[0080] The SW-Flow module of the Leaf switch obtains the extended LLDP message. The SW-Flow module of the Leaf switch can calculate the communication bandwidth and communication duration of the AI ​​application based on the communication algorithm, communication primitive, communication data block size, bandwidth and latency of the equivalent link of the switch where the AI ​​application is located, and schedule and select the optimal link forwarding path for the first hop based on the physical link resource occupancy.

[0081] The leaf switch sends an extended LLDP message to the selected next-hop spine switch. The data part of the LLDP extended message received by the spine switch is the same as that received by the leaf switch, but the message header is different.

[0082] The SW-Flow module of the spine switch obtains the LLDP extended message sent by the leaf switch, and calculates the communication bandwidth and communication duration of the AI ​​application based on the communication algorithm, communication primitive, communication data block size, bandwidth and latency of the equivalent link of the switch where the AI ​​application is located. Combined with the physical link resource occupancy, the module schedules and selects the optimal link forwarding path for the next hop.

[0083] Sending extended messages without redundancy can minimize network resource usage, reduce forwarding scheduling time, and improve AI application communication efficiency.

[0084] In some embodiments, in combination Figure 6 , based on the data stream metadata, determining first communication data required for the data stream metadata to perform data forwarding, including:

[0085] S201, determining the communication duration required for the communication session based on the data communication algorithm, the communication data block size and the number of application task communication members in the data stream metadata.

[0086] S202, when the data communication algorithm is the Ring algorithm, the communication duration h is:

[0087] h = 2 × (r-1) × s ÷ (r × B);

[0088] Where r is the number of communication members of the application task, s is the size of the communication data block, and B is the physical link carrying ratio.

[0089] Exemplarily, the business data communication algorithm of the AI ​​application (including communication source language such as ALL2ALL, AllReduce, etc., communication implementation algorithms such as Ring, double binary tree, collnet, etc.), communication data block size (chunksize, abbreviated as s), and the number of AI application task communication members (rank, abbreviated as r), can calculate the communication duration and communication bandwidth size of the communication session based on the above data parameters, and the calculation formulas for different communication algorithms are different. The above embodiment is the communication duration required for a communication session with the communication source language ALL2ALL under the ring algorithm. B is the physical link load ratio. In the above embodiment, more specifically, B is the physical link load ratio of the AI ​​application cluster in steady state, and the default empirical value is 0.193.

[0090] In some embodiments, determining first communication data required for data forwarding by the data stream metadata based on the data stream metadata further includes:

[0091] S203, determining the communication bandwidth requirement of the communication session based on the data communication algorithm, the communication data block size and the number of application task communication members in the data stream metadata.

[0092] S204, the communication bandwidth requirement w is:

[0093] w=2×(r-1)×s×R÷r;

[0094] Where R is the scaling factor.

[0095] Exemplarily, in combination with the above embodiment, the communication data block size is s, and the number of AI application task communication members is ). R is a scaling factor in different model scenarios, which can be obtained based on test statistical experience.

[0096] In some embodiments, the second communication data includes network bandwidth occupancy and duration occupancy.

[0097] Combination Figure 7 , based on the first communication data and the second communication data, determining hop by hop a target forwarding path between the switches when forwarding the same data flow metadata, including:

[0098] S301, according to the extended message, from multiple equal-cost forwarding paths between two switches, determine a forwarding path that meets the communication duration and communication bandwidth requirements required for the current data stream metadata communication session and has the smallest network bandwidth occupancy and duration occupancy as the target forwarding path.

[0099] Exemplarily, there are multiple equivalent forwarding paths between two switches that need to conduct a communication session. Based on the communication duration and communication bandwidth requirement required for the current data flow metadata communication session, the forwarding path that meets the communication duration and communication bandwidth requirement required for the current data flow metadata communication session and has the smallest network bandwidth occupancy and duration occupancy is determined as the target forwarding path according to the LLDP extended message.

[0100] For example, the number of equivalent forwarding paths is 6, of which the resource occupancy of 4 equivalent forwarding paths cannot meet the communication duration and communication bandwidth requirements of the current data stream metadata communication session, and the resource occupancy of the remaining 2 equivalent forwarding paths can meet the communication duration and communication bandwidth requirements of the current data stream metadata communication session. Then the network bandwidth occupancy and duration occupancy of these two equivalent forwarding paths are compared, and the forwarding path with the smallest network bandwidth occupancy and duration occupancy is the target forwarding path. Of the two factors of network bandwidth occupancy and duration occupancy, the target forwarding path can be determined based on the network bandwidth occupancy, that is, the network bandwidth occupancy can be given priority, so that the determined target forwarding path is more in line with the data transmission speed requirements of the communication session.

[0101] S302: When there are multiple data stream metadata, simultaneously and in parallel determine each target forwarding path corresponding to the data stream metadata.

[0102] Exemplary, for example, 8 AI applications include 8 data flow metadata, and the equivalent forwarding paths of the 8 data flow metadata can be determined in parallel. Each AI application performs routing scheduling independently and in parallel. The CL-Flow module in the AI ​​application communication library and the communication switch SW-Flow module use the LLDP protocol extension to learn the business communication flow metadata. There is no need to establish a long connection between the modules, and they do not perceive or record each other's existence. The SW-Flow module runs distributedly on all switches, does not rely on other members, and independently designs the forwarding scheduling of data flows, which improves the speed of determining the target forwarding path, and avoids the impact of different AI applications on other AI application communication sessions during communication sessions. Moreover, the CT-Flow module and the SW-Flow module are distributed and run independently of each other, with no theoretical upper limit on the network scale.

[0103] In some embodiments, the method further comprises:

[0104] When the target forwarding path is determined, a notification signal is sent to the AI ​​application corresponding to the source port. That is, after the target forwarding path is determined, the switch can notify the CL-Flow module in the AI ​​application communication library that the optimal path setting has been completed and the AI ​​application can communicate.

[0105] In some embodiments, the method further comprises:

[0106] The data stream to be forwarded at the source port is forwarded to the target port through the overall target forwarding path, wherein the overall target forwarding path includes the target forwarding path.

[0107] Exemplarily, after the target forwarding path is determined, the data stream to be forwarded by the AI ​​application can be forwarded from the source port to the target port through the total target forwarding path corresponding to the AI ​​application. During the forwarding process, the data stream may pass through multiple switches in sequence before reaching the target port. Here, the total target forwarding path is formed by a combination of multiple target forwarding paths. Each target forwarding path corresponds to an equivalent forwarding path between two switches. For multiple AI applications, their total target forwarding paths may be different, and the total target forwarding paths of each AI application may have overlapping parts. Of course, it is understandable that the total target forwarding paths of different AI applications may also be completely different.

[0108] The present application also provides a traffic forwarding scheduling device. Figure 8 , the traffic forwarding scheduling device comprises:

[0109] The notification module is configured to notify the switch of metadata of the data flow to be transmitted.

[0110] Exemplarily, the metadata of the data flow to be transmitted of the AI ​​application is notified to the switch to assist the switch in scheduling the traffic path forwarding. For example, a flow management module (CL-Flow module) can be added to the communication library of the AI ​​application, and a distributed flow management module (SW-Flow module) can be added to the RoCE switch. Before adding the collective communication, the metadata of the data flow to be transmitted is notified to the RoCE switch through the flow management module in the communication library to assist the distributed flow management module of the switch in scheduling the traffic path forwarding.

[0111] Collective communication refers to a communication method in which multiple processes (usually greater than 2) exchange data according to certain rules, involving data exchange between multiple processes. There is usually a set of predefined communication primitives (such as AllReduce) for users to use. For example, AllReduce: After adding the input data of all nodes (or other operations), the result is sent to the output of all nodes. This is one of the most commonly used collective communication primitives in distributed training, used to synchronize data such as model gradients.

[0112] The first determination module is configured to determine first communication data required for data forwarding based on the data stream metadata; wherein the first communication data is used to characterize the communication duration and communication bandwidth requirement required for the communication session.

[0113] Exemplarily, the data stream metadata is data of a certain size, which requires a certain communication duration and communication bandwidth requirement when forwarding data. Different data stream metadata may have different sizes, and the communication duration and communication bandwidth requirement required for transmission are also different. Based on the data stream metadata, the first communication data required for the data stream metadata to forward data is determined, and the first communication data is used to characterize the communication duration and communication bandwidth requirement required for the communication session.

[0114] For example, in combination with the above embodiment, the distributed flow management module of the RoCE switch can calculate the communication duration and communication bandwidth required for the AI ​​application communication session based on the data flow metadata notified by the flow management module in the communication library. For example, the communication duration can be calculated based on the algorithm and communication primitives used by the AI ​​communication library, such as RingAllReduce, the size of a single communication data block, the network transmission bandwidth, the transmission delay, and the link congestion level.

[0115] Based on the communication algorithm, operation primitives (such as Allreduce), single communication data block size, link bandwidth, and link latency of AI applications, the amount and duration of bandwidth resources occupied by AI application data flows are calculated. Combined with the data flow carrying capacity of equivalent physical links, the optimal communication forwarding path is scheduled and selected.

[0116] The second determination module is configured to determine, hop by hop, a target forwarding path between each of the switches when forwarding the same data flow metadata based on the first communication data and the second communication data, wherein the second communication data is used to characterize resource occupancy of an equivalent forwarding path between each of the switches.

[0117] Exemplarily, the first communication data can be used to characterize the communication time and communication bandwidth requirements for the data stream metadata to be transmitted. When selecting and determining the target forwarding path, the forwarding path that does not match the first communication data cannot achieve the transmission of the data stream metadata to be transmitted. Here, it is necessary to obtain the resource occupancy of each forwarding path based on the second communication data, and determine whether the forwarding path matches the first communication data corresponding to the data stream metadata to be transmitted based on the resource occupancy of each forwarding path.

[0118] For example, the first communication data represents that the communication time required for the number of data stream elements to be transmitted is 1 second, and the communication bandwidth requirement is 0.8Mbps (megabits per second). When the data is transmitted from one switch to another switch for the next hop, there are 6 equivalent forwarding paths between the two switches, and the delays of 5 of the equivalent forwarding paths are far more than 1 second, or the link congestion is serious, then these 5 equivalent forwarding paths cannot meet the transmission of the number of data stream elements to be transmitted. Among the 6 equivalent forwarding paths, only the delay and link congestion of 1 equivalent forwarding path can meet the transmission requirements of the number of data stream elements to be transmitted, then it can be determined that the equivalent forwarding path is the target forwarding path when the data stream metadata to be transmitted is transmitted from one switch to another interactive machine for the next hop data transmission. Of course, it can be understood that the number of equivalent forwarding paths between switches can also be other numbers, which is only used as an example and does not constitute a limitation on the scope of protection of the claims.

[0119] By scheduling AI application communication paths hop by hop, the theoretically optimal end-to-end communication path selection can be achieved.

[0120] In some embodiments, the device further includes a flow management module and a distributed flow management module, the flow management module is arranged in the communication library of the AI ​​application, and the distributed flow management module is arranged in each switch.

[0121] Exemplarily, before collective communication is added, the data flow metadata to be transmitted is first notified to the RoCE switch through the flow management module in the communication library to assist the distributed flow management module of the switch in scheduling the traffic path forwarding. The distributed flow management module of the RoCE switch can calculate the communication duration and communication bandwidth required for the AI ​​application communication session based on the data flow metadata notified by the flow management module in the communication library. For example, the communication duration can be calculated based on the algorithms and communication primitives used by the AI ​​communication library, such as RingAllReduce, single communication data block size, network transmission bandwidth, transmission delay, and link congestion level. The construction of all data flow metadata can be completed through the flow management module in the AI ​​application communication library.

[0122] In some embodiments, the flow management module and the distributed flow management module use link layer discovery protocol extension to perform data flow metadata learning, and the flow management module and the distributed flow management module are not aware of each other.

[0123] The CL-Flow module in the AI ​​application communication library and the SW-Flow module of the communication switch use the LLDP protocol extension to learn the metadata of the service data flow. There is no need to establish a long connection between the modules, and they do not perceive or record each other's existence. The SW-Flow module runs distributedly on all switches, does not rely on other members, and independently designs the forwarding scheduling of data flows. The SW-Flow module runs distributedly on all switches, does not rely on other members, and independently designs the forwarding scheduling of data flows, which improves the speed of determining the target forwarding path and avoids the impact of different AI applications on other AI application communication sessions during communication sessions. Moreover, the CT-Flow module and the SW-Flow module run independently of each other in a distributed manner, with no theoretical upper limit on the network scale.

[0124] Multiple embodiments of the present application are described in detail above, but the present application is not limited to these specific embodiments. Those skilled in the art can make various variations and modifications to the embodiments based on the concept of the present application, and these variations and modifications should all fall within the scope of protection required by the present application.

Claims

1. A traffic forwarding scheduling method, comprising: Notify the switch of metadata of the data stream to be transmitted; Based on the data stream metadata, determine first communication data required for the data stream metadata to perform data forwarding; wherein the first communication data is used to characterize the communication duration and communication bandwidth requirement required for the communication session; Based on the first communication data and the second communication data, a target forwarding path between the switches when forwarding the same data flow metadata is determined hop by hop, wherein the second communication data is used to characterize resource occupancy of an equivalent forwarding path between the switches.

2. According to the traffic forwarding scheduling method of claim 1, the step of notifying the switch of metadata of the data stream to be transmitted comprises: Constructing data stream metadata, the data stream metadata including source IP, destination IP, protocol type, source port, destination port, data communication algorithm, communication data block size, and number of application task communication members; wherein the data stream metadata includes multiple data stream metadata, each of which corresponds to one application; Generating a plurality of extended messages for the data stream metadata through a link layer discovery protocol; The plurality of extended messages are sent to the switch simultaneously and in parallel.

3. The traffic forwarding scheduling method according to claim 1, determining the first communication data required for data forwarding based on the data stream metadata, comprising: Determine the communication duration required for the communication session based on the data communication algorithm, the communication data block size, and the number of application task communication members in the data stream metadata; When the data communication algorithm is the Ring algorithm, the communication duration h is: h = 2 × (r-1) × s ÷ (r × B); Where r is the number of communication members of the application task, s is the size of the communication data block, and B is the physical link carrying ratio.

4. The traffic forwarding scheduling method according to claim 3, determining the first communication data required for the data stream metadata to perform data forwarding based on the data stream metadata, further comprising: Determine the communication bandwidth requirement of the communication session based on the data communication algorithm, the communication data block size and the number of application task communication members in the data stream metadata; The communication bandwidth requirement w is: w=2×(r-1)×s×R÷r; Where R is the scaling factor.

5. According to the traffic forwarding scheduling method of claim 2, the second communication data includes network bandwidth occupancy and duration occupancy; Determining, hop by hop, a target forwarding path between the switches when forwarding the same data flow metadata based on the first communication data and the second communication data, includes: According to the extended message, from a plurality of equal-cost forwarding paths between the two switches, a forwarding path that meets the communication duration and communication bandwidth requirements of the current data stream metadata communication session and has the smallest network bandwidth occupancy and duration occupancy is determined as the target forwarding path; In the case that there are multiple data stream metadata, each target forwarding path corresponding to the data stream metadata is determined simultaneously and in parallel.

6. The traffic forwarding scheduling method according to claim 1, further comprising: When the target forwarding path is determined, a notification signal is sent to the AI ​​application corresponding to the source port.

7. The traffic forwarding scheduling method according to claim 1, further comprising: The data stream to be forwarded at the source port is forwarded to the target port through the overall target forwarding path, wherein the overall target forwarding path includes the target forwarding path.

8. A traffic forwarding scheduling device, comprising: a notification module configured to notify the switch of metadata of the data stream to be transmitted; A first determination module is configured to determine first communication data required for data forwarding of the data stream metadata based on the data stream metadata; wherein the first communication data is used to characterize the communication duration and communication bandwidth requirement required for the communication session; The second determination module is configured to determine, hop by hop, a target forwarding path between the switches when forwarding the same data flow metadata based on the first communication data and the second communication data, wherein the second communication data is used to characterize resource occupancy of an equivalent forwarding path between the switches.

9. According to the traffic forwarding scheduling device according to claim 8, the device also includes a flow management module and a distributed flow management module, the flow management module is set in the communication library of the AI ​​application, and the distributed flow management module is set in each switch.

10. According to the traffic forwarding scheduling device according to claim 9, the flow management module and the distributed flow management module use link layer discovery protocol extension to perform data flow metadata learning, and the flow management module and the distributed flow management module are not aware of each other.

Images