A traffic forwarding scheduling method and device

Abstract

This application provides a traffic forwarding scheduling method and apparatus. The method includes announcing the metadata of the data stream to be transmitted to the switches; determining the first communication data required for data forwarding based on the data stream metadata; wherein the first communication data is used to characterize the communication duration and bandwidth requirements of the communication session; and determining the target forwarding path between the switches hop by hop based on the first communication data and the second communication data, wherein the second communication data is used to characterize the resource occupancy of the equivalent forwarding path between the switches.

Description

Technical Field

[0001] This application relates to the field of traffic scheduling technology, and in particular to a traffic forwarding scheduling method and apparatus. Background Technology

[0002] In large-scale RoCE network scenarios, multiple equal-cost paths exist end-to-end, and AI applications select the actual path based on the switch's equal-cost path load balancing algorithm. Existing RoCE network solutions do not have a targeted equal-cost multi-path forwarding and scheduling design for the network characteristics of AI applications. Due to the inherent defects of the existing path selection mechanism of switches, when facing AI application communication with elephant flow characteristics, uneven load distribution on equal-cost paths will occur (such as congestion on one path and idleness on other paths). This will cause the entire RoCE network to experience severe congestion far below its design capacity. Moreover, the larger the network scale and the more equal-cost links there are, the greater the probability of congestion and uneven link distribution. Summary of the Invention

[0003] The purpose of this application is to provide a traffic forwarding and scheduling method and apparatus to solve the problem of network-wide congestion caused by the inability to perform effective path selection during data transmission in large-scale RoCE network scenarios.

[0004] The embodiments of this application adopt the following technical solution: a traffic forwarding scheduling method, comprising:

[0005] Announce the metadata of the data stream to be transmitted to the switch;

[0006] Based on the data stream metadata, the first communication data required for data forwarding of the data stream metadata is determined; wherein, the first communication data is used to characterize the communication duration and communication bandwidth requirements of the communication session;

[0007] Based on the first communication data and the second communication data, the target forwarding path between each of the switches is determined hop-by-hop when the metadata of the same data stream is forwarded, wherein the second communication data is used to characterize the resource occupancy of the equivalent forwarding path between each of the switches.

[0008] In some embodiments, announcing the metadata of the data stream to be transmitted to the switch includes:

[0009] Construct data flow metadata, which includes source IP, destination IP, protocol type, source port, destination port, data communication algorithm, communication data block size, and number of communication members in the application task; wherein, there are multiple data flow metadata, and each data flow metadata corresponds to an application;

[0010] The data stream metadata is processed by a link layer discovery protocol to generate multiple extended messages;

[0011] Multiple extended messages are sent to the switch simultaneously in parallel.

[0012] In some embodiments, based on the data stream metadata, the first communication data required for data forwarding of the data stream metadata is determined, including:

[0013] Based on the data communication algorithm, communication data block size and number of communication members in the data stream metadata, the required communication duration of the communication session is determined.

[0014] When the data communication algorithm is the Ring algorithm, the communication duration h is:

[0015] h = 2 × (r - 1) × s ÷ (r × B);

[0016] In the formula, r is the number of communication members of the application task, s is the size of the communication data block, and B is the physical link carrying ratio.

[0017] In some embodiments, determining the first communication data required for data forwarding based on the data stream metadata further includes:

[0018] Based on the data communication algorithm, communication data block size, and number of communication members in the data stream metadata, the communication bandwidth requirement of the communication session is determined.

[0019] The required communication bandwidth w is:

[0020] w = 2 × (r - 1) × s × R ÷ r;

[0021] In the formula, R is the scaling factor.

[0022] In some embodiments, the second communication data includes network bandwidth usage and time usage;

[0023] Based on the first communication data and the second communication data, when determining the target forwarding path between the various switches hop-by-hop when forwarding the same data stream metadata, including:

[0024] Based on the extended message, from multiple equivalent forwarding paths between the two switches, the forwarding path that satisfies the communication duration and bandwidth requirements of the current data flow metadata communication session, and has the minimum network bandwidth and duration usage, is determined as the target forwarding path.

[0025] When there are multiple data stream metadata, each target forwarding path corresponding to the data stream metadata is determined simultaneously and in parallel.

[0026] In some embodiments, the method further includes:

[0027] Once the target forwarding path is determined, a notification signal is sent to the AI ​​application corresponding to the source port.

[0028] In some embodiments, the method further includes:

[0029] The data stream to be forwarded from the source port is forwarded to the target port through the total target forwarding path, wherein the total target forwarding path includes the target forwarding path.

[0030] This application also provides a traffic forwarding scheduling device, including:

[0031] The notification module is configured to notify the switch of the metadata of the data stream to be transmitted;

[0032] The first determining module is configured to determine the first communication data required for data forwarding based on the data stream metadata; wherein, the first communication data is used to characterize the communication duration and communication bandwidth requirements of the communication session.

[0033] The second determining module is configured to determine the target forwarding path between the switches hop-by-hop when the same data stream metadata is forwarded based on the first communication data and the second communication data, wherein the second communication data is used to characterize the resource occupancy of the equivalent forwarding path between the switches.

[0034] In some embodiments, the apparatus further includes a flow management module and a distributed flow management module, wherein the flow management module is located in the communication library of the AI ​​application, and the distributed flow management module is located in each switch.

[0035] In some embodiments, the flow management module and the distributed flow management module use the link layer discovery protocol extension to learn data flow metadata, and the flow management module and the distributed flow management module are unaware of each other. Attached Figure Description

[0036] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0037] Figure 1 This is a flowchart of the traffic forwarding scheduling method in this application;

[0038] Figure 2 This is a schematic diagram of the AI ​​application communication library and stream management module of this application;

[0039] Figure 3 This is a schematic diagram of the RoCE switch and distributed flow management module of this application;

[0040] Figure 4 For this application Figure 1 A flowchart of one embodiment of step S10;

[0041] Figure 5 This is a block diagram of the structure of the LLDP extension message in this application;

[0042] Figure 6 This is a flowchart of one embodiment of step S20 in Figure 1 of this application;

[0043] Figure 7 This is a flowchart of one embodiment of step S30 in Figure 1 of this application;

[0044] Figure 8 This is a structural block diagram of the scheduling and forwarding device of this application. Detailed Implementation

[0045] Various embodiments and features of this application are described herein with reference to the accompanying drawings.

[0046] It should be understood that various modifications can be made to the embodiments described herein. Therefore, the above description should not be considered as limiting, but merely as an example of embodiments. Other modifications within the scope and spirit of this application will be apparent to those skilled in the art.

[0047] The accompanying drawings, which are included in and form part of this specification, illustrate embodiments of the present application and, together with the general description of the present application given above and the detailed description of the embodiments given below, serve to explain the principles of the present application.

[0048] These and other features of this application will become apparent from the following description of preferred forms of embodiments given as non-limiting examples, with reference to the accompanying drawings.

[0049] It should also be understood that although this application has been described with reference to some specific examples, those skilled in the art can certainly implement many other equivalent forms of this application.

[0050] The above and other aspects, features and advantages of this application will become more apparent when taken in conjunction with the accompanying drawings and in view of the following detailed description.

[0051] Specific embodiments of this application are described thereafter with reference to the accompanying drawings; however, it should be understood that the claimed embodiments are merely examples of this application, which can be implemented in various ways. Well-known and / or repeated functions and structures are not described in detail to avoid unnecessary or redundant details that could obscure the application. Therefore, the specific structural and functional details claimed herein are not intended to be limiting, but merely serve as the basis and representative basis for the claims to teach those skilled in the art to use this application in a variety of substantially any suitable detailed structures.

[0052] This specification may use the phrases “in one embodiment,” “in another embodiment,” “in yet another embodiment,” or “in other embodiments,” all of which may refer to one or more of the same or different embodiments according to this application.

[0053] RoCE (RDMA over Converged Ethernet) is a network protocol that allows remote direct memory access over Ethernet. It's an application of RDMA (Remote Direct Memory Access) on Ethernet, designed to address latency in server-side data processing during network transmission. This protocol is widely used in high-performance computing, distributed storage, and AI to reduce CPU processing and latency, thus improving application performance. RoCE switches select the next hop for the equivalent path based on a five-tuple or seven-tuple of source MAC, destination MAC, source IP, destination IP, protocol type, source port, and destination port. However, RoCE switches inherently suffer from uneven traffic distribution when there are few data flows, leading to significant differences in the number of data flows on different equivalent paths (e.g., multiple flows on one path while other paths are idle).

[0054] LLDP (Link Layer Discovery Protocol) provides a standard link layer discovery protocol that enables network nodes (servers, switches, etc.) to discover each other and exchange their configuration information within the network. LLDP supports custom extensions based on TLV (Trinity Value List), and messages can be encapsulated in LLDPDU (Link Layer Discovery Protocol Data Unit) to publish data to directly connected network nodes.

[0055] AI application network communication characteristics: few data streams, but large data volume per stream, a typical "elephant stream." An elephant stream refers to a large, continuous data transmission process over a network link. A stream is typically identified using a 5-tuple; high-speed (high bandwidth usage), long-duration streams are called elephant streams, while low-speed, short-duration streams are called mouse streams. For example, sending emails, browsing web pages, and chatting on WeChat are all mouse streams. In the case of elephant streams, if the throughput of a single stream is too large, such as transferring a large file between two laptops via the SMB protocol, with a single stream rate reaching Gbps or hundreds of Mbps, this will lead to extremely high utilization of the security engine's central processing unit (CPU), even reaching 100%. This is because each stream is randomly assigned to different CPU cores through hashing. If a single stream is too large, the data packets of that stream will only be allocated to one CPU core for processing, resulting in extremely high utilization of that core while the utilization of other cores is very low.

[0056] After a brief introduction to the basic technology, this application provides a traffic forwarding and scheduling method to address the problems in the background technology.

[0057] Combination Figure 1 The method includes:

[0058] S10, announce the metadata of the data stream to be transmitted to the switch.

[0059] For example, the metadata of the data stream to be transmitted by the AI ​​application can be advertised to the switch to assist the switch in scheduling traffic path forwarding. For instance, combined with... Figure 2 and Figure 3 A flow management module (CL-Flow module) can be added to the communication library of the AI ​​application, and a distributed flow management module (SW-Flow module) can be added to the RoCE switch. Before the aggregated communication, the flow management module in the communication library will announce the metadata of the data flow to be transmitted to the RoCE switch, so as to assist the switch's distributed flow management module in scheduling the traffic path forwarding.

[0060] In distributed training, ensemble communication refers to a communication method where multiple processes (usually more than two) exchange data according to certain rules. It typically involves data exchange between multiple processes. A set of predefined communication primitives (such as AllReduce) are usually available for users. For example, AllReduce: sums (or performs other operations) the input data of all nodes and sends the result to the output of all nodes. This is one of the most commonly used ensemble communication primitives in distributed training, used to synchronize model gradients and other data.

[0061] S20, based on the data stream metadata, determine the first communication data required for data forwarding of the data stream metadata; wherein, the first communication data is used to characterize the communication duration and communication bandwidth requirements of the communication session.

[0062] For example, data stream metadata is data of a certain size, which requires a certain amount of communication time and bandwidth when forwarding data. Different data stream metadata may have different sizes, and the required communication time and bandwidth during transmission will also differ. Based on the data stream metadata, the first communication data required for data forwarding is determined, and this first communication data characterizes the required communication time and bandwidth of the communication session.

[0063] For example, continuing with the above embodiments, the distributed flow management module of the RoCE switch can calculate the required communication duration and bandwidth of an AI application communication session based on the data flow metadata announced by the flow management module in the communication library. For example, the communication duration can be calculated based on the algorithms and communication primitives used in the AI ​​communication library, such as RingAllReduce, single communication data block size, network transmission bandwidth, transmission latency, and link congestion level.

[0064] Based on the communication algorithms, operation primitives (such as Allreduce), single communication data block size, link bandwidth, and link latency of AI applications, the amount and duration of bandwidth resources occupied by AI application data streams are calculated. Combined with the data stream carrying capacity of equivalent physical links, the optimal communication forwarding path is selected through scheduling.

[0065] S30, based on the first communication data and the second communication data, determine the target forwarding path between each of the switches when forwarding the metadata of the same data stream hop by hop, wherein the second communication data is used to characterize the resource occupancy of the equivalent forwarding path between each of the switches.

[0066] For example, the first communication data can be used to characterize the communication duration and bandwidth requirements of the metadata of the data stream to be transmitted. When selecting and determining the target forwarding path, a forwarding path that does not match the first communication data cannot transmit the metadata of the data stream to be transmitted. Here, it is necessary to know the resource usage of each forwarding path based on the second communication data, and determine whether the forwarding path matches the first communication data corresponding to the metadata of the data stream to be transmitted based on the resource usage of each forwarding path.

[0067] For example, if the communication time required to represent the number of data stream elements to be transmitted in the first communication data is 1 second, and the communication bandwidth requirement is 0.8 Mbps (megabits per second), and there are 6 equivalent forwarding paths between these two switches during the next-hop data transmission from one switch to another, if the latency of 5 of these equivalent forwarding paths is much longer than 1 second, or if the link congestion is severe, then these 5 equivalent forwarding paths cannot meet the transmission requirements of the number of data stream elements to be transmitted. If only 1 of the 6 equivalent forwarding paths has the latency and link congestion conditions sufficient to meet the transmission requirements of the number of data stream elements to be transmitted, then this equivalent forwarding path can be determined as the target forwarding path for the next-hop data stream metadata transmission from one switch to another. Of course, it is understood that the number of equivalent forwarding paths between switches can be other numbers; this is merely an example and does not constitute a limitation on the scope of the claims.

[0068] By scheduling AI application communication paths hop by hop, the theoretically optimal end-to-end communication path selection is achieved.

[0069] In some embodiments, combined with Figure 4 The step of announcing the metadata of the data stream to be transmitted to the switch includes:

[0070] S101, construct data flow metadata, which includes source IP, destination IP, protocol type, source port, destination port, data communication algorithm, communication data block size, and number of communication members of application task; wherein, the data flow metadata includes multiple, and each data flow metadata corresponds to an application.

[0071] For example, before conducting aggregated communication, source ports are randomly assigned to all RoCE communication sessions, and data stream metadata is constructed. Data stream metadata may include source IP, destination IP, protocol type, source port, destination port, data communication algorithm, communication data block size, and the number of communication members in the application task. Source IP, destination IP, source port, destination port, and protocol type can identify a communication session or data stream. The communication library of an AI application can include multiple AI applications, and the number of data stream metadata entries can correspond to the number of AI applications. Each AI application requiring a communication session can have its own data stream metadata entry. This data stream metadata can uniquely represent its corresponding AI application.

[0072] The construction of all data stream metadata can be completed through the stream management module in the AI ​​application communication library.

[0073] S102, the data stream metadata is processed by a link layer discovery protocol to form multiple extended messages.

[0074] For example, the LLDP protocol has a Flow TLV extension (Type: Flow, Value: Flow metadata, CL-Flow listening address and port) for the CL-Flow module to advertise data flow metadata to the RoCE switch network. This means that Flow TLV support can be added to existing LLDP protocol extensions, reducing implementation costs.

[0075] The CL-Flow module and SW-Flow module of the communication switch in the AI ​​application communication library use the LLDP protocol extension to learn business data flow metadata. The modules do not need to establish long-term connections, are unaware of each other, and do not record each other's existence. The SW-Flow module runs distributed across all switches, does not depend on other members, and independently designs the forwarding and scheduling of data flows.

[0076] Each AI application performs routing and scheduling independently and in parallel. The LLDP source data packet payload is very low, the network bandwidth is almost not occupied, and there is no need to wait for other AI applications. There is almost no time consumption when communicating.

[0077] For example, LLDP extended messages can be encapsulated as such Figure 5 The format shown includes switch ID, interface ID, time, system TLVs, and flow TLV (communication quintuple and CL-Flow).

[0078] S103, send multiple extended messages to the switch simultaneously in parallel.

[0079] For example, data flow metadata and CL-Flow module information are sent to the directly connected Leaf switch via extended LLDP messages. The Leaf switch is the end-point switching device in the Spine-Leaf architecture of the data center network and a crucial device connecting servers and the network. The Leaf switch is responsible for connecting servers and other network devices to achieve data transmission and flow control. The Spine switch is primarily used to connect multiple Leaf switches and provides high-speed horizontal data transmission and forwarding capabilities. The Spine switch also ensures high availability and fault tolerance in the data center network, achieving load balancing and redundancy through a multi-path network architecture. This ensures that the communication path between any two Leaf switches is consistent, requiring only one Spine switch and one Leaf switch, thereby reducing latency and congestion. The Spine switch is responsible for high-speed forwarding of data traffic between Leaf switches, ensuring the efficient operation of the data center network. The Leaf switch is the next-level switch connected to the Spine switch, serving as the entry point connecting servers and other network devices.

[0080] The Leaf switch's SW-Flow module acquires LLDP extended messages. Based on the AI ​​application's communication algorithm, communication primitives, communication data block size, bandwidth and latency of the equivalent link of the switch, the SW-Flow module calculates the communication bandwidth and communication duration occupied by the AI ​​application. Combined with the physical link resource occupancy, it schedules and selects the optimal link forwarding path with the preferred hop.

[0081] The Leaf switch sends an LLDP extension message to the selected next-hop Spine switch. The data portion of the LLDP extension message received by the Spine switch is the same as that of the message received by the Leaf switch, but the header is different.

[0082] The Spine switch's SW-Flow module acquires the LLDP extended messages sent by the Leaf switch. Based on the AI ​​application's communication algorithm, communication primitives, communication data block size, bandwidth and latency of the equivalent link of the switch, it calculates the communication bandwidth and communication duration occupied by the AI ​​application. Combining the physical link resource occupancy, it schedules and selects the optimal link forwarding path for the next hop.

[0083] Sending extended messages without redundancy can minimize network resource consumption, reduce forwarding and scheduling time, and improve the communication efficiency of AI applications.

[0084] In some embodiments, combined with Figure 6 Based on the data stream metadata, determine the first communication data required for data forwarding from the data stream metadata, including:

[0085] S201, based on the data communication algorithm, communication data block size and number of communication members in the data stream metadata, determine the communication duration required for the communication session.

[0086] S202, when the data communication algorithm is the Ring algorithm, the communication duration h is:

[0087] h = 2 × (r - 1) × s ÷ (r × B);

[0088] In the formula, r is the number of communication members of the application task, s is the size of the communication data block, and B is the physical link carrying ratio.

[0089] For example, the business data communication algorithm of an AI application (including communication source terms such as ALL2ALL, AllReduce, etc., and communication implementation algorithms such as Ring, double binary tree, Collnet, etc.), the communication data block size (chunksize, abbreviated as s), and the number of communication members in the AI ​​application task (rank, abbreviated as r) can be used to calculate the communication session duration and bandwidth based on the above data parameters. Different communication algorithms have different calculation formulas. The above embodiment is the communication session duration required for a communication session with the communication source term ALL2ALL under the Ring algorithm. B is the physical link capacity ratio. In the above embodiment, more specifically, B is the physical link capacity ratio of the AI ​​application cluster under steady-state conditions, with a default empirical value of 0.193.

[0090] In some embodiments, determining the first communication data required for data forwarding based on the data stream metadata further includes:

[0091] S203, based on the data communication algorithm, communication data block size and the number of communication members of the application task in the data stream metadata, determine the communication bandwidth requirement of the communication session.

[0092] S204, the required communication bandwidth w is:

[0093] w = 2 × (r - 1) × s × R ÷ r;

[0094] In the formula, R is the scaling factor.

[0095] For example, continuing with the above embodiments, the communication data block size is s, and the number of communication members for the AI ​​application task is (). R is a scaling factor for different model scenarios, which can be derived based on test statistics experience.

[0096] In some embodiments, the second communication data includes network bandwidth usage and time usage.

[0097] Combination Figure 7 Based on the first and second communication data, when determining the target forwarding path between the switches hop-by-hop when forwarding the same data stream metadata, including:

[0098] S301, based on the extended message, from multiple equivalent forwarding paths between the two switches, determine the forwarding path that satisfies the communication duration and bandwidth requirements of the current data flow metadata communication session, and has the minimum network bandwidth and duration usage, as the target forwarding path.

[0099] For example, there are multiple equivalent forwarding paths between two switches that need to conduct a communication session. Based on the communication duration and bandwidth requirements of the current data flow metadata communication session, the forwarding path that satisfies the communication duration and bandwidth requirements of the current data flow metadata communication session and has the minimum network bandwidth and duration usage is determined according to the LLDP extended message and is the target forwarding path.

[0100] For example, if there are 6 equivalent forwarding paths, and the resource usage of 4 of them cannot meet the communication duration and bandwidth requirements of the current data stream metadata communication session, while the remaining 2 paths can, then the network bandwidth and duration usage of these 2 paths are compared. The forwarding path with the lowest network bandwidth and duration usage is selected as the target forwarding path. Between network bandwidth and duration usage, network bandwidth usage should be prioritized when determining the target forwarding path, ensuring that the chosen path better meets the data transmission speed requirements of the communication session.

[0101] S302, when there are multiple data stream metadata, simultaneously and in parallel determine each target forwarding path corresponding to the data stream metadata.

[0102] For example, corresponding to 8 AI applications, there are 8 data flow metadata. Equivalent forwarding paths can be determined simultaneously and in parallel for these 8 data flow metadata. Each AI application performs routing scheduling independently and in parallel. The CL-Flow module in the AI ​​application communication library and the SW-Flow module in the communication switch use the LLDP protocol extension to learn the business communication flow metadata. Modules do not need to establish long-lived connections and are unaware of each other's existence. The SW-Flow module runs distributed across all switches, independent of other members, and independently designs data flow forwarding scheduling, improving the speed of determining the target forwarding path while avoiding the impact of different AI applications' communication sessions on other AI application communication sessions. Moreover, the CL-Flow module and the SW-Flow module operate independently and in a distributed manner, with no theoretical upper limit on network size.

[0103] In some embodiments, the method further includes:

[0104] Once the target forwarding path is determined, a notification signal is sent to the AI ​​application corresponding to the source port. That is, after determining the target forwarding path, the switch can notify the CL-Flow module in the AI ​​application's communication library that the optimal path setting has been completed, and the AI ​​application can communicate.

[0105] In some embodiments, the method further includes:

[0106] The data stream to be forwarded from the source port is forwarded to the target port through the total target forwarding path, wherein the total target forwarding path includes the target forwarding path.

[0107] For example, once the target forwarding path is determined, the data flow to be forwarded by the AI ​​application can be forwarded from the source port to the target port through the overall target forwarding path corresponding to the AI ​​application. During the forwarding process, the data flow may pass through multiple switches before reaching the target port. Here, the overall target forwarding path is formed by combining multiple target forwarding paths. Each target forwarding path corresponds to an equivalent forwarding path between two switches. For multiple AI applications, their overall target forwarding paths may be different, and the overall target forwarding paths of each AI application may have overlapping parts. Of course, it is understandable that the overall target forwarding paths of different AI applications may also be completely different.

[0108] This application also provides a traffic forwarding and scheduling device, combined with Figure 8 The traffic forwarding and scheduling device includes:

[0109] The notification module is configured to notify the switch of the metadata of the data stream to be transmitted.

[0110] For example, the metadata of the data stream to be transmitted by the AI ​​application can be advertised to the switch to assist the switch in scheduling traffic path forwarding. For instance, a flow management module (CL-Flow module) can be added to the AI ​​application's communication library, and a distributed flow management module (SW-Flow module) can be added to the RoCE switch. By adding this module, the metadata of the data stream to be transmitted can be advertised to the RoCE switch through the flow management module in the communication library before the aggregated communication begins, thus assisting the switch's distributed flow management module in scheduling traffic path forwarding.

[0111] In distributed training, ensemble communication refers to a communication method where multiple processes (usually more than two) exchange data according to certain rules. It typically involves data exchange between multiple processes. A set of predefined communication primitives (such as AllReduce) are usually available for users. For example, AllReduce: sums (or performs other operations) the input data of all nodes and sends the result to the output of all nodes. This is one of the most commonly used ensemble communication primitives in distributed training, used to synchronize model gradients and other data.

[0112] The first determining module is configured to determine the first communication data required for data forwarding based on the data stream metadata; wherein the first communication data is used to characterize the communication duration and communication bandwidth requirements of the communication session.

[0113] For example, data stream metadata is data of a certain size, which requires a certain amount of communication time and bandwidth when forwarding data. Different data stream metadata may have different sizes, and the required communication time and bandwidth during transmission will also differ. Based on the data stream metadata, the first communication data required for data forwarding is determined, and this first communication data characterizes the required communication time and bandwidth of the communication session.

[0114] For example, continuing with the above embodiments, the distributed flow management module of the RoCE switch can calculate the required communication duration and bandwidth of an AI application communication session based on the data flow metadata announced by the flow management module in the communication library. For example, the communication duration can be calculated based on the algorithms and communication primitives used in the AI ​​communication library, such as RingAllReduce, single communication data block size, network transmission bandwidth, transmission latency, and link congestion level.

[0115] Based on the communication algorithms, operation primitives (such as Allreduce), single communication data block size, link bandwidth, and link latency of AI applications, the amount and duration of bandwidth resources occupied by AI application data streams are calculated. Combined with the data stream carrying capacity of equivalent physical links, the optimal communication forwarding path is selected through scheduling.

[0116] The second determining module is configured to determine the target forwarding path between the switches hop-by-hop when the same data stream metadata is forwarded based on the first communication data and the second communication data, wherein the second communication data is used to characterize the resource occupancy of the equivalent forwarding path between the switches.

[0117] For example, the first communication data can be used to characterize the communication duration and bandwidth requirements of the metadata of the data stream to be transmitted. When selecting and determining the target forwarding path, a forwarding path that does not match the first communication data cannot transmit the metadata of the data stream to be transmitted. Here, it is necessary to know the resource usage of each forwarding path based on the second communication data, and determine whether the forwarding path matches the first communication data corresponding to the metadata of the data stream to be transmitted based on the resource usage of each forwarding path.

[0118] For example, if the communication time required to represent the number of data stream elements to be transmitted in the first communication data is 1 second, and the communication bandwidth requirement is 0.8 Mbps (megabits per second), and there are 6 equivalent forwarding paths between these two switches during the next-hop data transmission from one switch to another, if the latency of 5 of these equivalent forwarding paths is much longer than 1 second, or if the link congestion is severe, then these 5 equivalent forwarding paths cannot meet the transmission requirements of the number of data stream elements to be transmitted. If only 1 of the 6 equivalent forwarding paths has the latency and link congestion conditions sufficient to meet the transmission requirements of the number of data stream elements to be transmitted, then this equivalent forwarding path can be determined as the target forwarding path for the next-hop data stream metadata transmission from one switch to another. Of course, it is understood that the number of equivalent forwarding paths between switches can be other numbers; this is merely an example and does not constitute a limitation on the scope of the claims.

[0119] By scheduling AI application communication paths hop by hop, the theoretically optimal end-to-end communication path selection is achieved.

[0120] In some embodiments, the apparatus further includes a flow management module and a distributed flow management module, wherein the flow management module is located in the communication library of the AI ​​application and the distributed flow management module is located in each switch.

[0121] For example, before the aggregated communication, the flow management module in the communication library announces the metadata of the data stream to be transmitted to the RoCE switch, assisting the switch's distributed flow management module in scheduling traffic path forwarding. The RoCE switch's distributed flow management module can calculate the required communication duration and bandwidth for the AI ​​application communication session based on the data stream metadata announced by the flow management module in the communication library. For instance, it can calculate the communication duration based on the algorithms and communication primitives used in the AI ​​communication library, such as RingAllReduce, single communication data block size, network transmission bandwidth, transmission latency, and link congestion level. The construction of all data stream metadata can be completed through the flow management module in the AI ​​application communication library.

[0122] In some embodiments, the flow management module and the distributed flow management module use the link layer discovery protocol extension to learn data flow metadata, and the flow management module and the distributed flow management module are unaware of each other.

[0123] The CL-Flow module and SW-Flow module of the communication switch in the AI ​​application communication library use the LLDP protocol extension for learning service data flow metadata. Modules do not need to establish long-lived connections and are unaware of each other's existence. The SW-Flow module runs distributed across all switches, independent of other members, and independently designs data flow forwarding and scheduling. This distributed operation improves the speed of determining the target forwarding path and avoids the impact of communication sessions between different AI applications on the communication sessions of other AI applications. Furthermore, the CL-Flow and SW-Flow modules operate independently and distributedly, with no theoretical upper limit on network size.

[0124] The foregoing has described in detail several embodiments of this application, but this application is not limited to these specific embodiments. Those skilled in the art can make various variations and modifications based on the concept of this application, and all such variations and modifications should fall within the scope of protection claimed in this application.

Claims

1. A traffic forwarding scheduling method, comprising: Announce the metadata of the data stream to be transmitted to the switch; Based on the data stream metadata, the first communication data required for data forwarding of the data stream metadata is determined; wherein, the first communication data is used to characterize the communication duration and communication bandwidth requirements of the communication session; Based on the first communication data and the second communication data, the target forwarding path between the switches is determined hop-by-hop when forwarding the same data stream metadata, wherein the second communication data is used to characterize the resource occupancy of the equivalent forwarding path between the switches; wherein, the step of announcing the data stream metadata to be transmitted to the switches includes: Construct data flow metadata, which includes source IP, destination IP, protocol type, source port, destination port, data communication algorithm, communication data block size, and number of communication members in the application task; wherein, there are multiple data flow metadata, and each data flow metadata corresponds to an application; The data stream metadata is processed by a link layer discovery protocol to generate multiple extended messages; Multiple extended messages are sent to the switch simultaneously in parallel.

2. The traffic forwarding scheduling method according to claim 1, wherein the first communication data required for data forwarding based on the data stream metadata is determined, includes: Based on the data communication algorithm, communication data block size and number of communication members in the data stream metadata, the required communication duration of the communication session is determined. When the data communication algorithm is the Ring algorithm, the communication duration h is: h = 2 × (r - 1) × s ÷ (r × B); In the formula, r is the number of communication members of the application task, s is the size of the communication data block, and B is the physical link carrying ratio.

3. The traffic forwarding scheduling method according to claim 2, further comprising determining the first communication data required for data forwarding based on data stream metadata, and including: Based on the data communication algorithm, communication data block size, and number of communication members in the data stream metadata, the communication bandwidth requirement of the communication session is determined. The required communication bandwidth w is: w = 2 × (r-1) × s × R ÷ r; In the formula, R is the scaling factor.

4. The traffic forwarding scheduling method according to claim 1, wherein the second communication data includes network bandwidth occupancy and time occupancy; Based on the first communication data and the second communication data, when determining the target forwarding path between the various switches hop-by-hop when forwarding the same data stream metadata, including: Based on the extended message, from multiple equivalent forwarding paths between the two switches, the forwarding path that satisfies the communication duration and bandwidth requirements of the current data flow metadata communication session, and has the minimum network bandwidth and duration usage, is determined as the target forwarding path. When there are multiple data stream metadata, each target forwarding path corresponding to the data stream metadata is determined simultaneously and in parallel.

5. The traffic forwarding and scheduling method according to claim 1, further comprising: Once the target forwarding path is determined, a notification signal is sent to the AI ​​application corresponding to the source port.

6. The traffic forwarding and scheduling method according to claim 1, further comprising: The data stream to be forwarded from the source port is forwarded to the target port through the total target forwarding path, wherein the total target forwarding path includes the target forwarding path.

7. A traffic forwarding and scheduling device, comprising: The notification module is configured to construct data flow metadata, which includes source IP, destination IP, protocol type, source port, destination port, data communication algorithm, communication data block size, and number of communication members in the application task. Multiple data flow metadata entries are included, each corresponding to an application. Multiple extended messages are generated from the data flow metadata using a link layer discovery protocol. These extended messages are then simultaneously and in parallel sent to the switch. The first determining module is configured to determine the first communication data required for data forwarding based on the data stream metadata; wherein, the first communication data is used to characterize the communication duration and communication bandwidth requirements of the communication session. The second determining module is configured to determine the target forwarding path between each of the switches hop-by-hop when the same data stream metadata is forwarded based on the first communication data and the second communication data, wherein the second communication data is used to characterize the resource occupancy of the equivalent forwarding path between each of the switches.

8. The traffic forwarding and scheduling device according to claim 7, the device further includes a flow management module and a distributed flow management module, the flow management module being set in the communication library of the AI ​​application, and the distributed flow management module being set in each switch.

9. The traffic forwarding and scheduling device according to claim 8, wherein the flow management module and the distributed flow management module use the link layer discovery protocol extension to learn data flow metadata, and the flow management module and the distributed flow management module are unaware of each other.

Images