A method for load balancing using a virtual in-network aggregation tree (VAT)
Patent Information
- Application Number
- PCT/US2024/018875
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2024-03-07
- Publication Date
- 2024-11-14
AI Technical Summary
Existing In-Network Aggregation (INA) systems are inflexible and inefficient in managing varying network topologies, device capabilities, and real-time job distributions in AI distributed training, leading to idle switches and inadequate resource allocation, which affects computation load balancing and network performance.
The method involves creating a Virtual In-Network Aggregation Tree (VAT) that dynamically maps servers and switches based on DCN topology, worker distribution, and available resources, using Bit Index Explicit Replication (BIER) bitmaps to indicate included workers and build multicast addresses for efficient data aggregation and load balancing.
This approach enables all servers and switches to participate in INA jobs, dynamically balancing computation loads and improving network performance by allowing flexible configuration of aggregation nodes and parameter servers within the DCN topology.
Smart Images

Figure US2024018875_14112024_PF_FP_ABST
Abstract
Description
A Method for Load Balancing Using a Virtual In-Network Aggregation Tree (VAT)TECHNICAL FIELD
[0001] The present disclosure is generally related to In-Network Computing (INC) and specifically to in-network aggregation for Artificial Intelligence (Al) distributed training.BACKGROUND
[0002] INC engages programmable network switches to execute an application function in part or in full — in cooperation with application servers — to improve performance (e g., lower latency, higher throughput) or reduce the system cost (e.g., lower power consumption, fewer servers). In- Network Aggregation (INA) is a promising INC application and has mature schemes and proven performance gains.SUMMARY
[0003] A first aspect relates to a method for performing in-network aggregation by a switch of a data center network (DCN), the method comprising receiving a data packet comprising a source bitmap, the source bitmap comprising bits where each bit represents a server in the DCN, wherein the bits in the source bitmap that have a value of one indicate servers whose data is included in the data packet; performing a bitwise AND of the source bitmap and a configuration bitmap to generate an inclusion bitmap, wherein the configuration bitmap comprises a bit representing each server in the DCN, wherein bits in the configuration bitmap that have a value of one indicate servers whose data packets are to be aggregated by the switch, and wherein the inclusion bitmap comprises a bit representing each server in the DCN, wherein bits in the inclusion bitmap that have a value of one indicate servers whose data packets are to be aggregated by the switch; when allbits in the inclusion bitmap have a value zero, forwarding the data packet to a destination address of the data packet; when the inclusion bitmap includes any bits with a value of one: storing the data packet in the switch; performing a bitwise OR of the source bitmap and an aggregation bitmap to generate a completion bitmap, wherein the aggregation bitmap comprises a bit representing each server in the DCN, wherein bits in the aggregation bitmap that have a value of one indicate servers whose data packets had previously been stored in the switch, and wherein the completion bitmap comprises a bit representing each server in the DCN, wherein bits in the completion bitmap that have a value of one indicate servers whose data packets are currently stored in the switch; and storing the completion bitmap in the aggregation bitmap.
[0004] Optionally, in any of the preceding aspects, another implementation of the aspect further includes receiving a configuration packet comprising the configuration bitmap; and in response to receiving the configuration packet, setting all bits of the aggregation bitmap to a value zero.
[0005] Optionally, in any of the preceding aspects, another implementation of the aspect further includes performing a bitwise AND of the configuration bitmap and the aggregation bitmap to generate a readiness bitmap; if the readiness bitmap is equal to the configuration bitmap: performing aggregation of data packets stored in the switch to generate an aggregation result; setting all bits of the aggregation bitmap to the value zero; selecting a target address for the aggregation result; and sending an aggregation packet comprising the aggregation result and the configuration bitmap to the target address.
[0006] Optionally, in any of the preceding aspects, another implementation of the aspect provides selecting the target address for the aggregation result comprises selecting the destination address of a data packet stored in the switch as the target address.
[0007] Optionally, in any of the preceding aspects, another implementation of the aspect provides the configuration packet further comprises a multicast address identifying servers of the DCN; and selecting the target address for the aggregation result comprises selecting the multicast address as the target address.
[0008] Optionally, in any of the preceding aspects, another implementation of the aspect provides the multicast address comprises a Bit Index Explicit Replication (BIER) header bitmap.
[0009] Optionally, in any of the preceding aspects, another implementation of the aspect provides selecting the target address for the aggregation result comprises selecting a plurality of target addresses comprising internet protocol (IP) addresses of all servers of the DCN with bits having a value of one in the configuration bitmap; and sending an aggregation packet to the target address comprises sending a point-to-point (P2P) packet to each of the plurality of target addresses.
[0010] Optionally, in any of the preceding aspects, another implementation of the aspect provides selecting the target address for the aggregation result comprises generating a multicast address identifying all servers of the DCN with bits having a value of one in the configuration bitmap.
[0011] A second aspect relates to a network apparatus, comprising a memory configured to store computer executable instructions; and a processor coupled to the memory and configured to execute the computer executable instructions to perform any of the methods of the first aspect.
[0012] Optionally, in any of the preceding aspects, another implementation of the aspect provides the memory comprises two or more memory components and the processor comprises two or more processor components.
[0013] A third aspect relates to a non-transitory computer readable storage medium comprising a computer program product for use by a network apparatus, the computer program productcomprising computer executable instructions stored on the non-transitory computer readable storage medium that, when executed by one or more processors, cause the network apparatus to execute any of the methods of the first aspect.
[0014] A fourth aspect relates to a network apparatus, comprising a transmitting means for transmitting a first packet comprising an aggregation result; a receiving means for receiving a second packet comprising one or more of a configuration packet and a data packet; and a processing means for: storing data packets in memory of the network apparatus and updating an aggregation bitmap stored in the memory of the network apparatus, based on a comparison of a source bitmap received in the data packet and a configuration bitmap received in the configuration packet; aggregating the data packets to generate the aggregation result, based on a comparison of the aggregation bitmap and the configuration bitmap; and selecting a target address for the first packet.
[0015] Optionally, in any of the preceding aspects, another implementation of the aspect provides the memory of the network apparatus comprises two or more memory components and a processor of the processing means comprises two or more processor components.
[0016] Optionally, in any of the preceding aspects, another implementation of the aspect provides the network apparatus is configured to perform any of the methods of the first aspect.
[0017] For the purpose of clarity, any one of the foregoing embodiments may be combined with any one or more of the other foregoing embodiments to create a new embodiment within the scope of the present disclosure.
[0018] These and other features, and the advantages thereof, will be more clearly understood from the following detailed description taken in conjunction with the accompanying drawings and claims.BRIEF DESCRIPTION OF THE DRAWINGS
[0019] For a more complete understanding of this disclosure, reference is now made to the following brief description, taken in connection with the accompanying drawings and detailed description, wherein like reference numerals represent like parts.
[0020] FIG. 1 A is a diagram of an example DCN according to an embodiment of the disclosure.
[0021] FIG. IB is a diagram showing only the nodes of the DCN of FIG. 1A assigned to a first VAT.
[0022] FIG. 2 A is a diagram of the DCN of FIG. 1 A.
[0023] FIG. 2B is a diagram showing only the nodes of the DCN of FIG. 2A assigned to a second VAT.
[0024] FIG. 3, comprising FIGS. 3A and 3B, illustrates a method for performing in-network aggregation in an aggregation node of a DCN according to an embodiment of the disclosure.
[0025] FIG. 4 is a diagram of a network apparatus according to an embodiment of the disclosure.
[0026] FIG. 5 is a diagram of an apparatus configured to implement one or more of the methods described herein according to embodiments of the disclosure.DETAILED DESCRIPTION
[0027] It should be understood at the outset that, although illustrative implementations of one or more embodiments are provided below, the disclosed systems and / or methods may be implemented using any number of techniques, whether currently known or in existence. The disclosure should in no way be limited to the illustrative implementations, drawings, and techniques illustrated below, including the exemplary designs and implementations illustrated and described herein, but may be modified within the scope of the appended claims along with their full scope of equivalents.
[0028] Al distributed training uses collective communication patterns such as many-to-one (e.g., synchronous all-reduce or asynchronous key-value reduce) in which networking and computing are two performance bottlenecks. In all-reduce Al distributed training, each of the M workers produces a gradient tensor vector (i.e., an array of values) almost at the same time. In keyvalue reduce Al distributed training, each of the M workers produces a set of {key, value} pairs almost at the same time. In either case, the workers’ data is sent to a Parameter Server (PS), in which the workers’ data is aggregated in an all-reduce or key-value reduce operation. The aggregated data is then sent back to the workers for another round of computing.
[0029] With the help of INA, the aggregation operation can be performed in one or more programmable switches on the packet forwarding path, where the switches can see all the tensor vectors on their way to the PS. While the following disclosure describes all-reduce applications, it will be understood that methods according to the disclosure also may be used for key-value reduce applications.
[0030] Existing INA systems such as Switch Machine Learning (SwitchML), Aggregation Transmission Protocol (ATP), and Scalable Hierarchical Aggregation and Reduction Protocol (SHARP) are all based on the same assumption: that N tensor vectors to be aggregated at this switch all come from N different physical ports. For SwitchML, only one switch is assumed to aggregate all the tensor vectors. For ATP, two levels of aggregation are conducted: one is at the worker side Top-of-Rack switches (TORs) and the other is at the PS-side TOR. For SHARP, an overlay tree is built on top of the DCN topology in which the workers are leaves, and each tree node is responsible to aggregate the tensor vectors from one child branch.
[0031] However, in an Al, ML, High Performance Computing (HPC) data center, characteristics such as network topology and condition, device capability, and real-time job distribution may allbe different in different DCNs. Tt may thus be inflexible to enforce a prior INA system designed for a prior set of characteristics, and doing so may create a situation in which, while some switches are idle, other switches may not have enough resources to accommodate their assigned INA job(s). Methods according to the disclosure enable all servers and switches in the DNC potentially to participate in INA jobs, while at the same time balancing computation loads on switches for improved network performance.
[0032] Methods according to the disclosure use mapping algorithms that consider the DCN topology, worker distribution, and available resources in switches, at the institution of a job, before dynamically and flexibly creating a VAT in the DCN topology to implement distributed INA for the job, according to the disclosure. The VAT is created in view of other jobs being executed in the DCN. The VAT is embedded in the physical DCN topology, but is not identical to the DCN physical tree topology of servers and switches. VAT leaf nodes correspond to servers (or workers) for a job, and certain switches (or internal VAT nodes) are responsible for aggregating the vectors each coming from one or more workers and / or other switches. However, while the workers / leaves are mapped to servers, the aggregation nodes and PS of the INA application may be mapped to either switches or servers of the physical tree, to obtain improved computation load balancing in the DCN.
[0033] Embodiments of the disclosure may use a BIER bitmap (typically a multicast addressing device) in a novel way in point-to-point (P2P) packets sent from workers to the PS. The BIER bitmaps of the packets may be used to (i) indicate to aggregation nodes which worker nodes’ tensor vectors (or partially or fully aggregated tensor vectors) are included in the packet and (ii) build up a multicast address for the PS to use in a BIER multicast header (or other multicast header) to multicast its aggregated tensor vector to the workers whose gradient vectors were aggregated. Aswill be described in more detail later, in such embodiments, a packet comprising a partially or fully aggregated tensor vector includes a BIER bitmap indicating all the workers whose vectors are included in the partially or fully aggregated tensor vector.
[0034] In various embodiments, the INA traffic follows a reverse multicast VAT created for a job, where each worker leaf node for the job sends a packet (comprising a job identifier and the worker’s gradient tensor vector) to the PS (the root node of the VAT). A switch (which may be referred to as an aggregation switch) receives a configuration bitmap that identifies the job and indicates the workers whose tensor vectors (or partially aggregated tensor vectors) the switch is to aggregate for the job.
[0035] The configuration bitmap (and other bitmaps referred to herein) comprise bits representing individual servers in the DCN. For example, for a DCN comprising eight servers, an 8-bit byte may be used as a bitmap for the DCN, with each bit of the byte representing one of the servers in the DCN.
[0036] Once the switch receives packets comprising all the indicated tensor vectors (or partially aggregated tensor vectors) relating to the identified job, the switch aggregates the vectors, selects the address of the PS as a target address, and sends the result to the target address. Upon performing final aggregation of the tensor vectors and / or partially aggregated tensor vectors (or upon receiving a fully aggregated tensor vector) for the job, the PS then multicasts the fully aggregated tensor vector to the worker leaf nodes for the job.
[0037] Methods according to the disclosure build a VAT on top of the DCN topology. The methods may be used with DCN topologies other than a tree, e.g., a Dragonfly topology. The leaves of the VAT tree are the workers, and the root is a PS of the INA application, which may be a switch or a server. The VAT offers more flexibility than a physical tree. One difference of theVAT from previous INA systems is that a physical link between switches may be shared by more than one virtual branch of the VAT. As a result, a switch may aggregate a plurality of vectors coming from a single physical port.
[0038] Each switch for a job is configured with a set of servers (workers) whose gradient tensor vectors the switch is to aggregate. The switch is configured by receiving a configuration bitmap having a bit representing each server in the DCN, wherein servers whose data packets are to be aggregated for the job by the switch are indicated by bits in the configuration bitmap having a value of one (1). The configuration bitmap for the root node of the job has bits set to one for all workers working on the job.
[0039] When a worker sends a vector packet to the root for aggregation, the packet also includes a bitmap. In some embodiments, the bitmap is a BIER bitmap in a BIER header as described above. As discussed above, this is a use of the BIER header in P2P packet sent in the leaf-to-root direction, which is an application where the BIER header is not normally used. In the packets sent by the servers, only the bit corresponding to the server sending the gradient tensor vector is set to a value of one in the bitmap.
[0040] When a switch configured as a non-leaf VAT node for the job receives a packet from a worker leaf node or another switch, the switch performs a bitwise AND operation between its configuration bitmap and the source bitmap received in the packet. (A bitwise AND operation performs a logical AND of each bit of a first bitmap with a corresponding bit of a second bitmap. If both bits are 1, the corresponding result bit is set to 1 . Otherwise, the corresponding result bit is set to 0.) If the result of the bitwise AND is all zeroes, the result indicates that the packet is not meant to be aggregated in this switch, so the switch forwards the packet to the packet’s destinationaddress, the PS (or root node). If the result of the bitwise AND is not all zeroes, the packet is terminated at this switch and is stored for aggregation.
[0041] The switch determines whether a complete set of packets for aggregation has been received, for example, by performing a bitwise OR of the switch’s configuration bitmap and bitmaps from all its stored packets and comparing the result to its configuration bitmap. (A bitwise OR operation performs a logical OR of each bit of a first bitmap with a corresponding bit of a second bitmap. If either or both bits are 1, the corresponding result bit is set to 1. Otherwise, the corresponding result bit is set to 0.) If the bitwise OR result and the configuration bitmap are equal, this indicates that the switch has all the tensor vectors needed for its aggregation operation. Once the switch completes its aggregation operation, it selects the destination address of the received packets — which is the address of the root node (or PS) — and sends the aggregation result to the target address.
[0042] A first switch may be configured to perform a partial aggregation of fewer than all the tensor gradient vectors for a job. For example, the first switch may be configured to aggregate vectors from only two of four workers on a job. The configuration bitmap of such a switch would include only two bits set to the value one and the packet comprising the aggregation result vector from the first switch would include a source bitmap comprising the same two bits set to the value of one. A second switch may be configured to aggregate the first switch’s partial aggregation vector with one or more worker vectors and / or other partial aggregation vectors. Such aggregation may continue in stages until the tensor vectors from all workers of a job have been aggregated and are available for the PS of the INA application.
[0043] Depending upon computation and communication loading of the DCN, a VAT may assign both the final aggregation operation and the PS function to a single server or switch of theDCN. Such a switch or leaf node is the root node of the VAT, to which all packets are sent. Like an intermediate partial aggregation switch, such a final aggregation / PS server or switch will wait to receive all worker vectors or partial aggregation vectors before performing the final aggregation operation. Once the final aggregation operation is performed, the final aggregation / PS server or switch will perform the functions of the PS on the complete aggregation vector. In VATs where the final aggregation operation and the PS function are assigned to different nodes of the DCN (servers or switches), the node performing the final aggregation operation sends the complete aggregation vector internally to its PS function.
[0044] FIG. 1A is a diagram of an example DCN 100 according to an embodiment of the disclosure. The DCN 100 includes eight servers, SRV1-SRV8, connected by seven switches, SW1-SW7. As indicated by their borders, the switches SW1, SW6, SW7, and the servers SRV1- SRV5 in the DCN 100 have been assigned to a VAT 110, explained in more detail with reference to FIG. IB.
[0045] FIG. IB is a diagram showing only the nodes of the DCN 100 assigned to the VAT 110. The servers and switches of the VAT 110 are arranged in FIG. IB with leaf nodes (workers) at the bottom of the figure and the root node (the PS of the INA application) at the top. The servers SRV1-SRV4 have been assigned as workers (indicated by double-line borders) and the server SRV5 as the PS (indicated by a heavy line border). Based on the path through the physical tree of the DCN 100 from the workers SRV1-SRV4 to the PS (SRV5), switches on that path (the switches SW1, SW7, and SW6) are assigned as non-leaf nodes (also indicated by a heavy line border). The borders of the nodes of the VAT 110 are similar in FIGS. 1A and IB. Packets comprising original, partially aggregated, and fully aggregated tensor vectors are represented by brackets, with numbers in the brackets indicating source workers whose tensor vectors are included in the packets.
[0046] As shown in FIG. 1 A, worker applications in the servers SRV1 and SRV2 send their original tensor vectors in packets addressed to the root node of the VAT 110 (the server SRV5). The packet sent by the server SRV 1 includes a source bitmap having only the bit corresponding to the server SRV1 set to the value one. The packet sent by the server SRV2 includes a source bitmap having only the bit corresponding to the server SRV2 set to the value one. As shown in FIG. 1A, the packets from the servers SRV1 and SRV2 are routed to the server SRV5 via switches SW1, SW5, SW7, SW6, and SW3 by a packet routing algorithm of the DCN 100. The switch SW1 has been configured in the VAT 110 to generate an aggregate tensor vector
[0012] from the tensor vectors received from the servers SRV1 and SRV2. The aggregate tensor vector
[0012] may also be referred to as an aggregation result or a partially aggregated tensor vector. The switch SW 1 selects the destination address of one of the received packets (i.e., the address of the PS) as a target address. The switch SW1 then sends a packet to the target address, where the packet comprises the aggregate tensor vector
[0012] and a source bitmap having both the bits corresponding to SRV1 and SRV2 set to the value one.
[0047] Similarly, worker applications in the servers SRV3 and SRV4 send their original tensor vectors and source bitmaps in packets addressed to the root node server SRV5. As shown in FIG. 1 A, the packets from the servers SRV3 and SRV4 are routed to the server SRV5 via switches SW2, SW5, SW7, SW6, and SW3 by the packet routing algorithm of the DCN.
[0048] The switch SW7 in the VAT 110 has been configured to aggregate only vectors received from the servers SRV3 and SRV4 and to send a packet having the root node server SRV5 as its destination, the packet comprising a partially aggregated tensor vector
[0034] and a source bitmap having bits corresponding to the servers SRV3 and SRV4 set to the value one. Because the switch SW7 has not been configured to further aggregate the partially aggregated tensor vector
[0012] received from the switch SW1, it merely forwards any packet containing such a vector to the packet’s destination address.
[0049] The switch SW6 in the VAT 110 has been configured to fully aggregate the tensor vectors from the servers SRV1-SRV4. When the switch SW6 has received both the packet comprising the partially aggregated tensor vector
[0012] and the packet comprising the partially aggregated tensor vector
[0034] , the switch SW6 aggregates the two partially aggregated tensor vectors into a fully aggregated tensor vector
[1234] , The switch SW6 then selects a target address and sends a packet to the target address, the packet comprising the fully aggregated tensor vector
[1234] and a source bitmap having bits corresponding to SRV1-SRV4 set to the value one. The switch SW6 selects the destination address of the received packets (the address of the PS) as the target address.
[0050] The server SRV5 is the PS of the VAT 110. When the SRV5 receives the fully aggregated tensor vector
[1234] and the source bitmap having bits corresponding to SRV1-SRV4 set to the value one, one function the server SRV5 performs as the PS is to select one or more target addresses for the worker nodes (servers SRV1-SRV4) and to send the fully aggregated tensor vector
[1234] to the target address(es). In embodiments having a small number of worker nodes, the PS selects the internet protocol (IP) addresses of each of the servers SRV1-SRV4 as target addresses and sends individual P2P packets to each worker. In other embodiments, the PS may select a multicast address as the target address and send the fully aggregated tensor vector
[1234] to all the workers in a multicast packet. In some such embodiments, the PS generates a multicast address that includes the worker nodes as the target address. In other such embodiments, the source bitmap received with the fully aggregated tensor vector
[1234] is a BIER multicast address, as described above. In such embodiments, the PS selects the BIER multicast address asthe target address. In these embodiments, the BIER multicast address has been aggregated by methods according to the disclosure, and fewer processing resources of the PS are required by simply selecting the BIER multicast address as the target address.
[0051] FIG. 2A is a diagram of the DCN of FIG. 1 A. As indicated by their borders, the switches SW3 and SW5, and the servers SRV1-SRV4 in the DCN 100 have been assigned to a VAT 210, explained in more detail with reference to FIG. 2B
[0052] FIG. 2B is a diagram showing only the nodes of the DCN 100 assigned to the VAT 210. The servers and switches of the VAT 210 are arranged in FIG. 2B with leaf nodes (workers) at the bottom of the figure and the root node (the PS of the INA application) at the top. The servers SRV1-SRV4 have been assigned as workers and the switch SW3 as the PS. Based on the path through the physical tree of the DCN 100 from the workers SRV1-SRV4 to the PS (the switch SW3), the switches SW5 and SW3 are assigned as non-leaf (or aggregation) nodes. In the VAT 210, the switch SW3 has been assigned as both an aggregation node and the PS The nodes of the VAT 210 are indicated in FIGS. 2 A and 2B with borders similar to those used to indicate the nodes of the VAT 110 in FIGS. 1A and IB. Also, similarly to FIGS. 1A and IB, packets comprising original, partially aggregated, and fully aggregated tensor vectors are represented by brackets, with numbers in the brackets indicating source workers whose tensor vectors are included in the packets.
[0053] Worker applications in the servers SRV1 and SRV2 send their original tensor vectors in packets addressed to the root node (the switch SW3) of the VAT 210. The packets also include source bitmaps, one with only the bit corresponding to SRV1 set to the value one and the other with only the bit corresponding to SRV2 set to the value one. As shown in FIG. 2A, the packets from the servers SRV1 and SRV2 are routed to the switch SW3 via switches SW1, SW5, SW7,and SW6, by a packet routing algorithm of the DCN. The switch SW5 has been configured in the VAT 210 to generate an aggregate tensor vector
[0012] from the tensor vectors received from the servers SRV1 and SRV2. The switch SW5 selects the destination address of one of the received packets (i.e., the address of the PS) as a target address. The switch SW5 then sends a packet to the target address, where the packet comprises the aggregate tensor vector
[0012] and a source bitmap having both the bits corresponding to SRV1 and SRV2 set to the value one.
[0054] Similarly, worker applications in the servers SRV3 and SRV4 send their original tensor vectors and source bitmaps in packets addressed to the switch SW3. However, because the switch SW5 in the VAT 210 has not been configured to aggregate the tensor vectors from the servers SRV3 and SRV4, the switch SW5 merely forwards any packets containing such vectors to the root node, switch SW3.
[0055] The switch SW3 in the VAT 210 has been configured to fully aggregate the aggregate tensor vector
[0012] and the tensor vectors from the servers SRV1-SRV4. When the switch SW3 has received both the packet comprising the partially aggregated tensor vector
[0012] and the packet comprising the tensor vectors from the servers SRV 1 -SRV4, the switch SW3 aggregates the tensor vectors into a fully aggregated tensor vector
[1234] , The switch SW3 also generates a source bitmap having bits corresponding to SRV1-SRV4 set to the value one.
[0056] As the switch SW3 has also been configured as the PS of the VAT 110, one function the switch SW3 performs is to send the fully aggregated tensor vector
[1234] to the worker nodes (servers SRV 1 -SRV4). As described above, in various embodiments, this may be done by sending individual P2P packets to each worker, by generating a multicast address indicating the worker nodes, or by using a BIER multicast address aggregated by methods according to the disclosure to include a bitmap indicating the worker nodes.
[0057] FIG. 3 illustrates a method 300 for performing in-network aggregation in a switch of a DCN according to an embodiment of the disclosure. The switch may be a programmable network switch, such as the switches SW1, SW6, or SW7 in FIGS. 1A and IB. The method may be applied to reduce gradient tensor vectors or to aggregate key-value pairs from a plurality of workers. The method 300 is described below with reference to elements of the DNC 100 in FIGS. 1A and IB.
[0058] Turning to FIG. 3A, in step 302, the switch receives a configuration packet that includes a configuration bitmap, where each server in the DCN is represented by one bit of the configuration bitmap. The bits of the configuration bitmap that have a value of one indicate servers whose data packets are to be aggregated by the switch. The bits of the configuration bitmap that have a value of zero indicate servers whose data packets are to be forwarded by the switch.
[0059] In step 304, in response to receiving the configuration bitmap, the switch sets all bits of an aggregation bitmap to a value zero. The aggregation bitmap comprises a bit representing each server in the DCN, wherein bits in the aggregation bitmap that have a value of one indicate servers whose data packets have been stored in the switch.
[0060] In step 306, the switch receives a data packet that includes a source bitmap. The source bitmap comprises a bit representing each server in the DCN, wherein bits in the source bitmap that have a value of one indicate servers whose data is included in the data packet. In step 308, the switch performs a bitwise AND (or comparison) of the source bitmap and the configuration bitmap to generate an inclusion bitmap. The inclusion bitmap comprises a bit representing each server in the DCN, wherein bits in the inclusion bitmap that have a value of one indicate servers whose data packets are to be aggregated by the switch. In step 310, the method determines whether all bits in the inclusion bitmap have the value zero, indicating that the data packet does not include data tobe aggregated by the switch. In response, in step 312, the switch forwards the packet to its destination address and the method 300 returns to step 306 to await another data packet.
[0061] If any bit in the inclusion bitmap of step 308 has a value of one, the data packet includes data to be aggregated by the switch and from step 310 the method continues to step 314, where the switch stores the data packet in its memory. The switch does not forward such a data packet to the destination address indicated in the packet.
[0062] In step 316, the switch performs a bitwise OR of the source bitmap and an aggregation bitmap to generate a completion bitmap. The completion bitmap comprises a bit representing each server in the DCN, wherein bits in the completion bitmap that have a value of one indicate servers whose data packets have been stored in the switch.
[0063] Further in step 316, the completion bitmap is stored the aggregation bitmap to update the aggregation bitmap to indicate all the servers whose data has so far been received and stored in the switch. Some data packets may be received from a worker (e.g., the server SRV1) and comprise a tensor vector and a source bitmap having only a single bit set. Other data packets may be received from another switch (e.g., the switch SW1) and comprise a partially aggregated tensor vector and a source bitmap having a plurality of bits set (e.g., bits indicating the servers SRV1 and SRV2).
[0064] Turning to FIG. 3B, in step 318, the switch performs a bitwise AND (or comparison) of the configuration bitmap and the aggregation bitmap to generate a readiness bitmap. In step 320, the switch compares the readiness bitmap to the configuration bitmap. If the readiness bitmap is not equal to the configuration bitmap, it is an indication that not all the expected tensor vectors and / or partially aggregated tensor vectors have been received, and the method 300 ends. If the readiness bitmap is equal to the configuration bitmap, it is an indication that all the expected tensorvectors and / or partially aggregated tensor vectors have been received (i. e. , the data is complete for aggregation), and the method 300 continues in step 322.
[0065] In step 322, the switch performs an aggregation of the data packets stored in the switch to generate an aggregation result. The aggregation may be an all-reduce aggregation or a keyvalue reduce aggregation. In step 324, the switch clears its aggregation bitmap, to indicate that the switch no longer has stored data packets to aggregate.
[0066] In step 326, the switch selects a target address and, in step 328, the switch sends (or transmits) a packet comprising the aggregation result (an aggregation packet) to the target address. If the switch is an intermediate partial aggregation node (such as the switches SW1 or SW7 of FIG. IB) or a final aggregation node (such as the switch SW6 of FIG. IB), the switch selects the destination address of one of its stored packets as the target address, as all original worker data and partially aggregated results are sent in packets addressed to the root node (the server SRV5, or PS, of FIG. IB).
[0067] If the switch is both a final aggregation node and the root node (or PS) (such as the switch SW3 of FIG. 2B), the switch aggregates the stored data packets to generate a final aggregation result and selects one or more target addresses to send the final aggregation result to the workers (e g., the servers SRV1, SRV2, SRV3, and SRV4 in either FIG. IB or 2B. As discussed above, in various embodiments, the target addresses may be individual addresses of the workers, a multicast address generated by the switch, or a multicast address included with the configuration packet received in step 302. In some such embodiments, the multicast address received in the configuration packet may be a BIER header bitmap.
[0068] In some embodiments, a network controller of a DCN is configured to generate a VAT that identifies a plurality of servers, one or more switches, and a root node of a network. Thenetwork controller is further configured to send a worker configuration packet to each worker node of the plurality of worker nodes, wherein the worker configuration packet comprises an IP address of the root node and a source bitmap having a bit representing each server in the network, wherein a bit representing the server has a value of 1 and all other bits have a value of zero. The network controller is further configured to send an aggregation configuration packet to each switch of the one or more switches, wherein the aggregation configuration packet comprises a configuration bitmap having a bit representing each server in the network, wherein one or more bits having a value of 1 indicate servers whose packets are to be aggregated by the switch and all other bits have a value of zero. The network controller is further configured to send a multicast address to the root node of the VAT, wherein the multicast address identifies the worker nodes of the VAT.
[0069] In some such embodiments, some or all of the worker configuration packets, the aggregation configuration packet, and the multicast address are sent via a control plane of the DCN. In some such embodiments, the multicast address comprises a Bit Index Explicit Replication (BIER) header bitmap.
[0070] FIG. 4 is a diagram of a network apparatus 400 (e.g., a network server or a programmable network switch, etc.) according to an embodiment of the disclosure. The network apparatus 400 is suitable for implementing the disclosed embodiments as described herein. The network apparatus 400 comprises ingress ports / ingress means 410 coupled to receiver units (Rx)Zreceiving means 420 for receiving data; a processor, logic unit, or central processing unit (CPU) / processing means 430 (coupled to the Rx / receiving means 420) to process the data; transmitter units (Tx) / transmitting means 440 and egress ports / egress means 450 (coupled to the processor / processing means 430) for transmitting the data; and a memory / memory means 460 (coupled to the processor / processing means 430) for storing the data. The network apparatus 400may also comprise optical-to-electrical (OE) components and electrical-to-optical (EO) components coupled to the ingress ports / ingress means 410, the receiver units / receiving means 420, the transmitter units / transmitting means 440, and the egress ports / egress means 450 for egress or ingress of optical or electrical signals.
[0071] The processor / processing means 430 is implemented by hardware and software. The processor / processing means 430 may be implemented as one or more processor components such as CPU chips, cores (e.g., as a multi-core processor), field-programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), or digital signal processors (DSPs). The processor / processing means 430 is in communication with the ingress ports / ingress means 410, receiver units / receiving means 420, transmitter units / transmitting means 440, egress ports / egress means 450, and memory / memory means 460. The processor / processing means 430 comprises an aggregation module 470. In various embodiments, the module 470 is stored in onboard memory of the processor / processing means 430. The aggregation module 470 is able to implement steps of the method 300, as described with reference to FIG. 3.
[0072] The inclusion of the aggregation module 470 therefore provides a substantial improvement to the functionality of the network apparatus 400 and effects a transformation of the network apparatus 400 to a different state. Alternatively, the aggregation module 470 is implemented as computer executable instructions stored in the memory / memory means 460 and executed by the processor / processing means 430.
[0073] The network apparatus 400 may also include input and / or output (I / O) devices / I / O means 480 for communicating data to and from a user. (I / O) devices / I / O means 480 may be coupled to the processor / processing means 430. The (I / O) devices / I / O means 480 may include output devices such as a display for displaying video data, speakers for outputting audio data, etc. The (I / O)devices / I / O means 480 may also include input devices, such as a keyboard, mouse, trackball, etc., and / or corresponding interfaces for interacting with such output devices.
[0074] The memory / memory means 460 comprises one or more memory components such as disks, tape drives, or solid-state drives and may be used as an over-flow data storage device, may be used to store programs when such programs are selected for execution, to store computer executable instructions that are read during program execution and to store data for execution or generated during execution. The memory / memory means 460 may be volatile and / or non-volatile and may be read-only memory (ROM), random access memory (RAM), ternary content- addressable memory (TCAM), and / or static random-access memory (SRAM). A computer program product may comprise computer executable instructions stored on a non-transitory computer readable storage medium, for instance the memory / memory means 460, that when executed by a processor, for instance the processor / processing means 430, cause an apparatus to perform any of the embodiments.
[0075] FIG. 5 illustrates a network apparatus 500 configured to implement one or more of the methods for aggregating worker data as described herein. For example, the network apparatus 500 is configured to implement the methods described with reference to FIG. 3. The network apparatus 500 may be implemented in the network device 400. The network apparatus 500 comprises a means 502 for aggregating worker data according to the method 300 as described with reference to FIG. 3.
[0076] The disclosed embodiments may be a system, an apparatus, a method, and / or a computer program product at any possible technical detail level of integration. The computer program product may include a computer readable storage medium (or media) having computer readable program instructions thereon for causing a processor to carry out aspects of the present disclosure.The computer readable storage medium may be a tangible device that can retain and store instructions for use by an instruction execution device. While several embodiments have been provided in the present disclosure, it may be understood that the disclosed systems and methods might be embodied in many other specific forms without departing from the spirit or scope of the present disclosure. The present examples are to be considered as illustrative and not restrictive, and the disclosure is not to be limited to the details given herein. For example, the various elements or components may be combined or integrated in another system or certain features may be omitted, or not implemented.
[0077] In addition, techniques, systems, subsystems, and methods described and illustrated in the various embodiments as discrete or separate may be combined or integrated with other systems, modules, techniques, or methods without departing from the scope of the present disclosure. Other items shown or discussed as coupled or directly coupled or communicating with each other may be indirectly coupled or communicating through some interface, device, or intermediate component whether electrically, mechanically, or otherwise. Other examples of changes, substitutions, and alterations are ascertainable by one skilled in the art and may be made without departing from the spirit and scope disclosed herein.
Claims
CLAIMSWhat is claimed is:
1. A method for performing in-network aggregation by a switch of a data center network (DCN), the method comprising: receiving a data packet comprising a source bitmap, the source bitmap comprising bits where each bit represents a server in the DCN, wherein the bits in the source bitmap that have a value of one indicate servers whose data is included in the data packet; performing a bitwise AND of the source bitmap and a configuration bitmap to generate an inclusion bitmap, wherein the configuration bitmap comprises a bit representing each server in the DCN, wherein bits in the configuration bitmap that have a value of one indicate servers whose data packets are to be aggregated by the switch, and wherein the inclusion bitmap comprises a bit representing each server in the DCN, wherein bits in the inclusion bitmap that have a value of one indicate servers whose data packets are to be aggregated by the switch; when all bits in the inclusion bitmap have a value zero, forwarding the data packet to a destination address of the data packet; when the inclusion bitmap includes any bits with a value of one: storing the data packet in the switch; performing a bitwise OR of the source bitmap and an aggregation bitmap to generate a completion bitmap, wherein the aggregation bitmap comprises a bit representing each server in the DCN, wherein bits in the aggregation bitmap that have a value of one indicate servers whose data packets had previously been stored in the switch, and wherein the completion bitmap comprises a bit representing each server in the DCN, wherein bits in the completion bitmap that have a value of one indicate servers whose data packets are currently stored in the switch; andstoring the completion bitmap in the aggregation bitmap.
2. The method of claim 1, further comprising: receiving a configuration packet comprising the configuration bitmap; and in response to receiving the configuration packet, setting all bits of the aggregation bitmap to a value zero.
3. The method of claim 1 or 2, further comprising: performing a bitwise AND of the configuration bitmap and the aggregation bitmap to generate a readiness bitmap; if the readiness bitmap is equal to the configuration bitmap: performing aggregation of data packets stored in the switch to generate an aggregation result; setting all bits of the aggregation bitmap to the value zero; selecting a target address for the aggregation result; and sending an aggregation packet comprising the aggregation result and the configuration bitmap to the target address.
4. The method of claim 3, wherein: selecting the target address for the aggregation result comprises selecting the destination address of a data packet stored in the switch as the target address.
5. The method of claim 3, wherein: the configuration packet further comprises a multicast address identifying servers of the DCN; and selecting the target address for the aggregation result comprises selecting the multicast address as the target address.
6. The method of claim 5, wherein the multicast address comprises a Bit Index Explicit Replication (BIER) header bitmap.
7. The method of claim 3, wherein: selecting the target address for the aggregation result comprises selecting a plurality of target addresses comprising internet protocol (IP) addresses of all servers of the DCN with bits having a value of one in the configuration bitmap; and sending an aggregation packet to the target address comprises sending a point-to-point (P2P) packet to each of the plurality of target addresses.
8. The method of claim 3, wherein: selecting the target address for the aggregation result comprises generating a multicast address identifying all servers of the DCN with bits having a value of one in the configuration bitmap.
9. A network apparatus, comprising: a memory configured to store computer executable instructions; anda processor coupled to the memory and configured to execute the computer executable instructions to perform the method of any of claims 1-8.
10. The network apparatus of claim 9, wherein the memory comprises two or more memory components and the processor comprises two or more processor components.
11. A non-transitory computer readable storage medium comprising a computer program product for use by a network apparatus, the computer program product comprising computer executable instructions stored on the non-transitory computer readable storage medium that, when executed by one or more processors, cause the network apparatus to execute the method of any of claims 1-8.
12. A network apparatus, comprising: a transmitting means for transmitting a first packet comprising an aggregation result; a receiving means for receiving a second packet comprising one or more of a configuration packet and a data packet; and a processing means for: storing data packets in memory of the network apparatus and updating an aggregation bitmap stored in the memory of the network apparatus, based on a comparison of a source bitmap received in the data packet and a configuration bitmap received in the configuration packet; aggregating the data packets to generate the aggregation result, based on a comparison of the aggregation bitmap and the configuration bitmap; and selecting a target address for the first packet.
13. The network apparatus of claim 12, wherein the memory of the network apparatus comprises two or more memory components and a processor of the processing means comprises two or more processor components.
14. The network apparatus of claim 12 or 13, wherein the network apparatus is configured to perform the method of any of claims 1-8.
Citation Information
Patent Citations
Credit-based flow control for multicast packets in lossless ethernet networks
US20140269415A1
Network interface device-based computations
US20220060418A1
Network interface device-based computations
US20230300063A1
In-network compute operations utilizing encrypted communications
US20230379309A1
