NETWORK-INTERNAL SPREAD OF COMPUTING OPERATIONS

Abstract

In one embodiment, a network switch device includes a network interface for receiving vectors from end devices and an aggregation and reduction accelerator for performing element-wise vector splitting operations with the vectors as input, yielding at least two intermediate vector results, wherein the network interface is to perform the following: sending the at least two intermediate vector results to different corresponding network switches in different switch aggregation trees, receiving at least two final vector results of an aggregation and reduction process from the different switch aggregation trees, and combining the at least two final vector results to obtain a combined final vector result, wherein the network interface is to send the combined final vector result to the end devices.

Description

FIELD OF THE INVENTION

[0001] The present invention relates to computer systems and particularly, but not exclusively, to network-internal computing operations. BACKGROUND

[0002] A network node (e.g., containing one or more suitable elements such as a graphics processing unit (GPU) and / or a central processing unit (CPU)) may initiate a parallel computing task with other GPUs over a network. In addition to the other GPUs participating in the parallel computing task, computing resources in switches in the network may also participate in the parallel computing task by providing the various GPUs with the necessary data to execute the parallel computing task and by performing tasks such as addition. The switches may perform reduction processing (e.g., summing two vectors) and aggregation of data received from other network switches and / or GPUs.

[0003] For example, in-network resources can perform reduction operations on vectors, including element-wise vector operations such as vector summation. The vectors can originate from end devices, and the goal of the reduction operation can be to perform a vector operation (e.g., sum) on all vectors. For example, if endpoint 1 has vector 1, endpoint 2 has vector 2, and so on, and endpoint N has vector N, then the goal of the reduction operation can be to sum vectors 1 through N. At each switch, an in-network compute resource aggregates the received vectors and provides the aggregated result to another switch toward the root of a reduction tree. This is repeated at each switch in each layer of the tree until the root of the tree computes the sum of vectors 1 through N based on the vectors it receives from other switches below the root switch.The vector result is then passed down the tree to the end devices until the respective (e.g., every) end device receives the sum of vectors 1 to N.

[0004] An example of aggregation and reduction can be found in US Patent 10,284,383 to Bloch, et al., which describes a switch in a data network that is configured to mediate data exchange between network elements. BRIEF DESCRIPTION OF THE INVENTION

[0005] The invention is defined by the claims. To illustrate the invention, aspects and embodiments are described herein, which may or may not fall within the scope of the claims.

[0006] According to one embodiment of the present disclosure, a network switch device is provided including a network interface for receiving vectors from end devices as part of an aggregation and reduction process, and an aggregation and reduction accelerator for performing element-wise and vector splitting operations with the received vectors as input, resulting in at least two intermediate vector results including a first intermediate vector result and a second intermediate vector result, wherein the network interface is configured to: send the at least two intermediate vector results to different corresponding network switches in different switch aggregation trees,including sending the first intermediate vector result to a first network switch in a first switch aggregation tree and the second intermediate vector result to a second network switch in a second switch aggregation tree, receiving at least two final vector results of the aggregation and reduction process from the various switch aggregation trees, including receiving a first final vector result from the first network switch and a second final vector result from the second network switch, and combining the at least two final vector results to obtain a combined final vector result, wherein the network interface is configured to send the combined final vector result to the end devices.

[0007] According to another embodiment of the present disclosure, the aggregation and reduction accelerator is configured to split the received vectors, resulting in at least two groups of split vectors, and to perform element-wise operations on the split vectors, wherein the at least two groups each result in the at least two intermediate vector results for the at least two groups.

[0008] According to another embodiment of the present disclosure, the aggregation and reduction accelerator is configured to perform element-wise operations on the received vectors yielding a given vector result and to split the given vector result yielding at least two intermediate vector results.

[0009] Additionally, according to an embodiment of the present disclosure, the aggregation and reduction accelerator is configured to perform element-wise operations on the received vectors as input, wherein the element-wise operations are selected from one or more of the following: summation, point-wise minimum, point-wise maximum, bit-wise OR, bit-wise NOR, bit-wise XOR, maximum with index, minimum with index, and a logical operation.

[0010] According to an embodiment of the present disclosure, the aggregation and reduction accelerator is further configured to perform element-wise summation with the received vectors as input, wherein the received vectors have a floating-point data type.

[0011] According to another embodiment of the present disclosure, the network interface is configured to receive the vectors from the end devices in aggregation and reduction protocol packets, and the aggregation and reduction accelerator is configured to perform vector partitioning of packet boundaries.

[0012] According to another embodiment of the present disclosure, the aggregation and reduction accelerator is configured to selectively perform element-wise operations with vector splitting for first received vectors and selectively perform element-wise operations without vector splitting for second received vectors.

[0013] Furthermore, according to an embodiment of the present disclosure, the aggregation and reduction accelerator is configured to determine to selectively perform element-wise operations with vector splitting for first received vectors based on the first received vectors including a floating-point data type, and wherein the element-wise operations include summation.

[0014] According to an embodiment of the present disclosure, the aggregation and reduction accelerator is further configured to determine to selectively perform element-wise operations without vector splitting for second received vectors based on either the second received vectors not having a floating-point data type or the element-wise operations not including summation.

[0015] According to another embodiment of the present disclosure, the aggregation and reduction accelerator is configured to selectively perform element-wise operations with vector splitting for first received vectors and selectively perform element-wise operations without vector splitting for second received vectors in response to an administrator decision.

[0016] According to another embodiment of the present disclosure, in response to a network topology-based decision of the aggregation manager, the aggregation and reduction accelerator is configured to selectively perform element-wise operations with vector splitting for first received vectors and selectively perform element-wise operations without vector splitting for second received vectors.

[0017] In accordance with another embodiment of the present disclosure, a network method is also provided, including: receiving vectors from end devices as part of an aggregation and reduction process, performing element-wise and vector splitting operations with the received vectors as input, resulting in at least two intermediate vector results, comprising a first intermediate vector result and a second intermediate vector result, sending the at least two intermediate vector results to different corresponding network switches in different switch aggregation trees, including sending the first intermediate vector result to a first network switch in a first switch aggregation tree and the second intermediate vector result to a second network switch in a second switch aggregation tree,Receiving at least two final vector results of the aggregation and reduction process from the various switch aggregation trees, including receiving a first final vector result from the first network switch and a second final vector result from the second network switch, combining the at least two final vector results to obtain a combined final vector result, and sending the combined final vector result to the end devices.

[0018] Additionally, according to an embodiment of the present disclosure, the method includes splitting the received vectors resulting in at least two groups of split vectors, and performing element-wise operations on the split vectors, wherein the at least two groups each result in the at least two intermediate vector results for the at least two groups.

[0019] Furthermore, according to an embodiment of the present disclosure, the method includes performing element-wise operations on the received vectors resulting in a given vector result and splitting the given vector result resulting in at least two intermediate vector results.

[0020] According to another embodiment of the present disclosure, the method includes performing element-wise operations on the received vectors as input, wherein the element-wise operations are selected from one or more of the following: summation, point-wise minimum, point-wise maximum, bit-wise OR, bit-wise NOR, bit-wise XOR, maximum value with index, minimum value with index, and a logical operation.

[0021] According to another embodiment of the present disclosure, the method includes performing an element-wise summation with the received vectors as input, wherein the received vectors have a floating point data type.

[0022] According to one embodiment of the present disclosure, receiving vectors from end devices includes receiving the vectors from the end devices in aggregation and reduction protocol packets, and performing includes performing vector splitting packet boundaries.

[0023] Furthermore, according to an embodiment of the present disclosure, the method includes selectively performing element-wise operations with vector splitting for first received vectors and selectively performing element-wise operations without vector splitting for second received vectors.

[0024] According to another embodiment of the present disclosure, the method includes determining selectively performing element-wise operations with vector partitioning for first received vectors based on the first received vectors having a floating point data type, wherein the element-wise operations include summation.

[0025] According to another embodiment of the present disclosure, the method includes determining to selectively perform element-wise operations without vector splitting for second received vectors based on either the second received vectors not having a floating point data type or the element-wise operations not involving summation.

[0026] Additionally, according to an embodiment of the present disclosure, the method includes, in response to an administrator decision, selectively performing element-wise operations with vector splitting for first received vectors and selectively performing element-wise operations without vector splitting for second received vectors.

[0027] Furthermore, according to an embodiment of the present disclosure, the method includes, in response to a decision of the aggregation manager based on a topology of the network, selectively performing element-wise operations with vector splitting for first received vectors, and selectively performing element-wise operations without vector splitting for second received vectors.

[0028] According to another embodiment of the present disclosure, a system is also provided that includes a plurality of network switches including different switch aggregation trees of the network switches with a leaf level of the network switches, wherein each of the network switches in the leaf level is configured to: receive vectors from respective end devices as part of an aggregation and reduction process, perform element-wise and vector splitting operations with the received vectors as input, resulting in intermediate vector results, and send different ones of the intermediate vector results to the network switches in the different switch aggregation trees, wherein the network switches in the different switch aggregation trees are to perform at least a portion of the aggregation and reduction process, resulting in different final vector results,and providing the different final vector results to the network switches in the leaf layer, and wherein each of the network switches in the leaf layer is configured to receive the different final vector results from the different switch aggregation trees, combine the different final vector results to obtain a combined final vector result, and send the combined final vector result to the respective end devices.

[0029] In one embodiment, a network switch device includes a network interface to receive vectors from end devices, and an aggregation and reduction accelerator to perform element-wise and vector splitting operations on the vectors as input, resulting in at least two intermediate vector results, wherein the network interface is to send the at least two intermediate vector results to different corresponding network switches in different switch aggregation trees, receive at least two final vector results of an aggregation and reduction process from the different switch aggregation trees, and combine the at least two final vector results to obtain a combined final vector result, wherein the network interface is to send the combined final vector result to the end devices.

[0030] Any feature of one aspect or embodiment may be applied to other aspects or embodiments in any suitable combination. In particular, any feature of a method aspect or embodiment may be applied to a device aspect or embodiment, and vice versa. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The present invention will become apparent from the following detailed description taken in conjunction with the drawings in which: Fig. 1 is a block diagram of a multi-tree aggregation and reduction system constructed and operative in accordance with an embodiment of the present invention; Fig. Figure 2 is a block diagram view of one of the network switch devices in the system of Fig. 1; Fig. 3 is a flowchart illustrating steps in a method of operating the system of Fig. 1 contains; Fig. 4 is a block diagram view showing a method of element-wise and partitioning operations in the system of Fig. 1 illustrates; Fig. 5 is a block diagram view showing a method of element-wise and partition operations in the system of Fig. 1 illustrates; Fig. 6 is a block diagram view illustrating the combining of vectors in the system of Fig. 1 illustrates; and Fig. 7 is a block diagram view of the system of Fig. 1, which works with a single aggregation and reduction tree. DESCRIPTION OF THE EXEMPLARY EMBODIMENTS OVERVIEW

[0032] The performance of in-network computing operations may be degraded due to physical limitations of the network, depending on its structure and operational requirements. It may be possible to overcome these limitations by changing how applications running on end devices utilize in-network computing resources, but such physical limitations may only apply to certain regions of the network. Furthermore, in many cases (e.g., cloud computing), the system operator may not want to disclose certain information (e.g., the network topology) to the user application. However, it is extremely difficult for the user application to change its use of in-network computing resources to avoid overloading a sensitive part of the network topology if the user application is not aware of the relevant information.

[0033] The following are examples of problems that can affect intra-network computation and the aggregation and reduction process.

[0034] If the vectors to be processed are very large and only a small number of network connections are used, the network connections will be bombarded, creating a local hotspot of contention on the network, which may be undesirable.

[0035] The SHArP aggregation and reduction protocol requires a receive and a transmit buffer to ensure transmission reliability. The buffer size is proportional to the link latency multiplied by the bandwidth share. In some cases, if the cables are too long (resulting in higher latency), SHArP can be compromised, possibly even to the point where its functionality is limited or rendered useless.

[0036] Sometimes the data type used by endpoints (float, double, FP16, FP32, FP64) is not the same as the one used by switches. This is a particular problem with floating-point operations. The data type refers to the number of bits used to represent a number. It is common for endpoints to use 16 bits to represent a single number, while switches use 32 bits to represent values ​​belonging to intermediate computation steps, even though the original values ​​are in 16 bits. This can be done to limit the accumulation of numerical errors. Therefore, every single bit that the endpoints send across the network is converted into two bits by the switches. If the endpoints are operating at full line rate, the network must operate at twice the line rate to keep up, and this is not always feasible.

[0037] Embodiments of the present invention address at least some of the above-mentioned disadvantages by providing an aggregation and reduction system in which vector operations are distributed among (two or more) different aggregation and reduction trees of switches.

[0038] In some embodiments, the switches in the leaf layer of the trees receive vectors from the respective end devices. For example, Switch 1 receives vectors 1-3 from endpoints 1-3, Switch 2 receives vectors 4-5 from endpoints 4-5, and so on. Each of the network switches in the leaf layer performs element-wise and vector splitting operations to obtain intermediate results for the vectors. In some cases, element-wise operations (e.g., summation) are performed before the vector splitting. For example, Switch 1 may perform element-wise vector operations on the vectors received from endpoints 1-3 to obtain a vector result, which is then split into two intermediate vector results. In other cases, the vector splitting is performed before the element-wise operations.In some embodiments, switch 1 may first split the vectors 1-3 received from endpoints 1-3 to obtain two sets of split vectors, and then perform element-wise vector operations on each set of split vectors to obtain two intermediate vector results.

[0039] An elementwise operation on two or more vectors performs operations on the elements of the vectors independently of the other elements of the vectors. For example, the summation of vector A with elements a1, a2, and a3 with vector B with elements b1, b2, and b3 results in a1 + b1, a2 + b2, and a3 + b3.

[0040] Each network switch in the leaf layer then sends different intermediate vector results to the different aggregation and reduction trees for further processing. Switch 1 then sends one of the intermediate vector results to a switch in one of the aggregation and reduction trees and the other intermediate vector result to a switch in the other aggregation and reduction tree.

[0041] The vectors are processed by the switches in the various aggregation and reduction trees and passed up the successive levels of the trees until all vectors are reduced to a single final vector result in the root switch of each of the trees. The final vector results are then passed down in each of the trees until the final vector results are received by the switches in the leaf level. The switches in the leaf level then combine the received final vector results to obtain a combined final vector result, which is returned to the respective end devices. For example, the final vector results 1 and 2 are received by the root switch of the respective tree (i.e.The final vector result 1 is forwarded from the root switch of tree 1, and the final vector result 2 is forwarded from the root switch of tree 2 to switch 1, which combines the final vector results 1 and 2 to obtain a combined final vector result, which switch 1 forwards to the end device 1-3. The above examples refer to two aggregation and reduction trees. Embodiments of the present invention may include any suitable number of aggregation and reduction trees.

[0042] Because the vector operations are performed element by element, the vectors can be split into two or more subvectors (depending on the number of aggregation and reduction trees), processed by the switches in the different trees, and then combined (e.g., concatenated) by the switches in the leaf layer to yield the same final vector result that would have been computed by the switches in a single aggregation and reduction tree. In some embodiments, the splitting operations are based on the packet enumeration of the different vectors, so the vectors are split according to packet boundaries, ensuring that all vectors in all regions of the network are split identically without the need for additional synchronization. In other embodiments, the vectors can be split using any suitable method.

[0043] Because the vectors are split (and recombined) in the switches and not in the end devices, the splitting and recombining process is transparent to the user applications running in the end devices and therefore does not require knowledge of the network topology on the end devices.

[0044] In some embodiments, the splitting of vectors and the use of more than one aggregation and reduction tree to process the split vectors may be applied selectively and depend on various factors, such as: a network administrator enforcing the splitting based on network congestion; a combination of data type and operation type (e.g., summing any floating-point value) may indicate the splitting of vectors, whereas other data types and operation types (e.g., calculating the minimum or maximum of floating-point numbers or summing integers) may not indicate splitting vectors; based on a decision of an aggregation manager (e.g., an entity constructing switch trees and knowing that the trees contain long links and therefore deciding that the vectors should be split to compensate for long links).

[0045] Using SHArP when the buffer requirements cannot be fully met may result in a situation where SHArP is technically functional, but its performance is degraded. Under certain circumstances, the performance degradation may be so great that the performance benefits of using SHARP are completely eliminated or may result in worse performance. Vector splitting reduces the buffer requirement (e.g., halves it) and may enable the use of SHArP without the performance degradation mentioned above. Vector splitting may result in less traffic on network links. Vector splitting can also eliminate the problem caused by the doubling of the size of the data representation due to different data types, as described above.Embodiments of the present invention improve the operation of a computer or other processing device by providing, among other things, better computer performance, faster processing speed, and lower latency. SYSTEM DESCRIPTION

[0046] It will now Fig. 1, which shows a block diagram view of the multi-tree aggregation and reduction system 10 constructed and operative in accordance with an embodiment of the present invention. The system 10 includes end devices 12 and network switches 14 (only a few are labeled for simplicity) arranged in a topology with network links 16 (only a few are labeled for simplicity) connecting the switches 14. Fig. Figure 1 shows two aggregation and reduction trees 18 (18-1 and 18-2) of switches 14. The aggregation and reduction trees 18 are also referred to herein as switch aggregation trees. Each aggregation and reduction tree 18 contains a root 20 and a leaf level 22. The leaf level 22 contains switches 14 that can be contained in both aggregation and reduction trees 18 (as in Fig. 1).

[0047] The switches 14 of the aggregation and reduction tree 18-1 are shown with a striped diagonal fill, while the switches 14 of the other aggregation and reduction tree 18-2 are shown with a checkerboard fill. The switches 14 in the leaf level 22 are shown with a striped diagonal fill section and a checkered fill section, since the switches 14 in the leaf level 22 of Fig. 1 are included in both aggregation and reduction trees 18. The network connections 16 between the switches 14 of the aggregation and reduction tree 18-1 are shown with composite lines, and the network connections 16 between the switches 14 of the aggregation and reduction tree 18-2 are shown with dashed lines.

[0048] The aggregation and reduction trees 18 from Fig. 1 contains 3 layers. The aggregation and reduction trees 18 may contain any number of layers and any number of switches 14, and may span any number of end devices 12. The aggregation and reduction trees 18 may span, for example, hundreds or thousands of end devices 12. Fig. Figure 1 shows two aggregation and reduction trees 18. Three or more aggregation and reduction trees 18 can also be used to perform a single aggregation and reduction process. An even number of aggregation and reduction trees 18 may be more practical than an odd number of aggregation and reduction trees 18.

[0049] Each network switch 14 in the leaf layer 22 is configured to receive vectors from the respective end devices 12 as part of an aggregation and reduction process. Each network switch 14 in the leaf layer 22 is configured to perform element-wise and vector splitting operations with the received vectors as input, resulting in intermediate vector results. In some cases, element-wise operations (e.g., summation) are performed before the vector splitting, as in Fig. 4. In other cases, the vector partitioning is performed before the element-wise operations, as in Fig. 5 described in more detail.

[0050] Each network switch 14 in the leaf layer 22 is configured to send different intermediate vector results to the network switches 14 in the different switch aggregation trees 18. For example, one of the intermediate vector results is delivered to the aggregation and reduction tree 18-1 and another intermediate vector result is delivered to the aggregation and reduction tree 18-2.

[0051] The switches 14 in the various switch aggregation trees 18 are configured to perform at least part of the aggregation and reduction process, resulting in different final vector results, and to deliver the different final vector results to the network switches 14 in the leaf level 22. Specifically, the intermediate results of the vectors are processed by the switches 14 in the various aggregation and reduction trees 18 and propagated up successive levels of the trees 18 until all vectors in the root switch 20 of each of the trees 18 are reduced to a single final vector result. The final vector results are then propagated down each of the trees 18 until the final vector results are received by the switches 14 in the leaf level 22.

[0052] Each network switch 14 in the leaf layer 22 is configured to receive the various final vector results from the various switch aggregation trees 18 and combine the various final vector results to obtain a combined final vector result. Each network switch 14 in the leaf layer 22 is configured to send the combined final vector result to the respective end devices 12.

[0053] Since the vector operations are performed element-wise, the vectors can be split into two or more subvectors (depending on the number of aggregation and reduction trees), processed by the switches 14 in the different trees 18, and then combined (e.g., concatenated) by the switches 14 in the leaf level 22 to obtain the same final vector result that would have been computed by switches in a single aggregation and reduction tree.

[0054] It will now Fig. 2, which is a block diagram view of one of the network switch devices 14 in the system 10 of Fig. 1. The switch 14 includes a network interface 24 with ports (not shown) configured to receive and transmit packets to other switches 14 and optionally to the end devices 12 (if directly connected to one or more of the end devices 12). The packets may include aggregation and reduction protocol packets. A vector may include multiple aggregation and reduction protocol packets.

[0055] The switch 14 also includes switching circuits 26 for establishing appropriate connections between the ports of the received packets and the ports of the packets to be transmitted. The switch 14 also includes an aggregation and reduction accelerator 28 for performing element-wise and vector partitioning operations with received vectors as input. The aggregation and reduction accelerator 28 is described with reference to Fig. 3-6 are described in more detail.

[0056] In practice, some or all of the functions of the aggregation and reduction accelerator 28 may be combined in a single physical component or, alternatively, implemented using multiple physical components. These physical components may comprise hard-wired or programmable devices, or a combination of both. In some embodiments, at least a portion of the functions of the aggregation and reduction accelerator 28 may be performed by a programmable processor under the control of suitable software. This software may be downloaded to a device in electronic form, for example, over a network. Alternatively or additionally, the software may be stored in tangible, non-transitory, computer-readable storage media, such as optical, magnetic, or electronic memory.

[0057] It will now Fig. 3, which is a flowchart 100 illustrating steps in a method for operating the system 10 of Fig. 1 contains. Fig. Figure 3 describes the processing within one of the switches 14 in the leaf layer 22. The network interface 24 of the switch 14 is configured to receive vectors from end devices 12 as part of an aggregation and reduction process (block 102). The end devices 12 from which the network interface 24 receives vectors may be defined by the aggregation and reduction process and may include all or a subset of the end devices 12 directly connected to the switch 14. In some embodiments, the network interface 24 is configured to receive vectors from the end devices 12 in aggregation and reduction protocol packets. The network interface 24 is generally configured to receive one or more vectors from each of the end devices 12 as part of an aggregation and reduction process.Each vector is generally carried in more than one packet, so that when the vector is split in the switch 14 or when a vector result is split in the switch 14, the splitting is done according to the packet boundaries, as described below with reference to FIG. Fig. 5 described in more detail.

[0058] In some embodiments, all vectors received by switches 14 are split at leaf level 22, and the split vectors are processed by different aggregation and reduction trees 18. In some embodiments, the splitting of the vectors and the use of more than one aggregation and reduction tree 18 to process the split vectors may be applied selectively and depend on various factors, so that in some cases, vectors are not split and only one aggregation and reduction tree 18 is used to process the vectors.

[0059] The factors that may be considered to determine whether or not to selectively split received vectors and use more than one aggregation and reduction tree 18 include factors such as: a network administrator may force a split based on network congestion; a combination of data type and operation type (e.g., summing any floating-point value) may indicate splitting of vectors, whereas other data types and operation types (e.g., calculating the minimum or maximum of floating-point numbers or summing integers) may not indicate splitting vectors; based on a decision of an aggregation manager (e.g., an entity constructing switch trees knows that the trees contain long links and therefore decides that the vectors should be split to compensate for long links).Therefore, in some embodiments, the aggregation and reduction accelerator 28 is configured to determine whether to perform a split (block 104).

[0060] In some embodiments, the aggregation and reduction accelerator 28 is configured to selectively perform element-wise vector splitting operations on some received vectors (described with reference to Fig. 4-6 and the steps of block 108-116 of Fig. 3) and selectively performs element-wise operations without vector splitting for other received vectors (described with reference to Fig. 7 and the steps of block 118-124 of Fig. 3).

[0061] One of the above factors is the data type and the operation type. Therefore, in some embodiments, the aggregation and reduction accelerator 28 is configured, at a decision block 106, to: determine to selectively perform element-wise operations with vector splitting for received vectors based on the received vectors having a floating-point data type and the element-wise operations including summation; and determine to selectively perform element-wise operations without vector splitting for other received vectors based either on the other received vectors not having a floating-point data type or on the element-wise operations not including summation.

[0062] In some embodiments, the aggregation and reduction accelerator 28 is configured, in response to an administrator decision, to selectively perform element-wise vector splitting operations on some received vectors and selectively perform element-wise non-vector splitting operations on other received vectors. In some embodiments, the aggregation and reduction accelerator 28 is configured, in response to a decision of the aggregation manager based on a topology of the network, to selectively perform element-wise vector splitting operations on some received vectors and selectively perform element-wise non-vector splitting operations on other received vectors. In some embodiments, the decision of the aggregation manager may determine that all received vectors undergo element-wise vector splitting operations.

[0063] It will now Fig. 4, which is a block diagram view illustrating a method for element-wise operations and partition operations in the system 10 of Fig. 1. It is also shown on Fig. 3. In general, the aggregation and reduction accelerator 28 of the switch 14 in the leaf layer 22 is configured to perform element-wise and vector partitioning operations on received vectors 30 (e.g., V1 and V2) as input, which include at least two intermediate vector results 32, including a first intermediate vector result (e.g., INT V1) and a second intermediate vector result (e.g., INT V2) (block 108).

[0064] In some cases, element-wise operations (e.g. summation) are performed before vector partitioning, as shown below in Fig. 4. In other cases, the vector partitioning is performed before the element-wise operations, as described below in Fig. 5 described in more detail.

[0065] The aggregation and reduction accelerator 28 is configured to perform element-wise operations (block 36) on the received vectors 30 that yield a particular vector result 34. In one example, the aggregation and reduction accelerator 28 is configured to perform an element-wise summation with the received vectors as input, where the received vectors have a floating-point data type. The element-wise operations may be selected from one or more of the following: summation, point-wise minimum, point-wise maximum, bitwise OR, bitwise NOR, bitwise XOR, maximum with index, minimum with index, and a logical operation.

[0066] The aggregation and reduction accelerator 28 is configured to split the given vector result 34 (block 38), resulting in the intermediate vector results 32 (e.g., INV V1 and INV V2). In some embodiments, the aggregation and reduction accelerator 28 performs the vector split of the vector result 34 according to the packet boundaries defined by the packets containing the received vectors 30. In other embodiments, the split of the vector result 34 may be performed using any suitable method.

[0067] The network interface 24 of the switch 14 is configured to send the intermediate vector results 32 to various corresponding network switches 14 in different switch aggregation trees 18 (block 110), which includes sending the first intermediate vector result (e.g., INT V1) to the network switch 14-1 in the aggregation and reduction tree 18-1 and the second intermediate vector result (e.g., INT V2) to the network switch 14-2 in the aggregation and reduction tree 18-2.

[0068] It will now Fig. 5, which is a block diagram view illustrating another method for element-wise operations and partition operations in the system 10 of Fig. 1. The aggregation and reduction accelerator 28 of the switch 14 in the leaf layer 22 is configured to split the received vectors 30 (blocks 38) so that at least two groups 40 of split vectors 42 (e.g., SV1, SV2, SV3, SV4) are created. In the example of Fig. 5, V1 is split into SV1 and SV2, and V2 is split into SV3 and SV4. One of the groups 40 of split vectors 42 contains SV1 and SV3, while another of the groups 40 of split vectors 42 contains SV2 and SV4.

[0069] In some embodiments, the aggregation and reduction accelerator 28 of the switch 14 in the leaf layer 22 is configured to perform vector splitting according to the packet boundaries of the received vectors 30 (e.g., V1 and V2). For streaming aggregations, each vector may be carried by multiple packets containing a header and a tail packet, as well as one or more body packets between the header and tail packets. All packet types may contain vector data. However, header and tail packets may contain other data (e.g., data identifying the aggregation and reduction process) that must be duplicated in packets sent to each of the aggregation and reduction trees 18. In other embodiments, the splitting of the received vectors 30 may be performed using any suitable method.

[0070] The aggregation and reduction accelerator 28 is configured to perform element-wise operations (block 36) on the partitioned vectors 42 corresponding to the groups 40, which yield the intermediate vector results 32 for the groups 40. In the example of Fig. 5, the aggregation and reduction accelerator 28 is configured to perform element-wise operations (block 36) on SV1 and SV3 resulting in INT V1, and element-wise operations on SV2 and SV4 resulting in INT V2.

[0071] The network interface 24 of the switch 14 is configured to send the intermediate vector results 32 to various corresponding network switches 14 in various switch aggregation trees 18 (block 110), including sending the first intermediate vector result (e.g., INT V1) to the network switch 14-1 in the aggregation and reduction tree 18-1 and the second intermediate vector result (e.g., INT V2) to the network switch 14-2 in the aggregation and reduction tree 18-2.

[0072] It will now Fig. 6, which is a block diagram view illustrating the combining of vectors in the system 10 of Fig. 1. The aggregation and reduction accelerator 28 of the switch 14 in the leaf layer 22 is configured to receive (via the network interface 24 of the switch 14) at least two final vector results 44 of the aggregation and reduction process from the various switch aggregation trees 18, including receiving a first final vector result (e.g., FIN V1) from the network switch 14-1 of the aggregation and reduction tree 18-1 and a second final vector result (e.g., FIN V2) from the second network switch 14-2 of the aggregation and reduction tree 18-2 (block 112).

[0073] The aggregation and reduction accelerator 28 is configured to combine the final vector results 44 (block 46) to obtain a combined final vector result 48 (e.g., COM V) (block 114). The network interface 24 is configured to transmit the combined final vector result 48 to the end devices 12 (block 116) from which the original vectors 30 were received.

[0074] It will now Fig. 7, which is a block diagram view of the system 10 of Fig. 1, which works with a single aggregation and reduction tree 18. It is also shown on Fig.3. As previously mentioned, in some embodiments, the splitting of the vectors and the use of more than one aggregation and reduction tree 18 to process the split vectors may be applied selectively and depend on various factors, so that in some cases, vectors are not split and only one aggregation and reduction tree 18 is used to process the vectors. If the decision is made not to split the vectors in the switches 14 (only a few are labeled for simplicity) of the leaf level 22, the following steps are performed.The aggregation and reduction accelerator 28 of each switch 14 in the leaf layer 22 is configured to: perform element-wise operations on the received given vectors (received from the respective end devices 12) that yield an intermediate vector result (block 118); send the intermediate vector result to one of the switches 14 in the layer above the leaf layer 22 in the aggregation and reduction tree 18 (block 120); receive a final vector result from the switch 14 in the layer above the leaf layer 22 in the aggregation and reduction tree 18 (block 122); and send the final vector result to the end devices 12 from which the given vectors were previously received by the switch 14 in the leaf layer 22 (block 124).

[0075] Various features of the invention that are described in conjunction with individual embodiments for clarity may also be provided in combination in a single embodiment. Conversely, various features of the invention that are described in conjunction with a single embodiment for brevity may also be provided individually or in any suitable subcombination.

[0076] The embodiments described above are given only by way of example, and the present invention is not limited to what has been specifically shown and described herein. Rather, the scope of the invention includes combinations and subcombinations of the various features described herein, as well as variations and modifications thereof that would occur to those skilled in the art upon reading the above description and that are not disclosed in the prior art.

[0077] It is to be understood that the aspects and embodiments described above are purely exemplary and that changes in detail may be made within the scope of the claims.

[0078] Each device, method, and feature disclosed in the description and (optionally) in the claims and drawings may be presented independently or in any suitable combination.

[0079] Reference signs in the claims are for illustrative purposes only and do not limit the scope of the claims. QUOTES CONTAINED IN THE DESCRIPTION

[0000] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited patent literature

[0000] US 10,284,383

[0004]

Claims

[1] Network switch device comprising: a network interface for receiving vectors from end devices as part of an aggregation and reduction process; and an aggregation and reduction accelerator for the following: Performing element-wise and vector partitioning operations with the received vectors as input, resulting in at least two intermediate vector results including a first intermediate vector result and a second intermediate vector result, wherein the network interface is to send the at least two intermediate vector results to different corresponding network switches in different switch aggregation trees, including sending the first intermediate vector result to a first network switch in a first switch aggregation tree and the second intermediate vector result to a second network switch in a second switch aggregation tree; Receiving at least two final vector results of the aggregation and reduction process from the various switch aggregation trees, including receiving a first final vector result from the first network switch and a second final vector result from the second network switch; and Combining the at least two final vector results to obtain a combined final vector result, wherein the network interface is to send the combined final vector result to the end devices. [2] The apparatus of claim 1, wherein the aggregation and reduction accelerator performs the following: Splitting the received vectors, resulting in at least two groups of split vectors; and Performing element-wise operations on the split vectors according to the at least two groups, yielding the at least two intermediate vector results for the at least two groups. [3] The apparatus of claim 1, wherein the aggregation and reduction accelerator performs the following: Performing element-wise operations on the received vectors that yield a given vector result; and Splitting the given vector result, yielding at least two intermediate vector results. [4] The apparatus of claim 1, wherein the aggregation and reduction accelerator is to perform element-wise operations on the received vectors as input, the element-wise operations being selected from one or more of the following: summation; point-wise minimum; point-wise maximum; bit-wise OR, bit-wise NOR, bit-wise XOR, maximum with index, minimum with index, and a logical operation. [5] The apparatus of claim 1, wherein the aggregation and reduction accelerator is to perform element-wise summation with the received vectors as input, the received vectors having a floating-point data type. [6] Device according to one of the preceding claims, wherein: the network interface shall receive the vectors from the end devices in aggregation and reduction protocol packets; and the aggregation and reduction accelerator divides the vectors according to the packet boundaries. [7] Apparatus according to any one of the preceding claims, wherein the aggregation and reduction accelerator is to selectively perform element-wise operations with vector splitting for first received vectors and selectively perform element-wise operations without vector splitting for second received vectors. [8] The apparatus of claim 7, wherein the aggregation and reduction accelerator determines the selective performance of element-wise vector splitting operations for first received vectors based on: the first vectors received have a floating-point data type; and the element-wise operations contain a summation. [9] The apparatus of claim 8, wherein the aggregation and reduction accelerator determines the selective performance of element-wise operations without vector splitting for second received vectors based on: the second vectors received do not have a floating-point data type; or the element-wise operations do not contain summation. [10] The apparatus of claim 1, wherein the aggregation and reduction accelerator is to selectively perform element-wise operations with vector splitting for first received vectors and selectively perform element-wise operations without vector splitting for second received vectors in response to an administrator decision. [11] The apparatus of claim 1, wherein the aggregation and reduction accelerator is to selectively perform element-wise operations with vector splitting for first received vectors and selectively perform element-wise operations without vector splitting for second received vectors in response to a decision of the aggregation manager based on a topology of the network. [12] Network procedure that includes: Receiving vectors from end devices as part of an aggregation and reduction process; Performing element-wise and vector partitioning operations with the received vectors as input, resulting in at least two intermediate vector results including a first intermediate vector result and a second intermediate vector result, Sending the at least two intermediate vector results to different corresponding network switches in different switch aggregation trees, including sending the first intermediate vector result to a first network switch in a first switch aggregation tree and the second intermediate vector result to a second network switch in a second switch aggregation tree; Receiving at least two final vector results of the aggregation and reduction process from the various switch aggregation trees, including receiving a first final vector result from the first network switch and a second final vector result from the second network switch; Combining the at least two final vector results to obtain a combined final vector result; and Sending the combined final vector result to the end devices. [13] The method of claim 12, further comprising: Splitting the received vectors, resulting in at least two groups of split vectors; and Performing element-wise operations on the split vectors according to the at least two groups, yielding the at least two intermediate vector results for the at least two groups. [14] The method of claim 12, further comprising: Performing element-wise operations on the received vectors that yield a given vector result; and Splitting the given vector result, yielding at least two intermediate vector results. [15] The method of claim 12, further comprising performing element-wise operations on the received vectors as input, wherein the element-wise operations are selected from one or more of the following: summation; point-wise minimum; point-wise maximum; bit-wise OR, bit-wise NOR, bit-wise XOR, maximum with index, minimum with index, and a logical operation. [16] The method of claim 12, further comprising performing an element-wise summation with the received vectors as input, wherein the received vectors have a floating-point data type. [17] A method according to any one of claims 12 to 16, wherein: receiving vectors from end devices includes receiving the vectors from the end devices in aggregation and reduction protocol packets; and performing the vector partitioning according to the packet boundaries. [18] A method according to any one of claims 12 to 17, further comprising: selectively performing element-wise operations with vector partitioning for first received vectors; and selectively performing element-wise operations without vector splitting for second received vectors. [19] The method of claim 18, further comprising determining the selective performance of element-wise vector splitting operations for first received vectors based on: the first vectors received have a floating-point data type; and the element-wise operations contain a summation. [20] The method of claim 19, further comprising determining the selective performance of element-wise operations without vector splitting for second received vectors based on: the second vectors received do not have a floating-point data type; or the element-wise operations do not contain summation. [21] The method of claim 12, further comprising, in response to a decision by the administrator: selectively performing element-wise operations with vector partitioning for first received vectors; and selectively performing element-wise operations without vector splitting for second received vectors. [22] The method of claim 12, further comprising, in response to a decision of the aggregation manager based on a topology of the network: selectively performing element-wise operations with vector partitioning for first received vectors; and selectively performing element-wise operations without vector splitting for second received vectors. [23] A system comprising a plurality of network switches containing different switch aggregation trees of the network switches having a leaf level of the network switches, where: each of the network switches in the leaf layer performs the following: Receiving vectors from respective end devices as part of an aggregation and reduction process; Performing element-wise and vector partitioning operations on the received vectors as input, resulting in intermediate vector results; and Sending various of the intermediate vector results to the network switches in the various switch aggregation trees; the network switches in the different switch aggregation trees should perform at least part of the aggregation and reduction process, resulting in different final vector results, and should deliver the different final vector results to the network switches in the leaf level; and each of the network switches in the leaf layer performs the following: Receiving the various final vector results from the various switch aggregation trees; Combining the different final vector results to obtain a combined final vector result; and Sending the combined final vector result to the respective end devices.

Images