Turn-based deadlock-free routing in Cartesian topology

Abstract

An apparatus includes an interface and a processor. The interface communicates with a network including network elements interconnected in a Cartesian topology. The processor defines a first set of turns and a second set of turns, each turn including a hop from a previous network element to a current network element and a hop from the current network element to a next network element. Based on the turns, the processor specifies rules that, when applied to packets traversing the various network elements, ensure that deadlock conditions do not occur in the network. The rules for a given network element include (i) a forwarding rule to reach a given destination without traversing the turns of the second set, and (ii) a virtual lane (VL) modification rule for reallocating packets that traverse the turns of the first set and are assigned to a first VL to a different second VL. The processor is configured with the rules for a given network element.

Description

Technical Field

[0001] Embodiments described herein relate generally to data communications, and more particularly to methods and systems for deadlock-free routing in a network having a Cartesian topology. Background Art

[0002] A communication network typically includes multiple network elements, such as switches or routers, that are interconnected with each other. Network elements typically buffer incoming packets before sending them to a selected next-hop network element and employ flow control measures to prevent the previous-hop network element from causing buffer overflows. When the buffers of multiple network elements with cyclic dependencies become full, a deadlock situation may occur in the network. Deadlock situations are likely to occur in Cartesian network topologies, such as mesh, torus, hypercube, and HyperX topologies.

[0003] Packet routing methods that avoid deadlock situations are known in the art. For example, U.S. Patent Application Publication 2019 / 0058651 describes routing packets using distance levels in a multidimensional network. A packet is received at a network device in a fully connected multidimensional network, and all possible candidate output ports for the packet to be routed to a destination network device are determined. The candidate output ports correspond to candidate minimum paths and candidate non-minimum paths between the network device and the destination network device along all remaining unaligned dimensions of the multidimensional network. The best output port is selected among all candidate output ports, and the packet is routed to the next hop in the network via the best output port and using the next distance level.

[0004] U.S. Patent No. 10 / 404,574 describes an apparatus including a network interface and a processor. The network interface is configured to communicate with a network including a plurality of switches interconnected in a Cartesian topology having a plurality of dimensions. The processor is configured to predefine an order between the dimensions of the Cartesian topology to search for a preferred route from a source switch to a destination switch via the network by evaluating candidate routes based at least on a corresponding number of switches along a candidate route that changes from one dimension to another dimension opposite to the predefined order to a next-hop switch, and configure one or more switches in the network to route packets from the source switch to the destination switch along the preferred route.

[0005] In the paper titled “HyperX: topology, routing, and packaging of efficient large-scale networks,” published in the Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis, November 14-20, 2009, the authors consider extensions of the hypercube and flat butterfly topologies, HyperX, and present an adaptive routing algorithm, DAL. Summary of the invention

[0006] Embodiments described herein provide an apparatus comprising a network interface and a processor. The network interface is configured to communicate with a network comprising a plurality of network elements interconnected in a Cartesian topology. The processor is configured to define a first set of turns and a second set of turns in the topology, wherein each turn in the first set and each turn in the second set comprises an input hop from a previous network element to a current network element and an output hop from the current network element to a next network element. Based on the turns in the first set and the second set, the processor is configured to specify rules that, when applied to packets traversing the corresponding network elements, ensure that no deadlock occurs in the network. The rules specified for a given network element include (i) forwarding rules from one or more input ports to one or more output ports to reach a given target coupled to the network without traversing any of the turns in the second set, and (ii) virtual channel (VL) modification rules for reallocating packets that traverse the turns of the first set and are assigned to a first VL to a different second VL. The processor is also configured to configure the given network element with the rules.

[0007] In some embodiments, the processor is configured to identify a given turn that occurs when traversing the given network element from an input port to an output port, and (i) specify at least one of the rules for packets assigned to the first VL depending on whether the given turn belongs to the first group, and (ii) specify at least one of the rules for packets assigned to the second VL depending on whether the given turn belongs to the second group. In other embodiments, the processor is configured to specify a forwarding rule that prohibits the given turn in response to detecting that the given turn belongs to the second group. In other embodiments, the processor is configured to specify a forwarding rule that prohibits the given turn in response to detecting that the packet traversing the given turn belongs to the first group.

[0008] In one embodiment, the processor is configured to, in response to detecting that the given turn belongs to the first group, specify (i) a forwarding rule that allows the given turn and (ii) a corresponding VL modification rule that modifies the VL in the group to the second VL. In another embodiment, the processor is configured to specify a forwarding rule that allows a turn from an input port to an output port only if traversing the turn reduces the number of remaining hops to the given destination. In yet another embodiment, the processor is configured to specify a forwarding rule that allows a turn that does not reduce the number of remaining hops to the given destination but satisfies a specified total number of hops per path.

[0009] In some embodiments, the processor is configured to, in response to detecting that the path from the given network element to the destination must contain a turn belonging to the second group, specify a forwarding rule that prohibits traversal of the path. In other embodiments, the Cartesian topology includes a Cartesian topology selected from a list of Cartesian topologies, the Cartesian topology including the mesh, torus, hypercube, generalized hypercube, and HyperX topologies. In other embodiments, the Cartesian HyperX topology includes up to three dimensions, and wherein the processor is configured to specify the rule so as to ensure that between any pair of network elements in the network, at least one path is at most three hops. In other embodiments, the processor is configured to specify a VL modification rule that prohibits modification of the VL of a packet assigned to the second VL.

[0010] According to embodiments described herein, a method is additionally provided, including, in a network manager in communication with a network, the network including a plurality of network elements interconnected in a Cartesian topology, defining a first set of turns and a second set of turns in the topology, wherein each turn in the first set and each turn in the second set include an input hop from a previous network element to a current network element and an output hop from the current network element to a next network element. Based on the turns in the first set and the second set, specifying rules that, when applied to packets traversing the corresponding network elements, ensure that no deadlock condition occurs in the network, wherein the rules specified for a given network element include (i) forwarding rules from one or more input ports to one or more output ports to reach a given destination coupled to the network without traversing any of the turns in the second set, and (ii) virtual lane (VL) modification rules for reallocating packets that traverse the turns in the first set and are assigned to a first VL to a different second VL. The given network element is configured with the rules.

[0011] According to another embodiment described herein, there is further provided a communication network comprising a plurality of network elements interconnected in a Cartesian topology, wherein the network elements are configured to forward packets according to rules that, when applied to packets traversing the corresponding network elements, ensure that no deadlock condition occurs in the network. The rules are specified based on a first set of turns and a second set of turns in the Cartesian topology, each turn in the first set and each turn in the second set comprising an input hop from a previous network element to a current network element and an output hop from the current network element to a next network element. The rules specified for a given network element include (i) forwarding rules from one or more input ports to one or more output ports to reach a given destination coupled to the network without traversing any of the turns in the second set, and (ii) virtual channel (VL) modification rules for reallocating packets that traverse the turns of the first set and are assigned to a first VL to a different second VL.

[0012] According to the embodiments described herein, there is further provided a communication method, including forwarding packets according to rules by the network elements in a communication network including a plurality of network elements interconnected in a Cartesian topology, and ensuring that no deadlock situation occurs in the network when the rules are applied to packets traversing the corresponding network elements. The rules are specified based on a first set of turns and a second set of turns in the Cartesian topology, each turn in the first set and each turn in the second set including an input hop from a previous network element to a current network element and an output hop from the current network element to a next network element. The rules specified for a given network element include (i) forwarding rules from one or more input ports to one or more output ports to reach a given destination coupled to the network without traversing any of the turns in the second set, and (ii) virtual channel (VL) modification rules for reallocating packets that traverse the turns of the first set and are assigned to a first VL to a different second VL.

[0013] These and other embodiments will be more fully understood from the following detailed description of embodiments thereof, taken in conjunction with the accompanying drawings, in which: BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1A is a block diagram schematically illustrating a computing system according to an embodiment described herein, wherein network nodes communicate via a network having a Cartesian three-dimensional (3D) HyperX topology;

[0015] Figure 1B is a block diagram schematically illustrating a computing system according to another embodiment described herein, wherein network nodes communicate through a network having a Cartesian three-dimensional (3D) network topology;

[0016] Figure 2 is a block diagram schematically illustrating a network switch according to embodiments described herein;

[0017] Figure 3 is a diagram schematically illustrating dependency cycles across two dimensions according to embodiments described herein;

[0018] Figure 4 is a diagram schematically illustrating dependency cycles across three dimensions according to embodiments described herein; and

[0019] Figure 5 is a flow chart schematically illustrating a method for specifying forwarding rules for switches in a network having a Cartesian topology according to an embodiment described herein. DETAILED DESCRIPTION

[0020] Overview

[0021] Packets traversing a communication network are often temporarily buffered in switches along the route. In networks that employ lossless link-level flow control, switches often send flow control notifications to their previous-hop neighbor switches indicating the status of the switch's buffers. The previous-hop switches restrict packet transmission on the relevant links based on these notifications. In some practical situations, the buffers of multiple switches interconnected in a loop become full, rendering these switches unable to further send or receive packets. For brevity, this undesirable situation is referred to as a "deadlock situation" or simply "deadlock."

[0022] In some networks, switches are arranged in a Cartesian topology, such as a mesh, torus, hypercube, generalized hypercube, or HyperX. Switches in a Cartesian topology are interconnected in a virtual grid and can be identified by their respective Cartesian coordinates. A Cartesian topology can have any suitable dimensionality, such as one-dimensional (1D), two-dimensional (2D), three-dimensional (3D), or higher dimensions.

[0023] Embodiments described herein provide improved methods and systems for deadlock-free routing in networks having a Cartesian topology. In principle, dependency cycles can be broken to prevent deadlock by (i) preventing routing via at least one of the switches participating in the dependency cycle, and (ii) modifying the VL of a packet traversing one of the switches in the dependency cycle. However, limiting the use of paths in the network reduces path diversity and routing flexibility, and may exclude the shortest paths between network elements. Furthermore, using a large number of VLs requires large buffer storage space, which is expensive and not always available.

[0024] In the disclosed embodiment, deadlock-free routing is guaranteed for Cartesian 3D topologies (and lower dimensions) using only two VLs. In addition, for certain topologies, such as Cartesian HyperX topologies, shortest paths can be guaranteed to be generated. For example, in a 3D HyperX topology, all paths with a length of at most three hops (in this case, three is also the network diameter) are guaranteed to be generated, including all shortest paths.

[0025] In some embodiments, a network management computer, also referred to as a "network manager," is connected to the network and used to configure the network switches.

[0026] To this end, the network manager defines a plurality of turn types in a Cartesian topology, wherein each turn traverses a first hop along a first dimension, followed by a second hop along a second dimension. The first dimension and the second dimension may have the same or different respective identities, and the first hop and the second hop may have the same or different directions. Each turn type is defined by the identity and direction of the first hop and the second hop. Based on the turn type, the network manager specifies rules provided to network elements to control packet forwarding, and VL allocations for packets traversing the network, while avoiding deadlock conditions.

[0027] Consider an embodiment of an apparatus (e.g., a network manager) including a network interface and a processor. The network interface communicates with a network including a plurality of network elements interconnected in a Cartesian topology. The processor defines a first set of turns and a second set of turns in the topology, wherein each turn in the first set and each turn in the second set include an input hop from a previous network element to a current network element and an output hop from the current network element to a next network element. Based on the turns in the first set and the second set, the processor specifies rules that, when applied to packets traversing the corresponding network elements, ensure that no deadlock condition occurs in the network. The rules specified for a given network element include (i) forwarding rules from one or more input ports to one or more output ports to reach a given destination coupled to the network without traversing any turns of the second set, and (ii) virtual lane (VL) modification rules for reallocating packets that traverse the turns of the first set and are assigned to a first VL to a different second VL. The processor is configured with the specified rules for the given network element.

[0028] In some embodiments, the processor identifies a given turn that occurs when traversing a given network element from an input port to an output port, and (i) specifies at least one rule for packets assigned to the first VL depending on whether the given turn belongs to a first group, and (ii) specifies at least one rule for packets assigned to a second VL depending on whether the given turn belongs to a second group.

[0029] In some implementations, the processor avoids modifying the VL of the packet assigned to the second VL, in which case only two VLs are used.

[0030] The processor may specify forwarding rules in a variety of ways. For example, the processor may specify rules by which all turns in the first group and the second group are prohibited independently of the group VL. As another example, the processor may specify rules for groups assigned to the first VL according to which turns in the first group are allowed, but when occupied, the group VL must be increased to the second VL. In this example, for groups assigned to the second VL, turns in the second group are prohibited.

[0031] In some embodiments, to ensure the shortest path, the processor specifies a forwarding rule that allows a turn from an input port to an output port only if traversing the turn reduces the number of hops remaining to a given destination. Alternatively, the processor may specify a forwarding rule that allows turns that do not reduce the number of hops remaining to a given destination but meet a specified total number of hops per path. This limits the maximum path length in the network.

[0032] In some embodiments, the only available path from the network element to the destination contains a forbidden turn. This means that a packet arriving at a given switch and destined for the destination cannot be forwarded further. To prevent such an event, the processor examines possible paths to the destination and, in response to detecting that the path from the given network element to the destination must contain a turn belonging to the second group, specifies a forwarding rule that prohibits traversing the path.

[0033] The disclosed embodiments are applicable to Cartesian topologies, including Cartesian HyperX topologies. For HyperX topologies with up to three dimensions, the processor may specify rules to ensure that there is at least one path of up to three hops between any pair of network elements in the network. Note that the processor may also generate longer paths.

[0034] In the disclosed technology, rules for ensuring deadlock-free routing in a Cartesian topology are specified based on two sets of predefined turns. Turns in the first set are allowed for packets assigned to the first VL only when the VL is modified to the second VL. Turns in the second set are prohibited for packets assigned to the second set. In 3D or lower dimensional topologies, the derived rules only need to use two VLs, thereby saving buffer storage space. In addition, in some topologies (such as Cartesian HyperX topology), the derived rules result in a high degree of path diversity and guarantee the shortest path between any two network elements in the network.

[0035] System Description

[0036] Figure 1A is a block diagram schematically illustrating a computing system 20 according to embodiments described herein, wherein network nodes communicate via a network having a Cartesian three-dimensional (3D) HyperX topology.

[0037] The computing system 20 includes a plurality of host computers 24 that communicate with each other via a network 30 and thus serve as network nodes of the network. The network nodes 24 are typically connected to the network 30 via respective network interface controllers (NICs) (not shown). Figure 1A In the diagram, a network node is represented as “ND”.

[0038] Network 30 (and Figure 1B The network 31 may include any suitable communication network, such as an InfiniBand (IB) switching structure, or other types of packet networks, such as Ethernet or Internet Protocol (IP) networks. Alternatively, the network 30 may operate according to any other suitable standard or protocol.

[0039] The network 30 typically includes a plurality of network elements 32 that are interconnected via physical links 34 according to some predefined network topology. The network elements 32 may include, for example, switches, routers, bridges, gateways, or any other suitable type of network element. Figure 1A In the fully connected Cartesian topology of the HyperX topology depicted in , each network node 24 is connected to a corresponding switch in the network using a physical link 34 and is able to communicate with all other network nodes through the network.

[0040] The network 30 may include one or more switches that are interconnected with other switches in the network but are not connected to any network node 24. Figure 1A In the example shown in FIG. 1 , each switch is connected to two network nodes. However, in practice, each switch in a Cartesian topology can be connected to any other suitable number of network nodes. Furthermore, different switches can be connected to different numbers of network nodes. Typically, the number of network nodes is limited by the number of switch ports that are not used to connect to other switches.

[0041] Switches in the network 30 that are directly interconnected using physical links 34 are referred to as "neighbor switches." A packet from a given switch to a neighbor switch traverses one "hop" in the underlying topology. In a Cartesian topology, switches are assigned Cartesian coordinates, which associate the switches with corresponding vertices on the underlying Cartesian grid. Each pair of vertices is separated along a given dimension in one grid cell (e.g., a grid cell is equal to the integer "1"). For example, in a 3D Cartesian topology, Cartesian coordinates can be specified by three integers (Cx, Cy, Cz). For example, the Cartesian coordinates (0,1,2) differ from the origin Cartesian coordinates (0,0,0) by one grid cell in the Y dimension and two grid cells in the Z dimension. In a Cartesian topology, the Cartesian coordinates of two adjacent switches that are directly interconnected differ in one or more grid cells only in one and only one topological dimension.

[0042] For example, the Cartesian coordinates of two adjacent switches in a mesh or hypercube topology differ by only one grid cell in one of the topological dimensions. In contrast, in torus, generalized hypercube, and HyperX topologies, the Cartesian coordinates of two connected switches may differ in one grid cell or in multiple grid cells in one of the topological dimensions. For example, in the HyperX topology, the switch at the origin coordinate (0,0,0) is directly connected to the switch at the coordinate (0,0,2), but not directly connected to the switch at the coordinate (0,1,1). In a Cartesian topology, the number of network elements in each dimension can be common across all dimensions, or can vary between different dimensions.

[0043] In the following description, for the sake of brevity, "Cartesian coordinates" are also simply referred to as "coordinates".

[0044] The embodiments described below are applicable to various Cartesian topologies, including mesh, torus, hypercube, generalized hypercube, and HyperX. It is assumed that each switch in a D-dimensional Cartesian topology is associated with a D-dimensional coordinate of the underlying virtual grid. In a mesh topology, switches are arranged in a D-dimensional virtual grid, and each switch is connected to its neighboring switch with the nearest corresponding coordinate in the virtual grid. The torus topology is similar to the mesh topology, but unlike the mesh topology, the torus topology also includes loop connections between switches at the edge of the virtual grid. The hypercube topology is a Cartesian mesh topology that traverses only one hop along each dimension. The generalized hypercube topology extends the torus topology by interconnecting all switches along a certain axis instead of only interconnecting neighboring switches with the nearest coordinates. In the Cartesian HyperX topology, each switch is connected to all other switches in each D dimension.

[0045] A route (or path) in a D-dimensional Cartesian topology traverses a sequence of one or more hops, where each hop traverses from one switch (or source node) to a successive switch (or destination node) along the route. Each hop traverses one of the topological dimensions, corresponding to a topological coordinate that changes between two switches.

[0046] exist Figure 1A In the example of FIG. 3 , the switches of network 30 are arranged in a Cartesian 3D HyperX topology. In a Cartesian HyperX topology, each switch is connected to all other switches along each dimension. Figure 1A , the switches are labeled with corresponding 3D coordinates along the X, Y, and Z dimensions. In this example, the Cartesian 3D HyperX topology includes four switches along the X and Y dimensions and three switches along the Z dimension.

[0047] Figure 1B is a block diagram schematically illustrating a computing system 90 according to another embodiment described herein, wherein network nodes communicate via a network 31 having a Cartesian three-dimensional (3D) mesh topology.

[0048] In this example, according to a mesh topology, the network 31 includes a plurality of switches 32 interconnected using physical links 34 in a 3D Cartesian grid. In this example, the mesh topology includes three switches along each of the X and Y dimensions, and two switches along the Z dimension. The network nodes 24 are coupled to the switches 32 and can communicate with each other through the network.

[0049] The above about Figure 1A The description of computing system 20 applies similarly to Figure 1B A computing system 90.

[0050] The embodiments disclosed herein are applicable to both the original network where all switches (32) and physical links (34) on the virtual grid are present and functional, as well as to a failed network where at least one switch or physical link is missing or has failed.

[0051] Networks 30 and 31 are managed using a central network management computer, referred to herein as a network manager 36. Figure 1A A block diagram depicting details of an example network manager 36 is depicted in FIG. Figure 1BThe network manager includes an interface 44 for connecting to the associated network 30 or 31, and a processor 46 coupled to the interface 44 and a memory 48 via a bus 49 (e.g., a peripheral component interconnect express (PCIe) bus or any other suitable bus). In some other embodiments, the network manager 36 can run on one of the network nodes 24. The network manager enables a network administrator to set up a network and provide routing information to network elements via the interface 44. Alternatively, at least a portion of the provision is performed automatically by the network administrator.

[0052] In some embodiments, the network manager 36 knows the underlying network topology, i.e., the identities of network elements and the manner in which they are interconnected, e.g., as manually configured by a network administrator via a suitable interface (not shown). Alternatively or additionally, the network manager automatically learns the network topology by querying the network elements 32.

[0053] Among other tasks, the network manager 36 specifies routes between pairs of network elements based on the network topology and configures relevant routing information to the network elements belonging to those routes.

[0054] In some embodiments, the network manager 36 represents the underlying topology in the memory 48 as a topology 50, including a set of switch coordinates and the connections between them. In some embodiments, in order to control packet forwarding and VL usage, the network manager defines multiple types of turns 52 in the topology 50, where each turn traverses two consecutive hops. Based on the defined turns, the network manager specifies rules 56 to be applied within each switch to prevent deadlock conditions. The network manager can also store one or more policies 54 that specify additional restrictions on allowed paths, such as a maximum number of hops per path. The network manager configures the switches in the network according to the rules 56, for example, via the interface 44.

[0055] Despite Figure 1A The computing system 20 and Figure 1B In the computing system 90 of FIG. 1 , the network manager 36 is implemented on a dedicated network node, but in an alternative embodiment, the network manager may be implemented within one of the network nodes 24 and its functions performed by a processor of the network node. Further alternatively, at least one of the switches in the network may include a "managed switch" attached to a processor, and the network manager (or a portion thereof) may reside on the managed switch.

[0056] Network Component Structure and Function

[0057] Figure 2is a block diagram schematically illustrating a network switch 32 according to embodiments described herein. Although the following description primarily relates to network switches, the disclosed techniques may be used with various other types of network elements, such as routers.

[0058] Figure 2 The description mainly involves Figure 1A The switch 32 in the network 30 of FIG. 1 is similarly applicable to Figure 1B A switch 32 in the network 31.

[0059] The switch 32 includes a plurality of ports 64 for exchanging data packets with the network 30 via the physical link 34. In some embodiments, a given port 64 may be used as an ingress interface for incoming packets or as an egress interface for outgoing packets. Alternatively, the port 64 may be used as both an ingress and an egress interface.

[0060] Packets traversing the network may belong to different flows. In some embodiments, each physical link 34 in the network 30 is shared by multiple logical communication links, also known as virtual channels. In InfiniBand, a virtual channel is also known as a virtual lane (VL).

[0061] In this example embodiment, the switch 32 includes a packet processing circuit 66 coupled to a data memory 68, which includes a VL0 buffer 72 and a VL1 buffer 74, which are allocated to store packets arriving from the network via the ingress interface of the port 64. The VL0 buffer and the VL1 buffer are also referred to as "virtual buffers". Received packets are allocated to VL0 or VL1 and stored in the virtual buffer according to the corresponding VL value carried in the packet header. In one embodiment, the switch stores packets received via a common port and belonging to one or more different flows in the same VL0 or VL1 buffer.

[0062] In one embodiment, the switch 32 receives a packet assigned to an input VL value denoted as "VL0" via port 64 and replaces the input VL value in the packet header with a different output VL value denoted as "VL1" before sending the packet back to the network via one of the egress interfaces of port 64. In some embodiments, the mapping from input VL values ​​to output VL values ​​is part of routing information (e.g., rule 56) provided to the switch by a network manager. In some embodiments disclosed below, changing the VL value assigned to a packet traversing a switch is used to prevent deadlock conditions in a network with a Cartesian topology. In some embodiments, based on a prescribed rule specifying VL modification information, the switch decides to modify the VL of a packet received from VL0 to VL1 for certain types of turns formed by two hops in between the switch.

[0063] In the example switch 32, the packet processing circuitry 66 is also coupled to a memory 76 that stores forwarding rules for forwarding packets between ports 64 and VL modification rules that specify VL modification conditions. In the present example, the forwarding rules are implemented in one or more forwarding tables 78, and the VL modification rules are implemented in a VL-to-VL table 80. In alternative implementations, the rules may be implemented in the switch using any other suitable method.

[0064] Typically, each packet belongs to a flow. By routing a flow to a port based on configured forwarding rules, the switch routes the packets of the flow through a routing path through the underlying network (eg, network 30).

[0065] In the context of this patent application and claims, the term "packet" is used to describe the basic unit of data routed through a network. Different network types and communication protocols use different terms for such data units, such as packet, frame, or cell. All of these data units are considered packets in this article.

[0066] Switch 32 includes switch controller 82, which configures the rules in rule memory 76 to apply a desired routing plan. By controlling the routing plan, switch 32 enables packets to traverse various routing paths through the underlying network (eg, network 30).

[0067] In some embodiments, the switch 32 supports adaptive routing by allowing packets received in a given ingress interface to be routed via one of a plurality of egress interfaces. A mapping from an ingress port to one or more egress ports is configured in the forwarding table 78. To reroute a given flow, the switch controller 82 selects a different egress interface for a flow packet received by the switch via an ingress interface. Alternatively, the switch 32 may maintain a set of routing rules, for example, per flow. In some cases, packets are routed without recording any information for future use.

[0068] In some embodiments, forwarding the packet is based on certain fields in the packet header. These fields may include, for example, at least one of the source and destination addresses, the underlying protocol, and the source and destination port numbers. In some embodiments, forwarding the packet includes computing a hash function on one or more fields in the packet header and using the resulting hash value to select a corresponding egress interface for the packet.

[0069] Figure 1A The computing system 20 and the network manager 36 in Figure 1B Computing systems in 90 and Figure 2The configurations of the switches 32 in are example configurations, which are selected purely for conceptual clarity. In alternative embodiments, any other suitable configuration may also be used. The various elements of the network manager 36 and the switches 32 may be implemented in hardware, for example using one or more application specific integrated circuits (ASICs) or field programmable gate arrays (FPGAs). In alternative embodiments, some elements of the network manager 36 and / or the switches 32 (e.g., the processor 46 and / or the switch controller 82) may be implemented in software executed on a suitable processor or using a combination of hardware and software elements.

[0070] Elements that are not necessary for understanding the principles of the present application, such as various interfaces, addressing circuits, timing and sequencing circuits, and debugging circuits, have been removed for clarity. Figure 1A , Figure 1B and Figure 2 Omitted in .

[0071] In some embodiments, the processor 46 and / or the switch controller 82 may include a general purpose processor that is programmed in software to perform the network manager and / or switch functions described herein. The software may be downloaded to the processor in electronic form, such as over a network, or it may alternatively or additionally be provided and / or stored on a non-transitory tangible medium, such as magnetic, optical or electronic memory.

[0072] Routing and deadlock prevention in Cartesian topologies

[0073] As mentioned above, in some topologies, such as in a Cartesian topology, the path between two adjacent switches that are directly connected to each other is also referred to herein as a "hop". A path from a given switch to any other switch in the underlying topology may include one or more hops. In a Cartesian topology, a packet can only traverse switches along the underlying Cartesian grid. This means that diagonal paths are not allowed.

[0074] In some embodiments, each jump in the Cartesian topology is associated with a dimension and a sign attribute. When the jump traverses to a higher coordinate value in the traversed dimension, the sign of the jump is called "positive". Similarly, when the jump traverses to a lower coordinate value in the traversed dimension, the sign of the jump is called "negative".

[0075] As described above, in some embodiments, the network manager 36 defines multiple types of turns (52) in a Cartesian topology. Each turn traverses a first hop along a first dimension in a first direction (positive or negative), followed by a second hop along a second dimension in a second direction. Each turn type can be defined by the identity of the first dimension and the second dimension and the direction (e.g., sign attribute) of the first hop and the second hop. The first hop and the second hop of the turn are also referred to as "input hop" and "output hop" respectively in this article. The sequence including the input hop and the subsequent output hop thus forms a corresponding turn in the Cartesian topology. For example, a turn that traverses the input hop in the positive direction of the Y dimension and then traverses the output hop in the negative direction of the X dimension is represented as "+Y to -X".

[0076] Now consider Figure 1A 3D HyperX topology in which three dimensions are specified in three corresponding dimensions denoted as X, Y, and Z. In this example, each of the X and Y dimensions includes four switches, and the Z dimension includes three switches. Each switch 32 in the Cartesian topology is identified by a Cartesian 3D coordinate that is included in a corresponding range (0 … 3)、(0 … 3) and (0 … 2) is represented by the coordinate value (Cx, Cy, Cz). For example, the switch located at the origin coordinate Cx=Cy=Cz=0 is represented as "S000", and the switch at the coordinate Cx=3, Cy=2, Cz=1 is represented as "S321".

[0077] exist Figure 1A In the 3D HyperX topology, each switch is directly connected to all other switches along each of the X, Y, and Z dimensions using corresponding physical links 34. For example, switch S110 is directly connected to switches S010, S210, and S310 along the X dimension, directly connected to switches S100, S120, and S130 along the Y dimension, and directly connected to S111 and S112 along the Z dimension.

[0078] In the Cartesian 3D HyperX topology, the shortest paths between switches with different 3D coordinates in one dimension, two dimensions, or three dimensions include a one-hop path, a two-hop path, and a three-hop path, respectively. Note that the shortest path is not necessarily unique. For example, a two-hop path from S110 to S230 may traverse the intermediate switch S210 or S130.

[0079] Next, we consider the conditions that produce deadlock in a Cartesian topology. Depending on the dimensions and sign properties of the two hops that comprise the turn, the conditions are specified using certain types of turns in the topology. For convenience, the conditions for deadlock are discussed for a Cartesian 3D topology.

[0080] When defining turn types, the following jump notation is used: Jumps with a positive sign in the X, Y, or Z dimension are represented as +X, +Y, or +Z, respectively. Jumps with a negative sign in the X, Y, or Z dimension are represented as -X, -Y, or -Z, respectively. Jumps with a positive or negative sign in the X, Y, or Z dimension are represented as ±X, ±Y, or ±Z, respectively.

[0081] A dependency cycle comprises a cyclic sequence of turns traversed by one or more pattern flows. A deadlock condition occurs when the buffers of switches involved in a dependency cycle become full, thereby preventing further transmission of packets from any of these switches. The deadlock condition can be prevented by (i) prohibiting any flow in the network from traversing at least one turn in a dependency cycle, or (ii) modifying the VL value assigned to a packet traversing a turn in a dependency cycle.

[0082] Without VL modification, a cyclic path can be formed by traversing two hops in opposite directions along each dimension. In a single dimension (e.g., the X dimension), a dependency cycle can occur in a path that includes at least one turn from -X to +X and another turn from +X to -X. Therefore, it is sufficient to prevent all -X to X, -Y to Y, and -Z to Z while the VL value remains unchanged.

[0083] In two dimensions, for example X and Y, dependency cycles may occur in paths that include (i) at least one turn from ±Y to +X and another turn from -X to ±Y, or (ii) at least one turn from ±X to +Y and another turn from -Y to ±X.

[0084] Preventing deadlock in two and three dimensions is described here.

[0085] Figure 3 is a diagram schematically illustrating dependency cycles across two dimensions according to embodiments described herein. Figure 3 Depicted is a network with Cartesian 3D topology represented as S1 … Each switch 84 includes a buffer 86 for storing packets received from the buffer of the previous switch and for sending packets to the buffer 86 of the subsequent switch. The loop path S1->S2->S3->S4->S1 forms a dependency loop in the dimensions X and Y.

[0086] exist Figure 3 In the example, switches S1, S2, S3 and S4 are assigned corresponding coordinates, as shown in Table 1.

[0087] SW / Coordinates Cx Cy C change S1 1 2 4 S2 1 0 4 -Y S3 3 0 4 +X S4 3 2 4 +Y S1 1 2 4 -X

[0088] Table 1: Assignment to Figure 3 The coordinates of the switches in

[0089] like Figure 3 As shown, the path from S1 to S3 via S2 creates a turn from -Y to +X, the path from S2 to S4 via S3 creates a turn from +X to +Y, the path from S3 to S1 via S4 creates a turn from +Y to -X, and the path from S4 to S2 via S1 creates a turn from -X to -Y.

[0090] When the buffers 86 of S1 ... S4 are all assigned to a common VL, the dependency cycle may be broken by the network manager 36 specifying a rule that prohibits one of the turns, e.g., -Y to +X as depicted in dashed lines. Alternatively, all turns are allowed, but the network manager specifies a rule that forces the VL to be modified from VL0 to VL1 within one turn, e.g., a turn from -Y to +X.

[0091] In some embodiments, the network manager 36 specifies a first set of turns to be applied to packets assigned to VL0 and a second set of turns to be applied to packets assigned to VL1. The first set specifies turns that packet VL must modify from VL0 to VL1. The second set specifies turns that are prohibited for packets assigned to VL1.

[0092] In some embodiments, the first and second sets of turns from which rules for preventing deadlock conditions across any pair of dimensions are derived are given by: first set of turns - (±Y to +X), (±Z to +X), and (±Z to +Y), second set of turns - (-Y to ±X), (-Z to ±X), and (-Z to ±Y).

[0093] Figure 4 is a diagram schematically illustrating dependency cycles across three dimensions according to embodiments described herein.

[0094] Figure 4 Depicted are switches 84 denoted S1 ... S6, belonging to a network having a Cartesian 3D topology. Each switch 84 comprises a buffer 86 for storing packets received from the buffer of the previous switch and for sending packets to the buffer of the subsequent switch. The cyclic path S1->S2->S3->S4->S5->S6->S1 forms a dependency cycle in dimensions X, Y and Z.

[0095] exist Figure 4 In the example, switches S1…S6 are assigned corresponding coordinates, as shown in Table 2.

[0096] SW / Coordinates Cx Cy C change S1 3 2 4 S2 3 2 1 -Z S3 3 0 1 -Y S4 1 0 1 -X S5 1 2 1 +Y S6 1 2 4 +Z S1 3 2 4 +X

[0097] Table 2: Assignment to Figure 4 The coordinates of the switches in

[0098] like Figure 4As shown, the path from S1 to S3 via S2 creates a turn from -Z to -Y, the path from S2 to S3 to S4 creates a turn from -Y to -X, the path from S3 to S5 via S4 creates a turn from -X to +Y, the path from S4 to S6 via S5 creates a turn from +Y to +Z, and the path from S5 to S1 via S6 creates a turn from +Z to +X.

[0099] When the buffers 86 of S1 ... S6 are all assigned to a common VL, the dependency cycle may be broken by the network manager specifying a rule that prohibits a turn in VL0, e.g., +Z to +X. However, if the VL is modified to VL1, the turn is allowed. As another example, to break the dependency cycle, a turn may be allowed in VL0 but prohibited in VL1, e.g., -Z to -Y.

[0100] In some embodiments, the first and second sets of turns from which rules for preventing deadlock conditions across three dimensions are derived are given by: first set - (±Y to +X), (±Z to +X), and (±Z to +Y) and second set - (-Y to ±X), (-Z to ±X), and (-Z to ±Y).

[0101] In an embodiment, Table 3 summarizes the types of turns specified for the 3D Cartesian topology in the first and second sets of turns.

[0102]

[0103] Table 3: Turns specified for 3D Cartesian topology in the first and second sets of turns.

[0104] How to specify forwarding rules and VL modification rules

[0105] In some embodiments, to prevent a deadlock condition in a network having a Cartesian topology, the processor specifies rules to be applied to the group assigned to VL0 based on turns belonging to the first group of turns, and specifies rules to be applied to the group assigned to VL1 based on turns specified in the second group of turns. In an example embodiment, the network includes a 3D Cartesian topology, in which case the number of turns in the first group and the second group can be specified as given in Table 3 above.

[0106] Figure 5 is a flow chart schematically illustrating a method for specifying forwarding rules for switches in a network having a Cartesian topology according to embodiments described herein.

[0107] The method will be described as being executed by the processor 46 of the network manager 36 to specify the Figure 1AThe method is similarly applicable to switches 32 in other suitable networks, such as Figure 1B 31 of the network. It is further assumed that each network node 24 has a unique address or local identifier (LID), for example, assigned by a network manager. Although the method will be described for a single switch 32, in practice the method is generally applied to multiple switches 32 in the network 30, for example, to all or part of the switches 32.

[0108] The method begins with the processor 46 selecting a first LID at a first LID selection step 100. The first LID may be any LID assigned to the network nodes 24, respectively.

[0109] At the port scanning step 104, the processor manages scanning all (input port, output port) pairs in the underlying switch that can forward packets received via the input port to the current LID via the output port. At the rule specifying step 108, the processor specifies a forwarding rule (or multiple forwarding rules) for the current input port and output port pair based on certain routing constraints. An example rule is given here for a given turn from an input hop to an output hop via the underlying switch:

[0110] For packets assigned to VL0, when it is detected that a given turn belongs to the first group, the processor specifies a forwarding rule that prohibits the given turn.

[0111] For packets assigned to VL0, when it is detected that a given turn belongs to the first group, the processor specifies a forwarding rule that allows the given turn, and specifies a VL modification rule corresponding to the given turn when allowed, for modifying the VL of the packet from VL0 to VL1.

[0112] For the groups assigned to VL1, when it is detected that a given turn belongs to the second group, the processor specifies a rule prohibiting the given turn.

[0113] For packets assigned to VL1, the processor specifies a rule to keep the VL1 value unchanged.

[0114] At the port loop termination step 112, the processor checks whether all input-output port pairs in the switch have been scanned, and if not, loops back to step 104 to select the subsequent input and output port pairs. Otherwise, the processor proceeds to the LID loop termination step 116 to check whether all LIDs in the network have been checked. When there are LIDs that have not been considered at step 116, the processor selects one of these LIDs at the next LID selection step 120 and loops back to step 104.

[0115] When all LIDs have been processed at step 116, the processor proceeds to providing step 124 to provide the specified rules to the switch. After step 124, the method terminates.

[0116] As described above, in some embodiments, the rules specified based on the turns in the first group that are allowed include VL modification rules. In this embodiment, for a packet traversing a turn belonging to the first group via a switch, the rules specify that VL is modified to VL1.

[0117] In some embodiments, forwarding rules are implemented within a switch using a forwarding table. In such embodiments, a forwarding rule specified to allow a turn from an input port to an output port associates an output port in the forwarding table with an input port. Typically, multiple output ports may be associated with a single input port, thereby supporting multiple possible paths to a destination. This may be useful for adaptive routing and / or bypassing failed links. In one embodiment, a forwarding rule that prohibits a given turn is implemented by omitting the association between the output port and the input port in question.

[0118] In some embodiments, the VL modification rules are implemented in the switch in the VL-to-VL table. In such embodiments, the VL-to-VL table contains a turn at which the switch modifies the packet's VL value to VL1 when the packet assigned to VL0 traverses.

[0119] Rules based on other policies

[0120] The above rules are mainly used to prevent deadlock situations in the network. In some embodiments, additional constraints can be imposed via the policy 54. For example, in various applications, the path length from the source node to the destination plays an important role and should therefore be limited.

[0121] In a Cartesian topology, the distance from a first switch to a second switch can be specified as the number of hops traversed from the first switch to the second switch. A turn made via a given switch may cause the remaining number of hops (eg, distance) to the destination to decrease, remain the same, or increase.

[0122] When a coordinate element that differs in one dimension becomes equal to the corresponding coordinate element of the target, the distance to the target decreases, for example, traversing from [1,1,1] to [1,1,3] when the destination coordinates are [1,2,3]. When the changed coordinate element is different from the coordinate element corresponding to the target and remains different, the distance to the target remains unchanged, for example, traversing from [1,1,1] to [1,4,1] when the target coordinates are [1,2,3]. The distance to the target increases when the changed coordinate element is already equal to the coordinate element corresponding to the target, for example, traversing from [1,1,1] to [3,1,1] when the target coordinates are [1,2,3].

[0123] In some embodiments, the processor specifies a forwarding rule based on a policy of only allowing turns that result in a reduced distance to a destination. In such embodiments, the processor specifies a forwarding rule that allows a turn from an input port to an output port only if traversing the turn reduces the number of remaining hops to a given destination.

[0124] In some embodiments, the processor specifies forwarding rules based on a policy that allows turns that increase the distance to a destination or keep the distance to a destination constant but limits the total path length to the destination. In such embodiments, the processor specifies forwarding rules that allow turns that do not reduce the number of remaining hops to a given destination but meet a specified total number of hops for each path.

[0125] In some embodiments, to limit the path length, the processor specifies a rule that allows only a predefined number of turns that do not result in a reduction in the length to the destination. For example, a rule that allows turns that do not reduce the number of hops to the destination only at selected switches (e.g., a switch directly coupled to the source node and / or a switch that is one hop away from a switch coupled to the destination node).

[0126] Constraining the total path length reduces path latency and forces path convergence. For example, consider a switch at coordinates (2,1,3) of a 3D HyperX topology, where the switch is located one hop away from a target switch at coordinates (2,5,3). When the path length is not constrained, the network manager may generate a non-convergent path that wraps around the target switch along the second dimension. Examples of such non-convergent paths are given below: (2,1,3)->(2,2,3)->(2,4,3)->(2,3,3) and so on.

[0127] In some embodiments, each available path from a given switch to a destination switch contains a turn belonging to the second set of turns, which is prohibited for packets assigned to VL1 (to prevent deadlock). In one embodiment, in response to detecting that a path from a given switch to a destination must contain a turn belonging to the second set, the processor specifies a forwarding rule that prohibits traversing the path.

[0128] In some embodiments, the network comprises a Cartesian HyperX topology having up to three dimensions, i.e., 1D, 2D, 3D topology. In such embodiments, the processor is configured to specify rules to ensure that there is at least one path of at most three hops between any pair of network elements in the network.

[0129] The embodiments described above are given as examples, and other suitable embodiments may also be used. For example, the turns given in Table 3 above are example turns, and other suitable turns may also be used. For example, in Table 3, turns are specified for the 3D topology in the first dimension, the second dimension, and the third dimension represented as X, Y, and Z. In alternative embodiments, other assignments of the three dimensions to the symbols X, Y, and Z may also be used.

[0130] In some embodiments, other turns that result in stricter rules may also be used. For example, a turn from -X to +X may be replaced with a turn from -X to ±X or a turn from ±X to ±X. (Similar replacement turns may be specified for the Y and Z dimensions.) In such embodiments, the processor of the network management specifies a rule that prohibits turns that include two consecutive hops along the same dimension. This may help avoid non-convergent paths. In switches with limited storage space for storing rules, such as in a forwarding table, it may be useful to use turns from which stricter rules are derived.

[0131] The turns in Table 3 are specified for 3D topologies, but other suitable turns for 2D and 1D Cartesian topologies may also be specified. In an example embodiment, in a Cartesian two-dimensional topology, the first group includes turns ±Y to +X, -X to +X, and -Y to +Y and the second group includes turns -Y to ±X, -X to +X, and -Y to +Y. For a one-dimensional topology, each of the first and second groups includes a single turn -X to +X.

[0132] Note that a 2D HyperX topology may be viewed as a 3D HyperX topology where one dimension (eg, Z) has a size of 1. Thus, in embodiments, turns specified in the 3D topology and involving a missing dimension may be ignored.

[0133] It should be understood that the embodiments described above are cited by way of example, and the following claims are not limited to what has been specifically shown and described above. On the contrary, the scope includes combinations and sub-combinations of the various features described above, as well as changes and modifications that would occur to a person skilled in the art after reading the above description and that are not disclosed in the prior art. Documents incorporated by reference into this patent application should be considered an integral part of this application, unless any term defined in these incorporated documents in a manner that conflicts with an explicit or implicit definition in this specification, only the definition in this specification should be considered.

Claims

1. A communication device, comprising: a network interface in communication with a network comprising a plurality of network elements interconnected in a Cartesian topology; and A processor, the processor: defining a first set of turns and a second set of turns in the topology, wherein each turn in the first set of turns and each turn in the second set of turns includes an input hop from a previous network element to a current network element and an output hop from the current network element to a next network element; wherein, for packets that are (i) initially assigned to a first virtual channel and (ii) reallocated to a different second virtual channel while traversing network elements in the network, turns belonging to the first group are allowed, and wherein for the group assigned to the second virtual channel, turns belonging to the second group are prohibited; specifying rules based on the turns in the first group and the second group that, when applied to packets traversing respective network elements, ensure that deadlock conditions do not occur in the network, wherein said rules specified for a given network element include (i) forwarding rules from one or more input ports to one or more output ports such that packets received in said given network element and assigned to a second virtual channel arrive at a given destination coupled to said network without traversing any of said turns of said second set, and (ii) virtual channel modification rules for reallocating packets traversing turns of said first set and assigned to a first virtual channel to a second virtual channel; and The given network element is configured with the rule.

2. The apparatus of claim 1 , wherein the processor is configured to identify a given turn occurring when traversing the given network element from an input port to an output port, and (i) specify at least one of the rules for packets assigned to the first virtual channel depending on whether the given turn belongs to the first group, and (ii) specify at least one of the rules for packets assigned to the second virtual channel depending on whether the given turn belongs to the second group. 3 . The apparatus of claim 2 , wherein the processor is configured to, in response to detecting that the given turn belongs to the second group, specify a forwarding rule that prohibits the given turn. 4 . The apparatus of claim 2 , wherein the processor is configured to, in response to detecting that the packet traversing the given turn belongs to the first group, specify a forwarding rule that prohibits the given turn.

5. The apparatus of claim 2, wherein the processor is configured to, in response to detecting that the given turn belongs to the first group, specify (i) a forwarding rule that allows the given turn and (ii) a corresponding virtual channel modification rule that modifies the virtual channel in the group to the second virtual channel.

6. The apparatus of claim 1, wherein the processor is configured to specify a forwarding rule that allows a turn from an input port to an output port only if traversing the turn reduces the number of remaining hops to the given destination.

7. The apparatus of claim 1, wherein the processor is configured to specify forwarding rules that allow turns that do not reduce the number of remaining hops to the given destination but satisfy a specified total number of hops per path.

8. The apparatus of claim 1, wherein the processor is configured to, in response to detecting that a path from the given network element to the destination must contain a turn belonging to the second group, specify a forwarding rule that prohibits traversing the path.

9. The apparatus of claim 1, wherein the Cartesian topology comprises a Cartesian topology selected from a list of Cartesian topologies, the Cartesian topology comprising a mesh topology, a torus topology, a hypercube topology, a generalized hypercube topology, and a HyperX topology.

10. The apparatus of claim 1, wherein the Cartesian topology comprises a Cartesian HyperX topology having up to three dimensions, and wherein the processor is configured to specify the rule so as to ensure that between any pair of network elements in the network, there is at least one path of at most three hops.

11. The apparatus of claim 1, wherein the processor is configured to specify a virtual channel modification rule that prohibits modification of the virtual channel of packets assigned to the second virtual channel.

12. A communication method, comprising: In a network manager in communication with a network, the network comprising a plurality of network elements interconnected in a Cartesian topology, defining a first set of turns and a second set of turns in the topology, wherein each turn in the first set and each turn in the second set includes an input hop from a previous network element to a current network element and an output hop from the current network element to a next network element; wherein, for packets that are (i) initially assigned to a first virtual channel and (ii) reallocated to a different second virtual channel while traversing network elements in the network, turns belonging to the first group are allowed, and wherein for the group assigned to the second virtual channel, turns belonging to the second group are prohibited; specifying rules based on the turns in the first group and the second group that, when applied to packets traversing the corresponding network element, ensure that no deadlock condition occurs in the network, wherein the rules specified for a given network element include (i) forwarding rules from one or more input ports to one or more output ports such that packets received in the given network element and assigned to a second virtual channel arrive at a given destination coupled to the network without traversing any of the turns in the second group, and (ii) virtual channel modification rules for reallocating packets that traverse a turn in the first group and are assigned to a first virtual channel to a different second virtual channel; The given network element is configured with the rule.

13. The method of claim 12, wherein specifying rules comprises identifying a given turn occurring when traversing the given network element from an input port to an output port, and (i) specifying at least one of the rules for packets assigned to the first virtual channel depending on whether the given turn belongs to the first group, and (ii) specifying at least one of the rules for packets assigned to the second virtual channel depending on whether the given turn belongs to the second group. 14 . The method of claim 13 , wherein specifying the rule comprises specifying a forwarding rule that prohibits the given turn in response to detecting that the given turn belongs to the second group.

15. The method of claim 13, wherein specifying the rule comprises specifying a forwarding rule that prohibits the given turn in response to detecting that a packet traversing the given turn belongs to the first group.

16. The method of claim 13, wherein specifying the rules comprises, in response to detecting that the given turn belongs to the first group, specifying (i) a forwarding rule that allows the given turn and (ii) a corresponding virtual channel modification rule that modifies the virtual channel in the group to the second virtual channel.

17. The method of claim 12, wherein specifying the rule comprises specifying a forwarding rule that allows a turn from an input port to an output port only if traversing the turn reduces the number of remaining hops to the given destination.

18. The method of claim 12, wherein specifying the rules comprises specifying forwarding rules that allow turns that do not reduce the number of remaining hops to the given destination but satisfy a specified total number of hops per path.

19. The method of claim 12, wherein specifying the rule comprises, in response to detecting that a path from the given network element to the destination must contain a turn belonging to the second group, specifying a forwarding rule that prohibits traversing the path.

20. The method of claim 12, wherein the Cartesian topology comprises a Cartesian topology selected from a list of Cartesian topologies, the Cartesian topology comprising a mesh topology, a torus topology, a hypercube topology, a generalized hypercube topology, and a HyperX topology.

21. The method of claim 20, wherein the HyperX topology comprises up to three dimensions, and wherein specifying the rule comprises specifying the rule to ensure that between any pair of network elements in the network, at least one path is at most three hops.

22. The method of claim 12, wherein specifying the rule comprises specifying a virtual channel modification rule that prohibits modifying the virtual channel of packets assigned to the second virtual channel.

23. A communication network system comprising a plurality of network elements interconnected in a Cartesian topology, wherein the network elements are configured to forward packets according to rules, and when the rules are applied to packets traversing the corresponding network elements, deadlock conditions are ensured not to occur in the communication network system, wherein the rule is specified based on a first set of turns and a second set of turns in the Cartesian topology, each turn in the first set and each turn in the second set comprising an input hop from a previous network element to a current network element and an output hop from the current network element to a next network element, in, For packets that are (i) initially assigned to a first virtual channel and (ii) are reassigned to a different second virtual channel while traversing network elements in the network, a turn belonging to the first group is allowed, and wherein for the group assigned to the second virtual channel, turns belonging to the second group are prohibited; and wherein said rules specified for a given network element include (i) forwarding rules from one or more input ports to one or more output ports such that packets received in said given network element and assigned to a second virtual channel reach a given destination coupled to said network without traversing any of said turns of said second set, and (ii) virtual channel modification rules for reallocating packets that traverse turns of said first set and are assigned to a first virtual channel to a different second virtual channel.

24. A communication method, comprising: In a communication network comprising a plurality of network elements interconnected in a Cartesian topology, packets are forwarded by said network elements according to rules, which ensure that no deadlock situation occurs in said network when said rules are applied to packets traversing the respective network elements, wherein the rule is specified based on a first set of turns and a second set of turns in the Cartesian topology, each turn in the first set and each turn in the second set comprising an input hop from a previous network element to a current network element and an output hop from the current network element to a next network element, wherein, for packets that are (i) initially assigned to a first virtual channel and (ii) reallocated to a different second virtual channel while traversing network elements in the network, turns belonging to the first group are allowed, and wherein for the group assigned to the second virtual channel, turns belonging to the second group are prohibited; and wherein said rules specified for a given network element include (i) forwarding rules from one or more input ports to one or more output ports such that packets received in said given network element and assigned to a second virtual channel reach a given destination coupled to said network without traversing any of said turns of said second set, and (ii) virtual channel modification rules for reallocating packets that traverse turns of said first set and are assigned to a first virtual channel to a different second virtual channel.

Images