Multi-path routing of switching fabric

Abstract

The invention relates to multi-path routing of a switch fabric. The switching fabric uses traffic congestion information to notify it of timely use of non-minimum routing. An ingress port of a network switch collects traffic congestion information from an egress port of the network switch. The traffic congestion information includes minimum and non-minimum routing congestion metrics for the egress port. A candidate egress port is identified for forwarding the data packet to the destination node. And selecting one of the candidate exit ports according to the traffic congestion information. The selection process is biased to select some candidate egress ports rather than other ports. In particular, a candidate egress port providing non-minimum routing to a destination node and having a high minimum routing congestion metric is not sponsored by a selection process.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to U.S. Provisional Application No. 63 / 589,116, filed on October 10, 2023, which is hereby incorporated by reference. Background Art

[0003] Network switching is a fundamental concept in computer networking that involves forwarding packets of data between nodes within a network. Network switches analyze the destination of incoming packets and use this information to make forwarding decisions, thereby performing data transfer within the network. Therefore, network switches can be used to facilitate connectivity and communication between multiple nodes within a network. Bandwidth is an important factor in network switching performance, as sustained high bandwidth results in faster communication between nodes. Latency is another important factor that affects network switching performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0004] Various aspects of the present disclosure are best understood from the following detailed description when read with the accompanying figures.

[0005] Figure 1 is a schematic diagram of a network system according to some embodiments.

[0006] Figure 2 is a schematic diagram of a switch fabric according to some embodiments.

[0007] Figure 3 is a block diagram of a network switch according to some embodiments.

[0008] Figure 4 is a block diagram of a network switch according to some embodiments.

[0009] Figure 5 is a schematic diagram of a packet forwarding method according to some implementations.

[0010] Figure 6 is a block diagram of a network switch according to some embodiments.

[0011] Corresponding numerals and symbols in the different figures generally refer to corresponding parts unless otherwise indicated.The figures are drawn to clearly illustrate the relevant aspects of the disclosure and are not necessarily drawn to scale. DETAILED DESCRIPTION

[0012] The following disclosure provides a number of different examples for implementing different features. Specific examples of components and arrangements are described below to simplify the disclosure. Of course, these are only examples and are not intended to be limiting.

[0013] A switching fabric is used for packet forwarding in a computing system. The switching fabric includes multiple network switches and links between the network switches. Nodes can use the switching fabric to communicate. For example, a first node can be connected to a first network switch of the switching fabric, a second node can be connected to a second network switch of the switching fabric, and the first node can send a data packet to the second node via the switching fabric. Specifically, the first node can send a data packet to an ingress port of a first network switch (to which the first node is connected), and the data packet can be forwarded to an egress port of a second network switch (to which the second node is connected) via the switching fabric.

[0014] The switching fabric has multiple routes that can be used to forward packets from a source node to a destination node. These routes can be minimum routes or non-minimum routes. Minimum routes have the fewest number of hops (or link crossings) from the source node to the destination node. Non-minimum routes have more hops than minimum routes. In general, minimum routes are the most efficient routes for forwarding packets from a source node to a destination node because minimum routes can have lower latency than non-minimum routes (because they inherently have fewer hops). However, in the case where there are not enough available minimum routes to carry the required bandwidth, opportunistic use of non-minimum routes can allow the performance of the switching fabric to be improved because higher bandwidth can be achieved by increasing the number of available routes. Minimum routes and non-minimum routes can be used simultaneously to forward packets from a source node to a destination node. This increased parallelism allows for increased bandwidth from a source node to a destination node by utilizing non-minimum routes on a link that would otherwise be idle or rarely used. However, overuse of non-minimum links between some nodes may increase congestion on links shared with minimum routes between other nodes, thereby reducing the overall performance of the network. Ideally, non-minimal routes should make use of link bandwidth that would otherwise go unused, and traffic should use minimal routes to avoid congestion on shared links.

[0015] The present disclosure describes a switching structure that uses traffic congestion information to inform it to use non-minimum routing when appropriate. The ingress port of a network switch collects traffic congestion information from the egress port of the network switch. The traffic congestion information includes minimum and non-minimum routing congestion metrics of the egress port. For example, the minimum routing congestion metric of the egress port can indicate the number of packets queued at the egress port for forwarding via the minimum routing using the egress port, and the non-minimum routing congestion metric of the egress port can indicate the number of packets queued at the egress port for forwarding via the non-minimum routing using the egress port. Other congestion metrics can also be used.

[0016] When forwarding a data packet received from a source node, the ingress port identifies a subset of egress ports that are candidates for forwarding the data packet to a destination node via a route of the network. The candidate egress ports may provide a minimal route from the source node to the destination node, or may provide a non-minimal route from the source node to the destination node. The network switch then selects a target egress port from the candidate egress ports based on traffic congestion information of the candidate egress ports.

[0017] The selection process is biased towards selecting some candidate egress ports over other ports. Specifically, if a candidate egress port provides a non-minimum route from a source node to a destination node, then when traffic congestion information indicates that the candidate egress port has excessive congestion of minimum route traffic, the candidate egress port will not be favored by the selection process. Therefore, excessive congestion of minimum route traffic at the egress port alleviates the risk of the egress port being selected by the ingress port for non-minimum route traffic. In this way, when it is necessary to increase the bandwidth between nodes when the non-minimum route link will be lightly loaded or idle, the non-minimum route can be used in a timely manner, but this timely use can avoid increasing congestion on the link that provides services for the minimum route between other nodes. Therefore, network bandwidth can be increased without excessively increasing network delay and / or power consumption.

[0018] Figure 1 is a schematic diagram of a network system 100 according to some embodiments. The network system 100 may be a high-performance network as part of a computing system, a high-performance computing (HPC) environment, etc. In the network system 100, processors 102 (e.g., processors 102A-102D) and devices 104 (e.g., devices 104A-104D) communicate with each other through a network switch 106 (e.g., network switches 106A-106D).

[0019] The processor 102 retrieves the executable code from the memory (not shown separately) and executes the executable code. When executed by the processor 102, the executable code may cause the processor 102 to perform any desired functions. The processor 102 may be a microprocessor, an application specific integrated circuit, a microcontroller, etc.

[0020] Device 104 includes various other hardware elements external and internal to network system 100. For example, device 104 may include an accelerator, a network interface device, a memory expansion device, etc. Device 104 may (or may not) have a local memory accessible to processor 102. In addition, device 104 may access system memory (not shown separately). Device 104 may communicate with processor 102 or communicate with each other through network switch 106.

[0021] The network switches 106 interconnect the processors 102 and the devices 104. The network switches 106 are connected to each other via network links 108 to form a switch fabric 110. The switch fabric 110 can have any suitable topology. In some implementations, the switch fabric 110 has a mesh topology.

[0022] The network switch 106 includes ports to which the processor 102, the device 104, and other devices of the network switch 106 are connected. The processor 102 and the device 104 communicate with each other via packets transmitted between ingress ports and egress ports of the network switch 106 in the switch fabric 110. In general, the switch fabric 110 can be used for communication between nodes (e.g., the processor 102 and the device 104). The processor 102 and the device 104 are merely examples of components that can be interconnected through the switch fabric 110. Other components can be connected to the switch fabric 110. Packets can be routed through the switch fabric 110. In some embodiments, the switch fabric 110 is a compute express link (CXL) fabric; the device 104 is a type 1, type 2, or type 3 CXL device; and the network link 108 is a PCI Express interface.

[0023] Figure 2 is a schematic diagram of a switch fabric 200 according to some embodiments. The switch fabric 200 is a schematic diagram of a switch fabric 200 according to some embodiments. Figure 1 An example of a switch fabric 110 is described. The switch fabric 200 includes network switches 202 (e.g., network switches 202A-202D) and network links 204 (e.g., network links 202AB-202CD). The network links 204 interconnect the network switches 202 to form a network having a desired topology. The switch fabric 200 may be used by nodes 206 (e.g., nodes 206A-206D) to communicate with each other. The nodes 206 may correspond to Figure 1 The processor 102 or device 104, or a combination of a processor and a device.

[0024] The switch fabric 200 includes a plurality of routes 208 that can be used to forward packets from a source node 206 to a destination node 206. Three example routes are shown from node 206A to node 206B: route 208A across link 204AB, route 208B across link 204AC and link 204BC, and route 208C across link 204AD and link 204BD. Further shown is an example route 208D from node 206A to node 206C (a route across link 204AC). Finally, shown is an example route 208E from node 206A to node 206D (a route across link 204AC and link 204CD).

[0025] Route 208 may be a minimal route, or may be a non-minimal route. Continuing with the previous example, route 208A is a minimal route from node 206A to node 206B because it has the lowest number of hops (or link crossings) from node 206A to node 206B. Route 208B and route 208C are non-minimal routes from node 206A to node 206B because they have more hops than route 208A. Specifically, route 208B has an additional hop (through switch 202C) compared to route 208A, and route 208C also has an additional hop (through switch 202D) compared to route 208A. In addition, route 208D is a minimal route from node 206A to node 206C. Finally, route 208E is a non-minimal route from node 206A to node 206D.

[0026] Generally speaking, a minimal route is the most efficient route for forwarding a data packet from a source node 206 to a destination node 206, however, the appropriate use of non-minimal routes may allow for improved performance of the switch fabric. For example, although route 208A (a minimal route from node 206A to node 206B) may be more efficient than routes 208B, 208C (a non-minimal route from node 206A to node 206B), the appropriate use of all routes 208A, 208B, 208C may allow for increased bandwidth from node 206A to node 206B.

[0027] Overuse of non-minimum routes between some nodes can lead to congestion of minimum routes between other nodes, thereby increasing network delay and / or power consumption. For example, overuse of route 208B (from node 206A to node 206B) can lead to blocking of route 208D (from node 206A to node 206C). As described in more detail later, network switch 202 will use traffic congestion information to notify the timely use of non-minimum routes, so that the forwarding of traffic on non-minimum routes in switching structure 200 will not reduce the forwarding of traffic on minimum routes in switching structure 200. In this example, routes 208B, 208D, 208E each pass through an identical network link, and timely use of route 208B (non-minimum route) will be avoided to facilitate the use of other traffic of route 208D (minimum route). However, timely use of 208B route will not be avoided in order to facilitate the use of other traffic of 208E route (another non-minimum route).

[0028] Figure 3 is a block diagram of a network switch 300 according to some embodiments. The network switch 300 is Figure 1 to Figure 2An example of a network switch described previously. Network switch 300 includes ports 302 (e.g., ports 302A, 302B, and 302N) and a switch core 304. Ports 302 serve as connection points for nodes (e.g., processors, devices, etc.). Switch core 304 manages and forwards data packets between ports 302.

[0029] Each port 302 includes an ingress port 306 and an egress port 308. The ingress port 306 is the entry point through which packets enter the network switch 300. The egress port 308 is the output point responsible for sending the packet to its designated destination. When a packet arrives at the ingress port 306, the network switch 300 examines the destination address of the packet to determine the appropriate egress port 308 for transmission. This process, called switching or forwarding, includes performing a lookup in the routing table of the network switch 300 to find (multiple) candidate forwarding paths for the packet. The candidate forwarding paths may include a list of egress ports for minimal routes, and a list of egress ports for non-minimal routes. The ingress port 306 controls how the packet is sent to the egress port 308.

[0030] The ports 302 (including the ingress port 306 and the egress port 308) are interconnected through the switch core 304, which provides the necessary path for the data packet to move from the ingress port 306 to the egress port 308. The switch core 304 connects the ingress port 306 and the egress port 308. Depending on the architecture of the network switch 300, the switch core 304 can be implemented using, for example, a single crossbar switch, a crossbar switch matrix, a shared bus, a shared memory, a chip-wide ring, etc. In an embodiment, the switch core 304 includes a plurality of crossbar switches, which are used for control and data transmission between the ingress port 306 and the egress port 308.

[0031] The components of the network switch 300 may be implemented as integrated circuits, such as in one or more integrated circuit chips and / or one or more integrated circuit packages. For example, the network switch 300 may include a processor, an application specific integrated circuit, a field programmable gate array, a memory, a combination thereof, etc. One or more modules in the network switch 300 may be partially or fully embodied as software and / or hardware for performing any of the functions described herein. For example, each of the buffers, crossbars, transmitters, receivers, structures, etc. described herein may be embodied as a logic block of an integrated circuit.

[0032] Figure 4 is a block diagram of a network switch 400 according to some embodiments. The network switch 400 is a block diagram of a network switch 400 according to some embodiments. Figures 1 to 3An example of a network switch is described. Additional components of a network switch 400 are shown, including components of a switch core 404, a plurality of ingress ports 406 (including ingress ports 406A and 406B), and a plurality of egress ports 408 (including egress ports 408A and 408B). A logical flow diagram during forwarding of a packet from an ingress port 406 to an egress port 408 is shown.

[0033] First, the components of the network switch 400 will be described. Each inlet port 406 includes a receiver 412, an input buffer 414, and an input queue 416. The receiver 412 receives a data packet on a physical link from a source node connected via the inlet port 406. The data packet goes to a destination node connected via one or more egress ports 408. The source node and / or the destination node may be directly connected to the network switch 400, or one or more network components (e.g., additional switches) may exist between the network switch 400 and (multiple) source / destination nodes. The input buffer 414 is connected to the receiver 412. The received data packet is stored in the input buffer 414. The input queue 416 is connected to the input buffer 414 and the receiver 412. The input queue 416 is an input controller that controls the sending of data packets from the input buffer 414 to the output buffer of the egress port 408. The request to send a data packet to the egress port 408 is queued by the receiver 412 at the input queue 416. The input queue 416 arbitrates among the requests queued therein and selects the request to be processed. The input queue 416 determines to which egress port 408 the data packet for the selected request should be forwarded. For example, a lookup unit (not separately shown) may extract the appropriate (multiple) headers from the data packet and use them to determine the destination node of the data packet. The input queue 416 may receive the lookup results from the lookup unit.

[0034] Each egress port 408 includes an output queue 422, an output buffer 424, a transmitter 426, and a load monitor 428. The output queue 422 is an output controller that controls the data packets received from the input buffer of the ingress port 406 to the output buffer 424. Requests to receive data packets from the ingress port 406 are queued at the output queue 422. The output queue 422 arbitrates among the requests in its queue and selects the request to be processed. The output buffer 424 is connected to the output queue 422. The received data packets are stored in the output buffer 424. The transmitter 426 is connected to the output buffer 424. The transmitter 426 reads the data packets from the output buffer 424 and sends the data packets to the destination node by sending a signal downward to the physical link. Therefore, by controlling the reception of the data packets in the output buffer 424, the output queue 422 effectively controls the reading of the data packets in the output buffer 424 by the transmitter 426.

[0035] The switch core 404 is described as having an example implementation based on a crossbar switch, including multiple crossbar switches that are different from each other. In this example, the switch core 404 includes a packet crossbar switch 432 and a load crossbar switch 434. As previously mentioned, other types of switch cores can also be utilized.

[0036] The packet crossbar 432 is connected to the input buffer 414 and input queue 416 of each ingress port 406, and to the output queue 422 and output buffer 424 of each egress port 408. Transmit requests, transmit grants, and data packets are transmitted through the packet crossbar 432. In some embodiments, multiple packet crossbars or other switch core architectures may be utilized. For example, transmit requests may be sent through a request crossbar, transmit grants may be sent through a grant crossbar, and data packets may be transmitted through a data crossbar.

[0037] The load crossbar 434 is connected to the input queue 416 of each ingress port 406 and to the load monitor 428 of each egress port 408. As described in more detail later, traffic congestion information will be transmitted through the load crossbar 434. The load crossbar 434 is a dedicated feedback structure separate from the packet crossbar 432.

[0038] A logic flow diagram for forwarding a data packet from an ingress port 406 to an egress port 408 will now be described. This logic flow diagram is an example, and other methods of packet forwarding may be utilized. A receiver 412 of an ingress port 406 receives a data packet and stores the data packet in an input buffer 414 of the ingress port 406. Requests to transmit data packets are queued at an input queue 416 of the ingress port 406, which then selects the request to be processed. The input queue 416 sends a transmit request for a data packet to an output queue 422 of an output port 408 via the switch core 404 (e.g., a packet crossbar 432). The transmit request includes a description of the data packet; for example, the request may include information from a header of the data packet.

[0039] The transmit request is queued at the output queue 422 of the egress port 408, and the egress port then selects the transmit request to be processed. The output queue 422 decides whether and when to grant the transmit request. For example, the output queue 422 can decide which transmit request to grant next based on the packet description of the transmit request and based on the current state of the output buffer 424 of the egress port 408. In response to the transmit request being granted, the output queue 422 sends the transmit grant to the input queue 416 of the ingress port 406 through the switch core 404 (e.g., the packet crossbar 432).

[0040] The transmit grant is a notification instructing the input queue 416 to move a data packet from the input buffer 414 of the ingress port 406 to the output buffer 424 of the egress port 408. In response to receiving the transmit grant, the input queue 416 transmits the data packet from the input buffer 414 to the output buffer 424 through the switch core 404 (e.g., the packet crossbar 432). Then, the transmitter 426 of the egress port 408 reads the data packet from the output buffer 424. Therefore, the output queue 422 controls the reading of the data packet by the transmitter 426 (thereby determining which packets are sent through the transmitter 426) through the grant of the transmit request.

[0041] When receiving a data packet from a source node or from another switch at an inlet port 406, there may be a plurality of candidate egress ports 408 for forwarding the data packet to a destination node via a network route. Each candidate egress port 408 may provide a minimum route or a non-minimum route to a destination node. That is, a first subset of egress ports 408 may be a candidate for forwarding a data packet to a destination node via a minimum route of a network, and a second subset of egress ports 408 may be a candidate for forwarding a data packet to a destination node via a non-minimum route of a network. Both minimum and non-minimum routes may be used in due course to increase the bandwidth from a source node to a destination node. For example, when forwarding a plurality of data packets, the plurality of data packets may be multiplexed across minimum and non-minimum routes. When forwarding a data packet to a destination node, an input port 406 may identify a plurality of candidate egress ports 408, and then select an egress port from the candidate egress ports 408 for forwarding.

[0042] The ingress port 406 may use the routing table 436 to identify the candidate egress ports 408. The routing table 436 may be stored at the input queue 416 of the ingress port 406. The routing table 436 includes a mapping from destination nodes to the egress ports 408 that provide routes to the destination nodes. When the ingress port 406 receives a packet, the input queue 416 may identify the destination node of the packet (e.g., using a destination node identifier in a packet header), and then look up the destination node in the routing table 436 to identify the candidate egress ports 408 for reaching the destination node. The routing table 436 may indicate whether each candidate egress port 408 provides a minimal route or a non-minimal route to the destination node.

[0043] Once the candidate egress ports 408 are identified by the ingress port 406, the ingress port 406 selects one of the candidate egress ports 408 based on the traffic congestion information of the candidate egress ports 408. For example, the traffic congestion information may be stored in the input queue 416. The input queue 416 may arbitrate between the candidate egress ports 408 based on the traffic congestion information to determine the selected egress port 408.

[0044] The traffic congestion information includes a metric for each egress port 408. Transmission requests for packets to different destination nodes may be queued at the output queue 422 of the egress port 408. The packets queued for transmission at the egress port 408 may be on a minimal route to its destination node, or may be on a non-minimal route to its destination node. The traffic congestion information includes the congestion metric for the minimal route of the egress port 408 and the congestion metric for the non-minimal route of the egress port 408.

[0045] For each egress port 408, the minimum route congestion metric indicates the degree of congestion of its minimum route traffic. As used herein, the minimum route traffic is the traffic queued at the egress port 408 (including traffic from the ingress port 406 and for all destinations) for which the egress port 408 provides the minimum route selection. In some embodiments, the minimum route congestion metric indicates the traffic backlog at the egress port 408 for the minimum route of the network. For example, the minimum route congestion metric indication may indicate the number of packets queued at the output queue 422 of the egress port 408 to be forwarded through the minimum route. In some embodiments, the minimum route congestion metric indicates the traffic delay at the egress port 408 for the minimum route of the network. Other suitable minimum route congestion metrics may also be utilized.

[0046] For each egress port 408, the non-minimum route congestion metric indicates the degree of congestion of its non-minimum route traffic. As used herein, non-minimum route traffic is traffic queued at the egress port 408 (including traffic from the ingress port 406 and for all destinations) and provided with non-minimum route selection by the egress port 408. In some embodiments, the non-minimum route congestion metric indicates the backlog of traffic at the egress port 408 for the non-minimum route of the network. For example, the non-minimum route congestion metric indication may indicate the number of packets queued at the output queue 422 of the egress port 408 to be forwarded through the non-minimum route. In some embodiments, the non-minimum route congestion metric indicates the delay of traffic at the egress port 408 for the non-minimum route of the network. Other suitable non-minimum route congestion metrics may also be utilized.

[0047] As is apparent from the foregoing description, the ingress port 406 tracks two separate metrics for each egress port 408: a minimum routing congestion metric and a non-minimum routing congestion metric. Therefore, each ingress port 406 knows the congestion level of the minimum routing traffic at each egress port 408. In addition, each ingress port 406 knows the congestion level of the non-minimum routing traffic at each egress port 408.

[0048] As previously described, once the ingress port 406 identifies a candidate egress port 408 for forwarding a data packet to a destination node, it can select one of the egress ports 408 by arbitrating between the candidate egress ports 408 based on the traffic congestion information. The arbitration process is weighted to give preference to some candidate egress ports 408 over other ports. Specifically, if the candidate egress port 408 provides a non-minimum route from the source node to the destination node, the candidate egress port 408 is not favored by the arbitration process when the traffic congestion information indicates that the candidate egress port 408 has too high a minimum route traffic congestion level. In this way, an egress port 408 that is a good candidate for timely non-minimum routing can be preferred over other egress ports 408 that are not good candidate ports because their use will have a negative impact on the minimum traffic sharing the same egress port 408.

[0049] The arbitration process may include, for each corresponding candidate egress port 408, calculating a weighted sum of the traffic congestion information (e.g., minimum and non-minimum route congestion metrics) of the corresponding candidate egress port 408. For the non-minimum candidate egress port 408, the minimum route congestion metric may be weighted more heavily than the non-minimum route congestion metric. Therefore, a non-minimum candidate egress port 408 that is congested due to the minimum traffic representing other routes is less likely to be selected than a non-minimum candidate egress port 408 that is also congested due to the non-minimum traffic representing other routes. Similarly, when calculating the weighted sum, the minimum route congestion metric of the non-minimum candidate egress port 408 may be weighted more heavily than the minimum route congestion metric of the minimum candidate egress port 408. Therefore, if the non-minimum candidate port 408 is also congested due to the minimum traffic, the possibility of selecting the non-minimum candidate egress port 408 is less than the possibility of selecting the minimum candidate egress port 408.

[0050] Traffic congestion information is collected at each egress port 408 and sent to each ingress port 406. The load monitor 428 at the egress port 408 can collect the traffic congestion information of the egress port 408 and send the traffic congestion information to the input queue 416 of the ingress port 406. The traffic congestion information can be sent through the switch core 404 (e.g., the load crossbar 434). The input queue 416 tracks the traffic congestion information of the egress port 408. The traffic congestion information can be used by the ingress port 406 to inform its timely use of non-minimal routing (as described above).

[0051] Figure 5 is a schematic diagram of a packet forwarding method 500 according to some embodiments. Figure 4A packet forwarding method 500 is described. The packet forwarding method 500 may be performed by the network switch 400 during forwarding of a packet from the ingress port 406 to the egress port 408.

[0052] The network switch 400 performs step 502 to receive a data packet from a source node at an ingress port of a first network switch, the data packet being destined for a destination node connected to a second network switch, the first network switch and the second network switch being located within a network. For example, the data packet may be received at an ingress port 406 of the network switch 400. The source node and the destination node are also located within the network. The network may have any desired topology, such as a mesh topology, a tree topology, etc.

[0053] The network switch 400 performs step 504 to identify an egress port of the first network switch that is a candidate for forwarding the data packet to the destination node via the network. For example, the ingress port 406 may look up an egress port 408 in the routing table 436 to which the data packet may be forwarded for routing to the destination node. The candidate egress port 408 may provide a minimal route or a non-minimal route to the destination node. A first subset of the egress ports 408 may be a candidate for forwarding the data packet to the destination node via the minimal route of the network, and a second subset of the egress ports 408 may be a candidate for forwarding the data packet to the destination node via the non-minimal route of the network.

[0054] The network switch 400 performs step 506 to select a target egress port by arbitrating between the egress ports based on the traffic congestion information of the egress ports, the traffic congestion information including a minimum routing congestion metric for the egress ports, and the traffic congestion information also including a non-minimum routing congestion metric for the egress ports. For example, the target egress port 408 may be selected by the ingress port 406 based on the minimum routing congestion metric and the non-minimum routing congestion metric.

[0055] In some embodiments, the minimum route congestion metric of the traffic congestion information indicates a traffic backlog at the egress port 408 for the minimum route of the network, and the non-minimum route congestion metric indicates a traffic backlog at the egress port 408 for the non-minimum route of the network. In some embodiments, the minimum route congestion metric of the traffic congestion information indicates a traffic delay at the egress port 408 for the minimum route of the network, and the non-minimum route congestion metric indicates a traffic delay at the egress port 408 for the non-minimum route of the network.

[0056] Arbitrating between egress ports 408 may include disfavoring egress ports with high minimum route congestion metrics in the second subset of egress ports 408 (for providing non-minimum routes to destination nodes). For example, egress ports in the second subset of egress ports 408 that have large traffic backlogs or large traffic delays on the minimum route may be disfavored.

[0057] In an embodiment, the combined metric of the candidate egress ports 408 may be calculated by calculating the sum of products: the minimum weight multiplied by the minimum routing congestion metric of the candidate egress port 408 may be added to the non-minimum weight multiplied by the non-minimum routing congestion metric of the candidate egress port 408. The egress ports 408 may then be statistically or deterministically selected such that egress ports 408 with lower combined metric values ​​are preferred over egress ports 408 with higher combined metric values.

[0058] Candidate exit ports 408 may be assigned weights for the arbitration process based on whether they provide a minimum or non-minimum route to a destination node, and disapproving of an exit port 408 may include increasing the weight of the exit port. When a candidate exit port 408 provides a minimum route to a destination node, a first minimum weight and a first non-minimum weight may be used to calculate a combined metric for the candidate exit port 408. When a candidate exit port 408 provides a non-minimum route to a destination node, a second minimum weight and a second non-minimum weight may be used to calculate a combined metric for the candidate exit port 408. The second minimum weight is greater than the first minimum weight. In other words, when calculating the combined metric of the exit port 408, the weight of the minimum route congestion metric for the second subset of the exit ports 408 is greater than the weight of the minimum route congestion metric for the first subset of the exit ports 408. As a result, the minimum route congestion metric for the second subset of the exit ports 408 is more heavily weighted than the minimum route congestion metric for the first subset of the exit ports 408. Similarly, the minimum route congestion metric for the second subset of the exit ports 408 may be more heavily weighted than the non-minimum route congestion metric for the second subset of the exit ports 408. Furthermore, the second non-minimum weight may (or may not) be different from the first non-minimum weight.

[0059] The traffic congestion information may be stored in the input queue 416 of the ingress port 406. The traffic congestion information may have been previously received from the load monitor 428 of the egress port 408. For example, the traffic congestion information may have been collected at the egress port 408 and then sent to the ingress port 406. The traffic congestion information may be sent from the egress port 408 to the ingress port 406 via a feedback structure, such as a load crossbar 434.

[0060] In some implementations, candidate egress ports 408 may also be identified based at least in part on traffic congestion information (in step 504). Specifically, if the minimum routing congestion metric of an egress port 408 is too large, then the egress port 408 may not be considered as a candidate egress port 408. In some implementations, candidate egress ports 408 are those ports whose minimum routing congestion metric is less than a predetermined threshold.

[0061] The network switch 400 performs a step 508 of forwarding the data packet to the target egress port. For example, the input queue 416 of the ingress port 406 may send a data packet transmission request to the output queue 422 of the target egress port 408. After receiving a corresponding transmission grant from the target egress port 408, the ingress port 406 transmits the data packet from its input buffer 414 to the output buffer 424 of the target egress port 408.

[0062] Figure 6 is a block diagram of a network switch 600 according to some embodiments. The network switch 600 is a block diagram of a network switch 600 according to some embodiments. Figure 4 An example of a network switch 400 is provided. The network switch 600 may include a processor 602 and a memory 604. The memory 604 may be a non-transitory computer-readable medium storing a program for execution by the processor. In this embodiment, one or more modules in the network switch 600 may be partially or entirely embodied as software for performing any of the functions described herein. For example, the memory 604 may include: instructions 606 for receiving a data packet from a source node at an ingress port of a first network switch, the data packet being destined for a destination node connected to a second network switch, the first network switch and the second network switch being located within a network; instructions 608 for identifying an egress port of the first network switch as a candidate for forwarding the data packet to the destination node via the network; instructions 610 for selecting a target egress port by arbitrating between the egress ports based on traffic congestion information of the egress ports, the traffic congestion information including a minimum routing congestion metric of the egress port, the traffic congestion information also including a non-minimum routing congestion metric of the egress port; and / or instructions 612 for forwarding the data packet to the target egress port.

[0063] Several variations are contemplated. For example, the switching techniques described herein may be applicable to other types of switching fabrics, such as Ethernet fabrics.

[0064] In an exemplary embodiment, a method includes: receiving a data packet from a source node at an ingress port of a first network switch, the data packet being destined for a destination node connected to a second network switch, the first network switch and the second network switch being located within a network; identifying an egress port of the first network switch, the egress port being a candidate for forwarding the data packet to the destination node through the network; selecting a target egress port by arbitrating between the egress ports based on traffic congestion information of the egress ports, the traffic congestion information including a minimum routing congestion metric of the egress ports, the traffic congestion information also including a non-minimum routing congestion metric of the egress ports; and forwarding the data packet to the target egress port. In some embodiments of the method, a first subset of the egress ports is a candidate for forwarding the data packet to the destination node through a minimum routing of the network, and a second subset of the egress ports is a candidate for forwarding the data packet to the destination node through a non-minimum routing of the network. In some embodiments of the method, arbitrating between the egress ports includes: calculating a weighted sum of the minimum routing congestion metric and the non-minimum routing congestion metric of the egress ports, the minimum routing congestion metric of the second subset of the egress ports having a greater weight than the non-minimum routing congestion metric of the second subset of the egress ports. In some embodiments of the method, the minimum route congestion metric indicates a traffic backlog at an egress port for a minimum route of the network, and the non-minimum route congestion metric indicates a traffic backlog at an egress port for a non-minimum route of the network. In some embodiments of the method, the minimum route congestion metric indicates a traffic delay at an egress port for a minimum route of the network, and the non-minimum route congestion metric indicates a traffic delay at an egress port for a non-minimum route of the network. In some embodiments of the method, the network has a mesh topology. In some embodiments, the method further includes: collecting traffic congestion information at the egress port; and sending the traffic congestion information to the ingress port. In some embodiments of the method, the traffic congestion information is sent from the egress port to the ingress port via a feedback structure of the first network switch.

[0065] In an exemplary embodiment, a network switch includes: a plurality of egress ports; and an ingress port, the ingress port being configured to: receive a data packet from a source node, the data packet being destined for a destination node, the source node and the destination node being located within a network; identify a candidate egress port of the plurality of egress ports, the candidate egress port being a candidate for forwarding the data packet to the destination node via the network; select a target egress port by arbitrating between the candidate egress ports based on traffic congestion information of the plurality of egress ports, the traffic congestion information including a minimum route congestion metric of the plurality of egress ports, the traffic congestion information also including a non-minimum route congestion metric of the plurality of egress ports; and forward the data packet to the target egress port. In some embodiments of the network switch, a first subset of the candidate egress ports are candidates for forwarding the data packet to the destination node via a minimum route of the network, and a second subset of the candidate egress ports are candidates for forwarding the data packet to the destination node via a non-minimum route of the network. In some embodiments of the network switch, the non-minimum route of the network has more hops than the minimum route of the network. In some embodiments of the network switch, arbitrating between candidate egress ports includes: calculating a weighted sum of the minimum route congestion metric and the non-minimum route congestion metric of the egress port, the weight of the minimum route congestion metric of the second subset of candidate egress ports being greater than the weight of the minimum route congestion metric of the first subset of candidate egress ports. In some embodiments of the network switch, the minimum route congestion metric indicates a traffic backlog at the candidate egress port for a minimum route of the network, while the non-minimum route congestion metric indicates a traffic backlog at the candidate egress port for a non-minimum route of the network. In some embodiments of the network switch, the minimum route congestion metric indicates a traffic delay at the candidate egress port for a minimum route of the network, while the non-minimum route congestion metric indicates a traffic delay at the candidate egress port for a non-minimum route of the network. In some embodiments, the network switch further includes a feedback structure, wherein each egress port is configured to: collect traffic congestion information; and send the traffic congestion information to the ingress port via the feedback structure.

[0066] In an exemplary embodiment, a system includes: a first node; a first network switch connected to the first node; a second node; and a second network switch connected to the second node, the first network switch and the second network switch are located within a network, and the second network switch is configured to: receive a data packet from the second node at an ingress port of the second network switch; identify an egress port of the second network switch, the egress port being a candidate for forwarding a data packet from the second node to the first node via the network; receive traffic congestion information from the egress port, the traffic congestion information including a minimum routing congestion metric of the egress port, the traffic congestion information also including a non-minimum routing congestion metric of the egress port; arbitrate between the egress ports based on the traffic congestion information of the egress port to select a target egress port; and forward the data packet to the target egress port. In some embodiments of the system, the network has a mesh topology. In some embodiments of the system, the second network switch includes a feedback structure, and receives traffic congestion information from the egress port at the ingress port via the feedback structure. In some embodiments of the system, arbitrating between the egress ports includes: assigning weights to the arbitration process of the egress port based on whether the egress port provides a minimum route or a non-minimum route to the first node. In some embodiments of the system, a first weight of the minimum route congestion indicator is assigned to an egress port that provides a minimum route to the first node, a second weight of the minimum route congestion indicator is assigned to an egress port that provides a non-minimum route to the first node, and the second weight is greater than the first weight.

[0067] The features of multiple examples are summarized above so that those skilled in the art can better understand the various aspects of the present disclosure. After referring to this specification, various modifications and combinations of the illustrative examples and other examples will be apparent to those skilled in the art. Therefore, the attached claims are intended to cover any such modifications.

Claims

1. A method comprising: receiving, at an ingress port of a first network switch, a data packet from a source node, the data packet being destined for a destination node connected to a second network switch, the first network switch and the second network switch being located within a network; identifying an egress port of the first network switch, the egress port being a candidate for forwarding the data packet to the destination node via the network; selecting a target egress port by arbitrating between the egress ports based on traffic congestion information of the egress ports, the traffic congestion information comprising a minimum routing congestion metric of the egress ports, the traffic congestion information further comprising a non-minimum routing congestion metric of the egress ports; as well as The data packet is forwarded to the target egress port.

2. The method of claim 1 , wherein a first subset of the egress ports are candidates for forwarding the data packet to the destination node via a minimal route of the network, and a second subset of the egress ports are candidates for forwarding the data packet to the destination node via a non-minimal route of the network.

3. The method of claim 2, wherein arbitrating between the egress ports comprises: A weighted sum of the minimum routing congestion metric and the non-minimum routing congestion metric of the egress port is calculated, the minimum routing congestion metric of the second subset of the egress ports being weighted more heavily than the non-minimum routing congestion metric of the second subset of the egress ports.

4. The method of claim 1 , wherein the minimum route congestion metric indicates a traffic backlog at the egress port for a minimum route of the network, and the non-minimum route congestion metric indicates a traffic backlog at the egress port for a non-minimum route of the network.

5. The method of claim 1 , wherein the minimum route congestion metric indicates a traffic delay at the egress port for a minimum route of the network, and the non-minimum route congestion metric indicates a traffic delay at the egress port for a non-minimum route of the network. The method of claim 1 , wherein the network has a mesh topology.

7. The method according to claim 1, further comprising: collecting the traffic congestion information at the egress port; as well as The traffic congestion information is sent to the ingress port.

8. The method of claim 7, wherein the traffic congestion information is sent from the egress port to the ingress port via a feedback structure of the first network switch.

9. A network switch, comprising: Multiple egress ports; as well as An ingress port, the ingress port being configured as: receiving a data packet from a source node, the data packet being destined for a destination node, the source node and the destination node being located within a network; identifying a candidate egress port among the plurality of egress ports, the candidate egress port being a candidate for forwarding the data packet to the destination node via the network; selecting a target egress port by arbitrating between the candidate egress ports based on traffic congestion information of the plurality of egress ports, the traffic congestion information comprising a minimum routing congestion metric of the plurality of egress ports, the traffic congestion information further comprising a non-minimum routing congestion metric of the plurality of egress ports; as well as The data packet is forwarded to the target egress port.

10. The network switch of claim 9, wherein a first subset of the candidate egress ports are candidates for forwarding the data packet to the destination node via a minimal route of the network, and a second subset of the candidate egress ports are candidates for forwarding the data packet to the destination node via a non-minimal route of the network.

11. The network switch of claim 10, wherein the non-minimal route of the network has a greater number of hops than the minimal route of the network.

12. The network switch of claim 10, wherein arbitrating between the candidate egress ports comprises: A weighted sum of the minimum routing congestion metric and the non-minimum routing congestion metric of the egress port is calculated, wherein the weight of the minimum routing congestion metric of the second subset of the candidate egress ports is greater than the weight of the minimum routing congestion metric of the first subset of the candidate egress ports.

13. The network switch of claim 9, wherein the minimum route congestion metric indicates a traffic backlog at the candidate egress port for a minimum route of the network, and the non-minimum route congestion metric indicates a traffic backlog at the candidate egress port for a non-minimum route of the network.

14. The network switch of claim 9, wherein the minimum route congestion metric indicates a traffic delay at the candidate egress port for a minimum route of the network, and the non-minimum route congestion metric indicates a traffic delay at the candidate egress port for a non-minimum route of the network.

15. The network switch of claim 9, further comprising a feedback structure, wherein each of the egress ports is configured to: collecting the traffic congestion information; and The traffic congestion information is sent to the ingress port via the feedback structure.

16. A system comprising: First node; a first network switch connected to the first node; Second node; as well as a second network switch connected to the second node, the first network switch and the second network switch being located within a network, the second network switch being configured as: receiving a data packet from the second node at an ingress port of the second network switch; identifying an egress port of the second network switch, the egress port being a candidate for forwarding the data packet from the second node to the first node via the network; receiving traffic congestion information from the egress port, the traffic congestion information comprising a minimum routing congestion metric of the egress port, the traffic congestion information further comprising a non-minimum routing congestion metric of the egress port; selecting a target egress port by arbitrating between the egress ports based on the traffic congestion information of the egress ports; as well as The data packet is forwarded to the target egress port.

17. The system of claim 16, wherein the network has a mesh topology.

18. The system of claim 16, wherein the second network switch comprises a feedback structure, and the traffic congestion information from the egress port is received at the egress port via the feedback structure.

19. The system of claim 16, wherein arbitrating between the egress ports comprises: The egress port is assigned a weight for an arbitration process based on whether the egress port provides a minimal route or a non-minimal route to the first node.

20. The system of claim 19, wherein a first weight for the minimum route congestion metric is assigned to the egress port that provides a minimum route to the first node, a second weight for the minimum route congestion metric is assigned to the egress port that provides a non-minimum route to the first node, and the second weight is greater than the first weight.

Images