Multipath routing for switch fabrics

Abstract

The present disclosure relates to multi-path routing of a switch fabric. The switch fabric uses traffic congestion information to inform its timely use of non-minimal routes. An ingress port of a network switch collects traffic congestion information from egress ports of the network switch. The traffic congestion information includes minimal and non-minimal route congestion metrics for the egress ports. Candidate egress ports are identified for forwarding a packet to a destination node. One of the candidate egress ports is selected in accordance with the traffic congestion information. The selection process favors selecting some candidate egress ports over others. In particular, non-minimal routes to the destination node and candidate egress ports having high minimal route congestion metrics are disfavored by the selection process.

Description

[0001] Cross-references to related applications

[0002] This application claims priority to U.S. Provisional Application No. 63 / 589,116, filed October 10, 2023, which is incorporated herein by reference. Background Technology

[0003] Network switching is a fundamental concept in computer networks, involving the forwarding of data packets between nodes within a network. Network switches analyze the destination of incoming data packets and use this information to make forwarding decisions, thereby executing data transmission within the network. Therefore, network switches can be used to facilitate connections and communication between multiple nodes within a network. Bandwidth is a crucial factor in network switching performance because consistently high bandwidth results in faster communication between nodes. Latency is another important factor affecting network switching performance. Attached Figure Description

[0004] The various aspects of this disclosure can be best understood by referring to the following detailed description with reference to the accompanying drawings.

[0005] Figure 1 This is a schematic diagram of a network system according to some implementation methods.

[0006] Figure 2 This is a schematic diagram of an exchange structure according to some implementation methods.

[0007] Figure 3 This is a block diagram of a network switch according to some implementation methods.

[0008] Figure 4 This is a block diagram of a network switch according to some implementation methods.

[0009] Figure 5 This is a schematic diagram of a packet forwarding method according to some implementation methods.

[0010] Figure 6 This is a block diagram of a network switch according to some implementation methods.

[0011] Unless otherwise stated, corresponding numbers and symbols in the various figures generally refer to the corresponding parts. The figures are drawn to clearly illustrate relevant aspects of this disclosure and are not necessarily drawn to scale. Detailed Implementation

[0012] The following disclosure provides several different examples for implementing different features. Specific examples of components and arrangements are described below to simplify this disclosure. Of course, these are merely examples and are not intended to be limiting.

[0013] A switching fabric is used for packet forwarding in a computing system. A switching fabric consists of multiple network switches and links between them. Nodes can use the switching fabric to communicate. For example, a first node can connect to a first network switch in the switching fabric, a second node can connect to a second network switch in the switching fabric, and the first node can send packets to the second node via the switching fabric. Specifically, the first node can send packets to the ingress port of the first network switch (to which the first node is connected), and the packets can be forwarded via the switching fabric to the egress port of the second network switch (to which the second node is connected).

[0014] A switching fabric has multiple routes that can be used to forward packets from the source node to the destination node. These routes can be minimum routes or non-minimum routes. A minimum route has the fewest hops (or link crossovers) from the source node to the destination node. A non-minimum route has more hops than a minimum route. Generally, the minimum route is the most efficient route for forwarding packets from the source node to the destination node because it can have lower latency than a non-minimum route (due to its inherently fewer hops). However, when there are not enough available minimum routes to carry the required bandwidth, opportunistic use of non-minimum routes can allow for improved switching fabric performance, as higher bandwidth can be achieved by increasing the number of available routes. Minimum and non-minimum routes can be used simultaneously to forward packets from the source node to the destination node. This increased parallelism, by utilizing non-minimum routes on links that would otherwise be idle or rarely used, allows for increased bandwidth from the source node to the destination node. However, overuse of non-minimum links between some nodes can increase congestion on links shared with minimum routes between other nodes, thus degrading overall network performance. Ideally, non-minimum routes should utilize link bandwidth in a timely manner, otherwise that bandwidth will be idle, and traffic should use minimum routes to avoid congestion on shared links.

[0015] This disclosure describes a switching architecture that uses traffic congestion information to notify the appropriate use of non-minimum routes. The ingress ports of a network switch collect traffic congestion information from the egress ports of that network switch. The traffic congestion information includes minimum and non-minimum route congestion metrics for the egress ports. For example, the minimum route congestion metric for an egress port may indicate the number of packets queued at that egress port for forwarding via a minimum route using that egress port, while the non-minimum route congestion metric for an egress port may indicate the number of packets queued at that egress port for forwarding via a non-minimum route using that egress port. Other congestion metrics may also be used.

[0016] When forwarding packets received from the source node, the ingress port identifies a subset of egress ports that are candidates for routing the packets to the destination node via the network. Candidate egress ports can provide either the minimum route from the source node to the destination node or a non-minimum route. The network switch then selects the destination egress port from the candidate egress ports based on traffic congestion information.

[0017] The selection process favors certain candidate egress ports over others. Specifically, if a candidate egress port provides a non-minimum route from the source node to the destination node, it will not be selected if traffic congestion information indicates excessive congestion of minimum route traffic. Therefore, excessive congestion of minimum route traffic at the egress port mitigates the risk of it being selected by the ingress port for non-minimum route traffic. This allows for timely use of non-minimum routes when increased bandwidth between nodes is needed due to light loads or idleness on non-minimum route links, avoiding increased congestion on links serving minimum routes between other nodes. Thus, network bandwidth can be increased without excessively increasing network latency and / or power consumption.

[0018] Figure 1 This is a schematic diagram of a network system 100 according to some embodiments. The network system 100 may be a high-performance network as part of a computing system, a high-performance computing (HPC) environment, etc. In the network system 100, processors 102 (e.g., processors 102A-102D) and devices 104 (e.g., devices 104A-104D) communicate with each other through network switches 106 (e.g., network switches 106A-106D).

[0019] Processor 102 retrieves executable code from memory (not shown separately) and executes it. When executed by processor 102, the executable code enables processor 102 to perform any desired function. Processor 102 may be a microprocessor, application-specific integrated circuit, microcontroller, etc.

[0020] Device 104 includes various other hardware components, both external and internal to network system 100. For example, device 104 may include accelerators, network interface devices, memory expansion devices, etc. Device 104 may or may not have local memory accessible to processor 102. Additionally, device 104 may access system memory (not shown separately). Device 104 may communicate with processor 102 via network switch 106, or with each other.

[0021] Network switch 106 interconnects processor 102 and device 104. Network switches 106 are interconnected via network link 108 to form switching structure 110. Switching structure 110 can have any suitable topology. In some embodiments, switching structure 110 has a mesh topology.

[0022] Network switch 106 includes ports to which processor 102, device 104, and other devices of network switch 106 are connected. Processor 102 and device 104 communicate with each other via data packets transmitted between the ingress and egress ports of network switch 106 in switching structure 110. Generally, switching structure 110 can be used for communication between nodes (e.g., processor 102 and device 104). Processor 102 and device 104 are merely examples of components that can be interconnected through switching structure 110. Other components can be connected to switching structure 110. Data packets can be routed through switching structure 110. In some implementations, switching structure 110 is a Compute Fast Link (CXL) structure; device 104 is a Type 1, Type 2, or Type 3 CXL device; and network link 108 is a PCI Fast Interface.

[0023] Figure 2 This is a schematic diagram of a switching structure 200 according to some embodiments. The switching structure 200 is previously designed for... Figure 1 An example of the described switching structure 110. Switching structure 200 includes network switches 202 (e.g., network switches 202A-202D) and network links 204 (e.g., network lines 202AB-202CD). Network links 204 interconnect the network switches 202 to form a network with a desired topology. Switching structure 200 can be used by nodes 206 (e.g., nodes 206A-206D) to communicate with each other. Nodes 206 can correspond to... Figure 1 The processor 102 or device 104, or a combination of processor and device.

[0024] The switching structure 200 includes multiple routes 208 that can be used to forward packets from source node 206 to destination node 206. Three example routes from node 206A to node 206B are shown: route 208A across link 204AB, route 208B across links 204AC and 204BC, and route 208C across links 204AD and 204BD. Further, an example route 208D (crossing link 204AC) from node 206A to node 206C is shown. Finally, an example route 208E (crossing links 204AC and 204CD) from node 206A to node 206D is shown.

[0025] Route 208 can be either a minimum route or a non-minimum route. Continuing the previous example, route 208A is the minimum route from node 206A to node 206B because it has the lowest number of hops (or link crossovers) from node 206A to node 206B. Routes 208B and 208C are non-minimum routes from node 206A to node 206B because they have more hops than route 208A. Specifically, route 208B has an extra hop (crossing switch 202C) compared to route 208A, and route 208C also has an extra hop (crossing switch 202D) compared to route 208A. Furthermore, route 208D is the minimum route from node 206A to node 206C. Finally, route 208E is a non-minimum route from node 206A to node 206D.

[0026] Generally, the minimum route is the most efficient route for forwarding packets from source node 206 to destination node 206. However, the timely use of non-minimum routes can allow for improvements in the performance of the switching infrastructure. For example, although route 208A (the minimum route from node 206A to node 206B) may be more efficient than routes 208B and 208C (non-minimum routes from node 206A to node 206B), the timely use of all routes 208A, 208B, and 208C can allow for increased bandwidth from node 206A to node 206B.

[0027] Overusing some non-minimum routes between nodes can lead to congestion on other minimum routes between nodes, thereby increasing network latency and / or power consumption. For example, overusing route 208B (from node 206A to node 206B) can lead to congestion on route 208D (from node 206A to node 206C). As described in more detail later, network switch 202 will use traffic congestion information to inform the timely use of non-minimum routes so that the forwarding of traffic on non-minimum routes in switching structure 200 does not reduce the forwarding of traffic on minimum routes in switching structure 200. In this example, routes 208B, 208D, and 208E each traverse the same network link, and the timely use of route 208B (a non-minimum route) will be avoided to benefit other traffic using route 208D (the minimum route). However, the timely use of route 208B will not be avoided to benefit other traffic using route 208E (another non-minimum route).

[0028] Figure 3 This is a block diagram of a network switch 300 according to some implementation methods. The network switch 300 is... Figures 1 to 2The example of the network switch described earlier. Network switch 300 includes ports 302 (e.g., ports 302A, 302B, and 302N) and switch core 304. Ports 302 serve as connection points for nodes (e.g., processors, devices, etc.). Switch core 304 manages and forwards data packets between ports 302.

[0029] Each port 302 includes an ingress port 306 and an egress port 308. Ingress port 306 is the entry point through which data packets enter the network switch 300. Egress port 308 is the output point responsible for sending data packets to their designated destination. When a data packet arrives at ingress port 306, the network switch 300 checks the packet's destination address to determine the appropriate egress port 308 for transmission. This process, called switching or forwarding, involves performing a lookup in the network switch 300's routing table to find multiple candidate forwarding paths for the data packet. Candidate forwarding paths may include a list of egress ports for the minimum route and a list of egress ports for non-minimum routes. Ingress port 306 controls how data packets are sent to egress port 308.

[0030] Ports 302 (including ingress port 306 and egress port 308) are interconnected via switch core 304, which provides the necessary path for data packets to move from ingress port 306 to egress port 308. Switch core 304 connects ingress port 306 and egress port 308. Depending on the architecture of network switch 300, switch core 304 can be implemented using, for example, a single crossbar switch, a crossbar switch matrix, a shared bus, shared memory, a chip-wide ring, etc. In some implementations, switch core 304 includes multiple crossbar switches used for control and data transmission between ingress port 306 and egress port 308.

[0031] Components of the network switch 300 may be implemented as integrated circuits, for example, in one or more integrated circuit chips and / or one or more integrated circuit packages. For example, the network switch 300 may include a processor, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a memory, combinations thereof, etc. One or more modules in the network switch 300 may be partially or wholly embodied as software and / or hardware for performing any of the functions described herein. For example, each of the buffers, cross switches, transmitters, receivers, structures, etc., described herein may be embodied as a logic block of an integrated circuit.

[0032] Figure 4 This is a block diagram of a network switch 400 according to some implementations. The network switch 400 is a previously designed... Figures 1 to 3An example of a network switch is described. The figure shows additional components of network switch 400 (including switch core 404, multiple ingress ports 406 (including ingress ports 406A and 406B) and multiple egress ports 408 (including egress ports 408A and 408B)). A logical flowchart of packet forwarding from ingress port 406 to egress port 408 is shown.

[0033] First, the components of network switch 400 will be described. Each ingress port 406 includes a receiver 412, an input buffer 414, and an input queue 416. Receiver 412 receives data packets on the physical link from a source node connected via ingress port 406. The data packets are destined for a destination node connected via one or more egress ports 408. The source node and / or destination node may be directly connected to network switch 400, or one or more network components (e.g., additional switches) may exist between network switch 400 and the source / destination nodes(s). Input buffer 414 is connected to receiver 412. Received data packets are stored in input buffer 414. Input queue 416 is connected to input buffer 414 and receiver 412. Input queue 416 is an input controller that controls the transmission of data packets from input buffer 414 to the output buffer of egress port 408. Requests to send data packets to egress port 408 are queued by receiver 412 at input queue 416. Input queue 416 arbitrates its queued requests and selects the request to be processed. Input queue 416 determines which egress port 408 the packet used for the selected request should be forwarded to. For example, a lookup unit (not shown separately) can extract the appropriate header(s) from the packet and use them to determine the destination node of the packet. Input queue 416 can receive the lookup results from the lookup unit.

[0034] Each egress port 408 includes an output queue 422, an output buffer 424, a transmitter 426, and a load monitor 428. The output queue 422 is an output controller that controls the reception of data packets from the input buffer of the ingress port 406 to the output buffer 424. Requests to receive data packets from the ingress port 406 are queued at the output queue 422. The output queue 422 arbitrates its queued requests and selects the request to process. The output buffer 424 is connected to the output queue 422. Received data packets are stored in the output buffer 424. The transmitter 426 is connected to the output buffer 424. The transmitter 426 reads data packets from the output buffer 424 and transmits the data packets to the destination node by sending signals down the physical link. Therefore, by controlling the reception of data packets in the output buffer 424, the output queue 422 effectively controls the transmitter 426's reading of data packets from the output buffer 424.

[0035] Switch core 404 is described as having an example implementation based on cross switches, comprising multiple cross switches that are different from each other. In this example, switch core 404 includes packet cross switches 432 and load cross switches 434. As previously mentioned, other types of switch cores can also be utilized.

[0036] Packet crossover switch 432 is connected to the input buffer 414 and input queue 416 of each ingress port 406, and to the output queue 422 and output buffer 424 of each egress port 408. Transmission requests, transmission grants, and packets are transmitted via packet crossover switch 432. In some implementations, multiple packet crossover switches or other switch core architectures may be utilized. For example, transmission requests can be sent via request crossover switch, transmission grants can be sent via grant crossover switch, and packets can be transmitted via data crossover switch.

[0037] The load crossbar switch 434 is connected to the input queue 416 of each inlet port 406 and to the load monitor 428 of each outlet port 408. Traffic congestion information is transmitted through the load crossbar switch 434, as described in more detail later. The load crossbar switch 434 is a dedicated feedback structure separate from the packet crossbar switch 432.

[0038] A logic flowchart for forwarding packets from ingress port 406 to egress port 408 will now be described. This logic flowchart is an example, and other methods of packet forwarding can be utilized. Receiver 412 at ingress port 406 receives a packet and stores it in input buffer 414 of ingress port 406. Requests to transmit packets are queued at input queue 416 of ingress port 406, where the ingress port selects a request to process. Input queue 416 sends a transmission request for the packet to output queue 422 of output port 408 via switch core 404 (e.g., packet crossbar switch 432). The transmission request includes a description of the packet; for example, the request may include information from the packet header.

[0039] Transmission requests are queued at output queue 422 of egress port 408, and the egress port then selects a transmission request to process. Output queue 422 determines whether and when to authorize the transmission request. For example, output queue 422 may determine which transmission request to authorize next based on the packet description of the transmission request and the current state of output buffer 424 of egress port 408. In response to the authorization of the transmission request, output queue 422 sends the transmission authorization to input queue 416 of ingress port 406 through switch core 404 (e.g., packet crossbar switch 432).

[0040] A transmission grant is a notification instructing input queue 416 to move a data packet from input buffer 414 at ingress port 406 to output buffer 424 at egress port 408. In response to receiving the transmission grant, input queue 416 transmits the data packet from input buffer 414 to output buffer 424 via switch core 404 (e.g., packet crossbar switch 432). The transmitter 426 at egress port 408 then reads the data packet from output buffer 424. Therefore, output queue 422 controls the reading of data packets by transmitter 426 (and thus determines which packets are sent by transmitter 426) through the authorization of the transmission request.

[0041] When a data packet is received at ingress port 406 from a source node or another switch, multiple egress ports 408 may exist as candidate egress ports 408 for forwarding the packet to the destination node via network routing. Each candidate egress port 408 can provide either a minimum route or a non-minimum route to the destination node. That is, a first subset of egress ports 408 may be candidates for forwarding data packets to the destination node via the minimum route of the network, and a second subset of egress ports 408 may be candidates for forwarding data packets to the destination node via a non-minimum route of the network. Both minimum and non-minimum routes can be used as needed to increase the bandwidth from the source node to the destination node. For example, when forwarding multiple data packets, the multiple data packets can be multiplexed across minimum and non-minimum routes. When forwarding a data packet to the destination node, ingress port 406 can identify multiple candidate egress ports 408 and then select one egress port from the candidate egress ports 408 for forwarding.

[0042] Ingress port 406 can use routing table 436 to identify candidate egress ports 408. Routing table 436 can be stored in input queue 416 of ingress port 406. Routing table 436 includes a mapping from destination nodes to egress ports 408 that provide routes to those destination nodes. When ingress port 406 receives a packet, input queue 416 can identify the destination node of the packet (e.g., using a destination node identifier in the packet header) and then look up that destination node in routing table 436 to identify candidate egress ports 408 for reaching that destination node. Routing table 436 can indicate whether each candidate egress port 408 provides a minimum or non-minimum route to the destination node.

[0043] Once a candidate egress port 408 is identified by an ingress port 406, the ingress port 406 selects one of the candidate egress ports 408 based on the traffic congestion information of the candidate egress ports 408. For example, the traffic congestion information can be stored in an input queue 416. The input queue 416 can arbitrate among the candidate egress ports 408 based on the traffic congestion information to determine the selected egress port 408.

[0044] Traffic congestion information includes metrics for each egress port 408. Transmission requests for packets destined for different destination nodes can be queued in output queue 422 of egress port 408. Packets queued for transmission at egress port 408 can be on the minimum route to their destination node or on a non-minimum route to their destination node. Traffic congestion information includes minimum route congestion metrics for egress port 408 and non-minimum route congestion metrics for egress port 408.

[0045] For each egress port 408, the minimum route congestion metric indicates the degree of congestion of its minimum route traffic. As used herein, minimum route traffic is the traffic queued at egress port 408 (including traffic from ingress port 406 and for all destinations) for which minimum routing is provided by egress port 408. In some implementations, the minimum route congestion metric indicates the traffic backlog at egress port 408 for minimum routes to the network. For example, the minimum route congestion metric could indicate the number of packets queued at output queue 422 of egress port 408 for forwarding via minimum routes. In some implementations, the minimum route congestion metric indicates the traffic delay at egress port 408 for minimum routes to the network. Other suitable minimum route congestion metrics may also be utilized.

[0046] For each egress port 408, the non-minimum route congestion metric indicates the degree of congestion of its non-minimum route traffic. As used herein, non-minimum route traffic is traffic queued at egress port 408 (including traffic from ingress port 406 and destined for all destinations) for which non-minimum routing is provided by egress port 408. In some implementations, the non-minimum route congestion metric indicates traffic backlog at egress port 408 for non-minimum routes to the network. For example, the non-minimum route congestion metric could indicate the number of packets queued at output queue 422 of egress port 408 for forwarding via non-minimum routes. In some implementations, the non-minimum route congestion metric indicates traffic latency at egress port 408 for non-minimum routes to the network. Other suitable non-minimum route congestion metrics may also be utilized.

[0047] As can be clearly seen from the preceding description, ingress port 406 tracks two separate metrics for each egress port 408: the minimum route congestion metric and the non-minimum route congestion metric. Therefore, each ingress port 406 knows the level of congestion of the minimum route traffic at each egress port 408. Furthermore, each ingress port 406 knows the level of congestion of the non-minimum route traffic at each egress port 408.

[0048] As previously described, once ingress port 406 identifies candidate egress ports 408 for forwarding packets to the destination node, it can select one of the egress ports 408 through arbitration among the candidate egress ports 408 based on traffic congestion information. The arbitration process is weighted to prioritize some candidate egress ports 408 over others. Specifically, if a candidate egress port 408 provides a non-minimum route from the source node to the destination node, then the candidate egress port 408 will not be favored by the arbitration process if the traffic congestion information indicates that the candidate egress port 408 has excessively high minimum route traffic congestion. Thus, a good candidate egress port 408 for timely non-minimum routing can be preferred over other egress ports 408 that are considered bad candidates because their use would negatively impact the minimum traffic sharing the same egress port 408.

[0049] The arbitration process may include, for each corresponding candidate egress port 408, calculating a weighted sum of traffic congestion information (e.g., minimum and non-minimum route congestion metrics) for that candidate egress port 408. For a non-minimum candidate egress port 408, the minimum route congestion metric may have a higher weight than the non-minimum route congestion metric. Therefore, a non-minimum candidate egress port 408 congested due to minimum traffic representing other routes is less likely to be selected than a non-minimum candidate egress port 408 equally congested due to non-minimum traffic representing other routes. Similarly, when calculating the weighted sum, the minimum route congestion metric of a non-minimum candidate egress port 408 may have a higher weight than the minimum route congestion metric of a minimum candidate egress port 408. Therefore, if a non-minimum candidate egress port 408 is also congested due to minimum traffic, the probability of selecting a non-minimum candidate egress port 408 is lower than the probability of selecting a minimum candidate egress port 408.

[0050] Traffic congestion information is collected at each egress port 408 and sent to each ingress port 406. A load monitor 428 at egress port 408 can collect the traffic congestion information and send it to the input queue 416 of ingress port 406. The traffic congestion information can be sent via switch core 404 (e.g., load crossbar 434). Input queue 416 tracks the traffic congestion information at egress port 408. The traffic congestion information can be used by ingress port 406 to inform it of the timely use of non-minimum routes (as previously described).

[0051] Figure 5 This is a schematic diagram of a packet forwarding method 500 according to some implementation methods. (The diagram will be combined with...) Figure 4The packet forwarding method 500 is described. The packet forwarding method 500 can be performed by the network switch 400 during the forwarding of packets from the ingress port 406 to the egress port 408.

[0052] Network switch 400 executes step 502, receiving a data packet from the source node at the ingress port of the first network switch. This data packet is destined for a destination node connected to the second network switch, which is located within the network. For example, the data packet can be received at ingress port 406 of network switch 400. The source and destination nodes are also located within the network. This network can have any desired topology, such as mesh topology, tree topology, etc.

[0053] Network switch 400 executes step 504, identifying the egress ports of the first network switch. Egress ports are candidates for forwarding data packets to a destination node via the network. For example, ingress port 406 can look up egress port 408 in routing table 436 to which data packets can be forwarded, thus routing them to the destination node. Candidate egress ports 408 can provide either a minimum or non-minimum route to the destination node. A first subset of egress ports 408 can be candidates for forwarding data packets to the destination node via the minimum route through the network, and a second subset of egress ports 408 can be candidates for forwarding data packets to the destination node via a non-minimum route through the network.

[0054] Network switch 400 executes step 506, selecting a target egress port by arbitrating among egress ports based on traffic congestion information of the egress port. The traffic congestion information includes both a minimum routing congestion metric and a non-minimum routing congestion metric for the egress port. For example, the target egress port 408 can be selected by ingress port 406 based on both the minimum and non-minimum routing congestion metrics.

[0055] In some implementations, the minimum route congestion metric of the traffic congestion information indicates traffic backlog at egress port 408 for the minimum route to the network, and the non-minimum route congestion metric indicates traffic backlog at egress port 408 for the non-minimum route to the network. In some implementations, the minimum route congestion metric of the traffic congestion information indicates traffic delay at egress port 408 for the minimum route to the network, while the non-minimum route congestion metric indicates traffic delay at egress port 408 for the non-minimum route to the network.

[0056] Arbitration among egress ports 408 may include disapproving egress ports with high minimum route congestion metrics from the second subset of egress ports 408 (used to provide non-minimum routes to the destination node). For example, egress ports in the second subset of egress ports 408 that have large traffic backlogs or large traffic delays on minimum routes may be disapproved.

[0057] In this implementation, the combined metric of candidate egress port 408 can be calculated by summing the products: the minimum route congestion metric of candidate egress port 408 multiplied by the minimum weight can be added to the non-minimum route congestion metric of candidate egress port 408 multiplied by the non-minimum weight. Egress port 408 can then be selected statistically or deterministically such that egress port 408 with a lower combined metric value is preferred over egress port 408 with a higher combined metric value.

[0058] Candidate egress ports 408 can be weighted for the arbitration process based on whether they provide a minimum or non-minimum route to the destination node, and disapproving of egress port 408 can include increasing the weight of the egress port. When a candidate egress port 408 provides a minimum route to the destination node, a first minimum weight and a first non-minimum weight can be used to calculate the combined metric of the candidate egress port 408. When a candidate egress port 408 provides a non-minimum route to the destination node, a second minimum weight and a second non-minimum weight can be used to calculate the combined metric of the candidate egress port 408. The second minimum weight is greater than the first minimum weight. In other words, when calculating the combined metric of egress ports 408, the weight of the minimum route congestion metric for a second subset of egress ports 408 is greater than the weight of the minimum route congestion metric for a first subset of egress ports 408. As a result, the minimum route congestion metric for a second subset of egress ports 408 has a heavier weight than the minimum route congestion metric for a first subset of egress ports 408. Similarly, the minimum route congestion metric for a second subset of egress ports 408 can have a heavier weight than the non-minimum route congestion metric for a second subset of egress ports 408. Furthermore, the second non-minimum weight may (or may not) be different from the first non-minimum weight.

[0059] Traffic congestion information can be stored in the input queue 416 of the ingress port 406. This traffic congestion information may have been previously received from the load monitor 428 of the egress port 408. For example, traffic congestion information may have already been collected at the egress port 408 and then sent to the ingress port 406. Traffic congestion information can be sent from the egress port 408 to the ingress port 406 via a feedback structure, such as a load crossbar switch 434.

[0060] In some implementations, candidate egress ports 408 may also be identified, at least in part, based on traffic congestion information (in step 504). Specifically, if the minimum routing congestion metric of an egress port 408 is too large, then that egress port 408 may not be considered a candidate egress port 408. In some implementations, candidate egress ports 408 are those ports whose minimum routing congestion metric is less than a predetermined threshold.

[0061] Network switch 400 performs step 508 of forwarding data packets to the target egress port. For example, the input queue 416 of ingress port 406 may send a data packet transmission request to the output queue 422 of the target egress port 408. After receiving the corresponding transmission authorization from the target egress port 408, ingress port 406 transmits the data packet from its input buffer 414 to the output buffer 424 of the target egress port 408.

[0062] Figure 6 This is a block diagram of a network switch 600 according to some implementations. The network switch 600 is the one previously described. Figure 4 An example of a network switch 400. Network switch 600 may include a processor 602 and a memory 604. Memory 604 may be a non-transitory computer-readable medium storing programs for execution by the processor. In this embodiment, one or more modules in network switch 600 may be embodied in part or in whole as software for performing any of the functions described herein. For example, memory 604 may include: instructions 606 for receiving data packets from a source node at an ingress port of a first network switch destined for a destination node connected to a second network switch, the first and second network switches being located within a network; instructions 608 for identifying the first network switch as a candidate egress port for forwarding data packets to the destination node via the network; instructions 610 for selecting a target egress port by arbitration among egress ports based on traffic congestion information, including a minimum routing congestion metric for the egress port, and a non-minimum routing congestion metric for the egress port; and / or instructions 612 for forwarding data packets to the target egress port.

[0063] Several variations are envisioned. For example, the switching technology described in this paper can be applied to other types of switching architectures, such as Ethernet.

[0064] In an example implementation, a method includes: receiving a data packet from a source node at an ingress port of a first network switch, the data packet destined for a destination node connected to a second network switch, the first and second network switches being located within a network; identifying an egress port of the first network switch, the egress port being a candidate for forwarding the data packet to the destination node via the network; selecting a target egress port by arbitration among the egress ports based on traffic congestion information of the egress port, the traffic congestion information including a minimum routing congestion metric of the egress port and a non-minimum routing congestion metric of the egress port; and forwarding the data packet to the target egress port. In some implementations of the method, a first subset of egress ports are candidates for forwarding the data packet to the destination node via a minimum route through the network, and a second subset of egress ports are candidates for forwarding the data packet to the destination node via a non-minimum route through the network. In some implementations of the method, arbitration among the egress ports includes: calculating a weighted sum of the minimum routing congestion metric and the non-minimum routing congestion metric of the egress ports, the minimum routing congestion metric of the second subset of egress ports having a greater weight than the non-minimum routing congestion metric of the second subset of egress ports. In some embodiments of the method, the minimum route congestion metric indicates traffic backlog at the egress port for the minimum route of the network, while the non-minimum route congestion metric indicates traffic backlog at the egress port for a non-minimum route of the network. In some embodiments of the method, the minimum route congestion metric indicates traffic delay at the egress port for the minimum route of the network, while the non-minimum route congestion metric indicates traffic delay at the egress port for a non-minimum route of the network. In some embodiments of the method, the network has a mesh topology. In some embodiments, the method further includes: collecting traffic congestion information at the egress port; and sending the traffic congestion information to the ingress port. In some embodiments of the method, the traffic congestion information is sent from the egress port to the ingress port via a feedback structure of a first network switch.

[0065] In an example implementation, a network switch includes: a plurality of egress ports; and an ingress port configured to: receive data packets from a source node destined for a destination node, the source and destination nodes being located within a network; identify candidate egress ports from the plurality of egress ports, the candidate egress ports being candidates for forwarding data packets to the destination node via the network; select a target egress port by arbitration among the candidate egress ports based on traffic congestion information of the plurality of egress ports, the traffic congestion information including a minimum route congestion metric of the plurality of egress ports, and a non-minimum route congestion metric of the plurality of egress ports; and forward the data packets to the target egress port. In some implementations of the network switch, a first subset of the candidate egress ports are candidates for forwarding data packets to the destination node via the minimum route of the network, and a second subset of the candidate egress ports are candidates for forwarding data packets to the destination node via a non-minimum route of the network. In some implementations of the network switch, the non-minimum route of the network has a higher hop count than the minimum route of the network. In some implementations of network switches, arbitration among candidate egress ports includes: calculating a weighted sum of minimum route congestion metric and non-minimum route congestion metric for the egress ports, where the weight of the minimum route congestion metric for a second subset of candidate egress ports is greater than the weight of the minimum route congestion metric for a first subset of candidate egress ports. In some implementations of network switches, the minimum route congestion metric indicates traffic backlog at the candidate egress port for the minimum route to the network, while the non-minimum route congestion metric indicates traffic backlog at the candidate egress port for the non-minimum route to the network. In some implementations of network switches, the minimum route congestion metric indicates traffic latency at the candidate egress port for the minimum route to the network, while the non-minimum route congestion metric indicates traffic latency at the candidate egress port for the non-minimum route to the network. In some implementations, the network switch also includes a feedback structure, wherein each egress port is configured to: collect traffic congestion information; and send the traffic congestion information to the ingress port via the feedback structure.

[0066] In an example implementation, a system includes: a first node; a first network switch connected to the first node; a second node; and a second network switch connected to the second node, the first and second network switches being located within a network. The second network switch is configured to: receive data packets from the second node at an ingress port of the second network switch; identify egress ports of the second network switch, the egress ports being candidates for forwarding data packets from the second node to the first node via the network; receive traffic congestion information from the egress ports, the traffic congestion information including a minimum route congestion metric for the egress port, the traffic congestion information also including a non-minimum route congestion metric for the egress port; arbitrate among the egress ports based on the traffic congestion information of the egress ports to select a target egress port; and forward data packets to the target egress port. In some implementations of the system, the network has a mesh topology. In some implementations of the system, the second network switch includes a feedback structure and receives traffic congestion information from the egress ports at the ingress port via the feedback structure. In some implementations of the system, arbitrating among the egress ports includes: assigning weights to the arbitration process of the egress ports based on whether the egress port provides a minimum or non-minimum route to the first node. In some implementations of the system, a first weight of the minimum route congestion metric is assigned to the egress port that provides the minimum route to the first node, and a second weight of the minimum route congestion metric is assigned to the egress port that provides a non-minimum route to the first node, with the second weight being greater than the first weight.

[0067] The foregoing has outlined features of several examples to enable those skilled in the art to better understand the various aspects of this disclosure. Various modifications and combinations of the illustrative examples and other examples will be apparent to those skilled in the art upon reference to this specification. Therefore, the appended claims are intended to cover any such modifications.

Claims

1. A method comprising: A data packet from a source node is received at the ingress port of a first network switch, and the data packet is destined for a destination node connected to a second network switch. The first network switch and the second network switch are located within a network. Identify the egress port of the first network switch, the egress port being a candidate for forwarding the data packet to the destination node via the network, the egress port including non-minimum candidate egress ports, the non-minimum candidate egress ports being candidates for forwarding the data packet to the destination node via a non-minimum route of the network; Obtain traffic congestion information for the egress port. The traffic congestion information includes the minimum route congestion metric of the non-minimum candidate egress port and the non-minimum route congestion metric of the non-minimum candidate egress port. The minimum route congestion metric represents the congestion status of the minimum route of the network for the queued data packets at the non-minimum candidate egress port, and the non-minimum route congestion metric represents the congestion status of the non-minimum route of the network for the queued data packets at the non-minimum candidate egress port. The target egress port is selected from the non-minimum candidate egress ports by favoring the non-minimum candidate egress ports with lower values ​​of the minimum route congestion metric rather than the non-minimum candidate egress ports with higher values ​​of the minimum route congestion metric, wherein the target egress port is one of the biased non-minimum candidate egress ports. as well as The data packet is forwarded to the target egress port.

2. The method of claim 1, wherein the egress port further includes a minimum candidate egress port, the minimum candidate egress port being a candidate for forwarding the data packet to the destination node via the minimum route of the network.

3. The method of claim 2, wherein the bias towards the non-minimum candidate egress port having a lower value of the minimum routing congestion metric comprises: Calculate the weighted sum of the minimum route congestion metric and the non-minimum route congestion metric for the egress port, wherein the minimum route congestion metric for the non-minimum candidate egress port has a higher weight than the non-minimum route congestion metric for the non-minimum candidate egress port.

4. The method of claim 1, wherein the minimum route congestion metric indicates traffic backlog at the egress port for the minimum route of the network, and the non-minimum route congestion metric indicates traffic backlog at the egress port for the non-minimum route of the network.

5. The method of claim 1, wherein the minimum route congestion metric indicates the traffic delay at the egress port for the minimum route of the network, and the non-minimum route congestion metric indicates the traffic delay at the egress port for the non-minimum route of the network.

6. The method of claim 1, wherein the network has a mesh topology.

7. The method according to claim 1, further comprising: Collect the traffic congestion information at the exit port; as well as The traffic congestion information is sent to the ingress port.

8. The method according to claim 7, wherein the traffic congestion information is sent from the egress port to the ingress port via the feedback structure of the first network switch, the data packet is forwarded to the target egress port via the data structure of the first network switch, and the feedback structure is separate from the data structure.

9. A network switch, comprising: Multiple exit ports; as well as The ingress port is configured as follows: Receive data packets from a source node, the data packets being destined for a destination node, the source node and the destination node being located within the network; Identify candidate egress ports among the plurality of egress ports. The candidate egress ports are candidates for forwarding the data packets to the destination node via the network. The candidate egress ports include non-minimum candidate egress ports, which are candidates for forwarding the data packets to the destination node via a non-minimum route of the network. Obtain traffic congestion information for the candidate egress port. The traffic congestion information includes the minimum route congestion metric for the non-minimum candidate egress port and the non-minimum route congestion metric for the non-minimum candidate egress port. The minimum route congestion metric represents the congestion status of the minimum route of the network for the queued data packets at the non-minimum candidate egress port, and the non-minimum route congestion metric represents the congestion status of the non-minimum route of the network for the queued data packets at the non-minimum candidate egress port. The target egress port is selected from the non-minimum candidate egress ports by favoring the non-minimum candidate egress ports with lower values ​​of the minimum route congestion metric rather than the non-minimum candidate egress ports with higher values ​​of the minimum route congestion metric, wherein the target egress port is one of the biased non-minimum candidate egress ports. as well as The data packet is forwarded to the target egress port.

10. The network switch of claim 9, wherein the candidate egress port further includes a minimum candidate egress port, the minimum candidate egress port being a candidate for forwarding the data packet to the destination node via the minimum route of the network.

11. The network switch of claim 10, wherein the non-minimum route of the network has more hops than the minimum route of the network.

12. The network switch of claim 10, wherein the non-minimum candidate egress port biased towards having a lower value of the minimum routing congestion metric comprises: Calculate the weighted sum of the minimum route congestion metric and the non-minimum route congestion metric for the candidate egress port, wherein the weight of the minimum route congestion metric for the non-minimum candidate egress port is greater than the weight of the non-minimum route congestion metric for the non-minimum candidate egress port.

13. The network switch of claim 9, wherein the minimum route congestion metric indicates traffic backlog at the candidate egress port for the minimum route of the network, and the non-minimum route congestion metric indicates traffic backlog at the candidate egress port for the non-minimum route of the network.

14. The network switch of claim 9, wherein the minimum route congestion metric indicates the traffic delay at the candidate egress port for the minimum route of the network, and the non-minimum route congestion metric indicates the traffic delay at the candidate egress port for the non-minimum route of the network.

15. The network switch of claim 9, further comprising a feedback structure, wherein each egress port in the candidate egress ports is configured as follows: Collect the traffic congestion information; and The traffic congestion information is sent to the ingress port via the feedback structure.

16. A system comprising: First node; A first network switch connected to the first node; Second node; as well as A second network switch is connected to the second node. The first network switch and the second network switch are located within the network. The second network switch is configured as follows: Receive data packets from the second node at the ingress port of the second network switch; Identify the egress port of the second network switch, the egress port being a candidate for forwarding the data packet from the second node to the first node via the network, the egress port including non-minimum candidate egress ports, the non-minimum candidate egress ports being candidates for forwarding the data packet to the first node via a non-minimum route of the network; Traffic congestion information is received from the egress port. The traffic congestion information includes the minimum route congestion metric of the non-minimum candidate egress port and the non-minimum route congestion metric of the non-minimum candidate egress port. The minimum route congestion metric represents the congestion status of the minimum route of the network for the queued data packets at the non-minimum candidate egress port, and the non-minimum route congestion metric represents the congestion status of the non-minimum route of the network for the queued data packets at the non-minimum candidate egress port. The target egress port is selected from the non-minimum candidate egress ports by favoring the non-minimum candidate egress ports with lower values ​​of the minimum route congestion metric rather than the non-minimum candidate egress ports with higher values ​​of the minimum route congestion metric, wherein the target egress port is one of the biased non-minimum candidate egress ports. as well as The data packet is forwarded to the target egress port.

17. The system of claim 16, wherein the network has a mesh topology.

18. The system of claim 16, wherein the second network switch includes a feedback structure and receives the traffic congestion information from the egress port at the ingress port via the feedback structure.

19. The system of claim 16, wherein the non-minimum candidate egress port biased towards having a lower value of the minimum routing congestion metric comprises: The arbitration process weights are assigned to the egress port based on whether it provides a minimum route or a non-minimum route to the first node.

20. The system of claim 19, wherein a first weight for the minimum route congestion metric is assigned to the egress port providing the minimum route to the first node, a second weight for the minimum route congestion metric is assigned to the egress port providing a non-minimum route to the first node, and the second weight is greater than the first weight.

Images