Multi-level network switch

Abstract

A network switch has multiple crosspoints and multiple header levels for controlling the crosspoints. In an example implementation, a device has: a plurality of header levels, each header level having an inbound queue, an outbound queue, a request crosspoint connected to the inbound and outbound queues, and a grant crosspoint connected to the inbound and outbound queues; a receiver; and a header level selector. The header level selector is configured to: receive a transmission request for a packet from the receiver; select a header level from the header levels based on a header of the packet; and place the transmission request in the inbound queue of the header level.

Description

BACKGROUNDNetwork switching is a basic concept in computer networks that involves forwarding data packets between devices within a network. A network switch analyzes the destination of an incoming packet and uses this information to make decisions about forwarding and thus perform the data transfer within the network. Network switches reduce network congestion and improve overall efficiency compared to legacy hub-based networks. Bandwidth and transmission rate are important factors for network switch performance because persistently high bandwidth results in faster communication between devices.BRIEF DESCRIPTION OF THE DRAWINGSAspects of the present disclosure will be best understood from the following detailed description when read in conjunction with the accompanying figures. FIG. 1 is a schematic illustration of a network system, in accordance with some embodiments. FIG. 2 is a block diagram of a network switch as employed in some cases. FIG. 3 is a block diagram of a network switch as employed in some cases. FIG. 4 is a diagram of a method for forwarding packets, in accordance with some implementations. FIG. 5 is a diagram of a method for selecting levels according to some implementations. FIG. 6 is a block diagram of a network switch as used in some implementations.Corresponding numerals and symbols in the various figures generally refer to the corresponding parts, unless otherwise indicated. The figures are for illustrating the relevant aspects of the disclosure and are not necessarily drawn to scale.DETAILED DESCRIPTIONThe following disclosure provides many different examples of implementing different features. To simplify the present disclosure, specific examples of components and arrangements will be described below. These are, of course, only examples and are not intended to be limiting.Network switches are used for forwarding data packets in local area networks. The packets may be large (e.g., in the case of Ethernet frames) or small (e.g., in the case of HPC network frames). For example, a large packet may comprise more than 256 bytes. While network switches may typically operate at line rates in forwarding large packets, they sometimes have problems in forwarding small packets to maintain line rates. This may be attributed at least in part to the effects of the clock rates. Increasing the clock frequency is a rudimentary way to maintain the line rates in forwarding small packets, but may not be practicable with increasing demand for higher line rates.The present disclosure describes a network switch that includes multiple data crossbar. Packets are transmitted from an input buffer of an input port to an output buffer of an output port over the data crossbar. In particular, multiple packets are transmitted in parallel across the data crossbar. Indeed, multiple packets may be transmitted from the input buffer to the output buffer per clock cycle. The number of packets transmitted per clock cycle can thus be increased compared to network switches using a single data crossbar, whereby the total amount of transmitted packets can be increased without the need to increase the clock speed. The line rate of the network switch can thus be maintained even in the case of small packets without the disadvantages of an increase in the clock rates having to be accepted.In an example implementation, each port of a network switch includes components of multiple header plans that control the parallel transmission of packets over the data crossbar. Each header layer may transmit packets independently of each other. The use of multiple header levels allows for more efficient control of the multiple data crossbar, which may be advantageous over using a single data crossbar or multiple data crossbar without multiple header levels. A first header level of the header levels may control the transfer of a first packet from an input buffer to an output buffer via the data crossbar, while a second header level of the header levels may control the transfer of a second packet from the input buffer to the output buffer via the data crossbar. The packets may be part of a packet flow. In this case, a single header level of the header levels may control the transmission of the packets from the input buffer to the output buffer over the multiple data crossbar. In the transmission of a packet, an input queue of a header level may receive a sequence number for the packet and transmit this sequence number to the output buffer together with the packet. The packets may be read from the output buffer and transmitted from the output port in the order of their sequence numbers. In this way, the order of the packets in the packet flow can be maintained even if congestion on the data crossbar causes the packets to not arrive in the proper order in the output buffer.FIG. 1 is a diagram of a network system 100, in accordance with some implementations. The network system 100 may be a high-performance network that is part of a high-performance computing (HPC) environment. In the network system 100, client devices 102 function as sending nodes and access servers 104. The servers 104 interact with each other and can interact with storage devices 106. The servers 104 and / or storage devices 106 function as receiving nodes and provide software / services to the client devices 102. The servers 104 and / or the storage devices 106 may be part of an HPC cluster. Servers 104 and storage devices 106 are only examples of devices accessible by client devices 102.The client devices 102 interact with the servers 104 via a network 108. The network 108 may be any wired or wireless network that enables communication between the client devices 102 and the servers 104. The client devices 102 and / or the servers 104 have network interface(s) for accessing the network 108. For example, the client devices 102 and / or the servers 104 may include network interface cards (NICs), wireless interface cards, or the like. The network 108 may include controllers, access points, switches, routers, or the like, for forwarding traffic between the client devices 102 and the servers 104. The client devices 102 may be any electronic devices suitable for executing executable code. The servers 104 may be bare metal machines that can host cloud components (e.g., virtual machines, containers, etc.). The storage devices 106 may be Network-Attached Storage (NAS) devices.The servers 104 and the storage devices 106 are interconnected via one or more network switches, including a network switch 110. The servers 104 and the storage devices 106 communicate with each other via packets transmitted between transmitting and receiving nodes. The servers 104 and the storage devices 106 are only examples of devices that may be interconnected via the network switch 110. Other devices may also be connected to the network switch 110. The packets may be forwarded via the network switch 110. The network switch 110 may have a high line rate, e.g., 800Gbit / s, 1600Gbit / s, or more. A high line rate may be desirable when the servers 104 and / or the storage devices 106 are part of an HPC cluster. The use of a wider data bus or higher clock frequency is a way to increase the line rate, however such techniques may not be suitable for routing small packets. As described in more detail below, the network switch 110 uses multiple header layers and multiple data crossbars to allow the network switch 110 to forward multiple packets per clock cycle. This has the advantage that the network switch 110 can maintain high line rates even when forwarding small packets.FIG. 2 is a block diagram of a network switch 200 as employed in some cases. Network switch 200 is an example of network switch 110 described above for FIG. 1. Network switch 200 includes ports 202 and a switching fabric 204. The ports 202 serve as connection points for nodes (e.g., servers, storage devices, etc.). The switching fabric 204 manages and forwards data packets between the ports 202.Each terminal 202 includes an input terminal 206 and an output terminal 208. Input ports 206 are the input points through which packets enter network switch 200. The output ports 208 are the output points responsible for transmitting the packets to the intended destinations. When a packet arrives at an input port 206, the network switch 200 checks the destination address of the packet to determine the appropriate output port 208 for transmission. This process, referred to as switching or forwarding, involves performing a search in a forwarding table of the network switch 200 to find the forwarding path for the packet.Each input port 206 receives packets from a node connected to the input port 206 and sends the packets to the output ports 208 in a one-to-multiple configuration. An input port 206 controls how the packets are sent to the output ports 208. Similarly, each output port 208 receives packets from the input ports 206 and sends the packets to a node connected to the output port 208 in a many-to-one configuration. An output port 208 controls how packets are received from the input ports 206.The ports 202 (including the input ports 206 and the output ports 208) are interconnected via the switching fabric 204 which provides the necessary paths for the transmission of packets from the input ports 206 to the output ports 208. The switching fabric 204 connects the input ports 206 and the output ports 208. Depending on the architecture of the network switch 200, the switching fabric 204 may be based on crossbar arrays, shared buses, shared memory, or the like. In one implementation, the switching fabric 204 includes multiple crossbares that are used for both control and data transfer between the input ports 206 and the output ports 208.The components of the network switch 200 may be implemented as integrated circuits, e.g., in one or more integrated circuits and / or one or more integrated circuit packages. The network switch 200 may include, for example, a processor, an application specific integrated circuit, a field programmable gate array, a memory, combinations thereof, or the like. One or more modules within the network switch 200 may be embodied in part or in whole as software and / or hardware for performing the functions described herein. For example, the buffers, queues, crossbar, transmitters, receivers, fabrics, etc. described herein may each be embodied as integrated circuit logic blocks. In some implementations, the components of the network switch 200 are implemented in multiple integrated circuits / packages that can be assembled into a common package.As described in more detail below, the switching fabric 204 includes multiple data crossbar. Packets are transmitted from an input port 206 to an output port 208 in parallel across the data crossbar of the switching fabric 204. Moreover, each port of network switch 200 includes components of multiple header layers that control the parallel transmission of packets over the data crossbar. Thus, multiple packets may be transmitted from the input port 206 to the output port 208 per clock cycle. The line rate of the network switch 200 can thus be maintained even with small packets.FIG. 3 is a block diagram of a network switch 300 as employed in some cases. The network switch 300 is an example of the network switch 200 described above in FIG. 2. Additional components of network switch 300 (including components of switching fabric 304, input port 306, and output port 308) are shown below. The network switch 300 includes a plurality of header layers 310 (including a first header layer 310A and a second header layer 310B) that control the parallel transmission of packets from the input port 306 to the output port 308. A logic flow is shown during the forwarding of packets from the input port 306 to the output port 308. Although forwarding of packets from an input port 306 to an output port 308 is described, it should be appreciated that each input port 306 may transmit packets to multiple output ports 308 and each output port 308 may receive packets from multiple input ports 306.First, the components of the network switch 300 will be described. The input port 306 includes a receiver 312, an input buffer 314, a header plane selector 316, and a plurality of input queues 318 (including a first input queue 318A and a second input queue 318B). Receiver 312 receives packets from a source node connected via input port 306. The packets are destined for a destination node connected via the output port 308. The source node and / or the destination node may be directly connected to the network switch 300, or one or more network components (e.g., additional switches) may be present between the network switch 300 and the source / destination node. Input buffer 314 is coupled to receiver 312. The received packets are stored in the input buffer 314.The header level selector 316 is also connected to the receiver 312. A header level selector 316 is circuitry, firmware, or other component used to select the components of the network switch 300 to control the transmission of a packet over a data crossbar. The input queues 318 are connected to the header level selector 316 and the input buffer 314, respectively. Thus, the input queues 318 are connected to the receiver 312 via the header level selector 316. Each input queue 318 is an input controller that controls the transfer of packets from the input buffer 314 to the output buffers of the various output ports 308. Requests to send packets to the various output ports 308 are queued by the receiver 312 at the input queues 318. Each input queue 318 decides the requests in the queue and selects a request to be processed. As described in more detail below, header level selector 316 provides load balancing between the requests in input queues 318. An input queue 318 determines to which output port 308 a packet for a selected request is to be forwarded. For example, an Ethernet lookup unit (not separately shown) may extract appropriate headers from the packet and use them to perform address translation for the destination node of the packet. An input queue 318 may receive the search result from the Ethernet lookup unit.The output port 308 includes a plurality of output queues 322 (including a first output queue 322A and a second output queue 322B), an output buffer 324, and a transmitter 326. Each output queue 322 is an output controller that controls the receipt of packets in output buffer 324 from the input buffers of the various input ports 306. Requests to receive packets from the various input ports 306 are queued in the output queues 322. Each output queue 322 decides the queued requests and selects a request to be processed. Output buffer 324 is coupled to output queues 322. The received packets are stored in output buffer 324. Transmitter 326 is connected to output buffer 324. Transmitter 326 reads packets from output buffer 324 and sends the packets to their destination nodes. Thus, by controlling the reception of packets in output buffer 324, an output queue 322 effectively controls the reading of the packets from output buffer 324 by transmitter 326.The network switch 300 may support packet pulling. Packet pulling, such as Virtual Output Queuing (VOQ), is an architectural technique that allows network switches to maintain line rates in congestion by reducing head-of-line blocking. An input queue 318 may maintain a virtual queue for each of the output ports 308. If congestion occurs at one output port 308, the virtual queue for that output port 308 may be blocked, while the virtual queues for the other output ports 308 may be further processed. In this way, the network switch 300 may support congestion management and / or traffic shaping properties.The switching fabric 304 comprises a plurality of cross-bars separated from each other. In this example, the switching fabric 304 includes multiple request crossbar 332 (including a first request crossbar 332A and a second request crossbar 332B), multiple grant crossbar 334 (including a first grant crossbar 334A and a second grant crossbar 334B), and multiple data crossbar 336 (including a first data crossbar 336X and a second data crossbar 336Y). A request crossbar 332 and a grant crossbar 334 are connected to an input queue 318 of each input port 306 and to a corresponding output queue 322 of each output port 308. As described in more detail below, transmit requests are sent over request crossbar 332, while transmit permissions are sent over grant crossbar 334. The data crossbar 336 is connected to the input buffer 314 of each input port 306 and to the output buffer 324 of each output port 308. As described in more detail below, packets are transmitted over the data crossbar 336.The data crossbar 336 may each have the same width (and thus provide the same bandwidth), or they may be different widths. In some implementations, the data crossbar 336 includes first data crossbar with a first (e.g., small) width and second data crossbar with a second (e.g., large) width. The first data crossbar 336 may be used to route small packets (requiring less bandwidth), while the second data crossbar 336 may be used to route large packets (requiring more bandwidth).The network switch 300 includes a plurality of header layers 310 (including a first header layer 310A and a second header layer 310B) for controlling the transmission of packets from the input port 306 to the output port 308. The first header layer 310A controls the transfer of first packets from the input buffer 314 to the output buffer 324 via one of the data crossbar 336, while the second header layer 310B controls the transfer of second packets from the input buffer 314 to the output buffer 324 via another of the data crossbar 336. The first header layer 310A may transmit packets independently of the second header layer 310B. The output queues 322 of the header layers 310 may coordinate traffic design through credits.The first header layer 310A includes the first input queue 318A of each input port 306, the first output queue 322A of each output port 308, the first request crossbar 332A, and the first grant crossbar 334A. The first request crossbar 332A and the first grant crossbar 334A are connected to the first input queue 318A of each input port 306 and to the first output queue 322A of each output port 308. The first header layer 310A handles first requests to transfer first packets from the input buffer 314 to the output buffer 324. The first header layer 310A may use either the first data crossbar 336X or the second data crossbar 336Y to transmit a packet. As described in more detail below, the selection of a data crossbar 336 by the first header layer 310A for packet transmission may be based on the utilization of the data crossbar 336.The second header layer 310B includes the second input queue 318B of each input port 306, the second output queue 322B of each output port 308, the second request crossbar 332B, and the second grant crossbar 334B. The second request crossbar 332B and the second grant crossbar 334B are connected to the second input queue 318B of each input port 306 and to the second output queue 322B of each output port 308. The second header layer 310B handles second requests to transfer second packets from the input buffer 314 to the output buffer 324. The second header layer 310B may use either the first data crossbar 336X or the second data crossbar 336Y to transmit a packet. As described in more detail below, the selection of a data crossbar 336 by the second header layer 310B for packet transmission may be based on the utilization of the data crossbar 336.Header level selector 316 provides load balancing between the requirements of receiver 312 and header levels 310. When receiver 312 stores a packet in input buffer 314, it also sends a request to header level selector 316. The header level selector 316 selects one of the header levels 310 to handle the request. The request is then forwarded to the input queue 318 of the selected header layer 310. The header level selector 316 may include a first-in-first-out (FIFO) buffer to queue requests before being distributed to the header levels 310. Since multiple packets are forwarded per clock cycle, the FIFO buffer of header level selector 316 is large, e.g., deep enough to accommodate requests for a burst of packets. In some implementations, the FIFO buffer of header level selector 316 is low enough to store two requests per header level 310.A logic flow for forwarding packets from input port 306 to output port 308 will now be described. Receiver 312 receives a packet and stores the packet in input buffer 314. A request to transmit the packet is sent to the header level selector 316. The header level selector 316 selects a header level 310. The criteria for selecting the header level will be described later. The header level selector 316 then queues the request to the input queue 318 of the selected header level 310, which then selects the request for processing. The input queue 318 sends a transfer request for the packet to the output queue 322 of the selected header layer 310 via the request crossbar 332 of the selected header layer 310. The transmission request includes a description of the packet; for example, the request may include information from a header of the packet.The transfer request is queued in the output queue 322 of the selected header layer 310, which then selects the transfer request for processing. The output queue 322 decides whether and when to issue the transmission request. For example, the output queue 322 may decide which transmission request to be issued next based on the packet descriptions of the transmission requests and the current state of the output buffer 324. In response to the grant of the transfer request, the output queue 322 sends a transfer grant to the input queue 318 via the grant crossbar 334 of the selected header layer 310.The transmit grant is a message instructing the input queue 318 to transmit the packet from the input buffer 314 to the output buffer 324. In response to receiving the transmit grant, the input queue 318 selects a data crossbar 336 and then transmits the packet from the input buffer 314 to the output buffer 324 via the selected data crossbar 336. The criteria for selection of the data crossbar will be described later. Transmitter 326 then reads the packet from output buffer 324. Output queues 322 thus control the reading of packets by transmitter 326 (and thus determine which packets are transmitted via transmitter 326) by granting the transmission request.The packets are transmitted in parallel over the data crossbar 336. For example, both a first and a second packet may be transferred from input buffer 314 to output buffer 324 via data crossbar 336. The first packet may be transmitted over the first data crossbar 336X while the second packet is transmitted over the second data crossbar 336Y. Because the data crossbar 336 is connected to each input buffer 314 and to each output buffer 324, the packets transmitted in parallel across the data crossbar 336 may be transmitted to the same output (as in this example) or to different outputs. A data crossbar 336 may be selected by an input queue 318 or other module based on the load of the data crossbar 336. For example, an input queue 318 or other module may select the data crossbar 336 that has the least pending transfer requests. Additionally, an input queue 318 or other module may select different data crossbar 336 for different packet transfers. For example, the input queue 318 or other module may select the first data crossbar 336X for a first packet transmission and then the second data crossbar 336Y for a second packet transmission.When a header layer 310 is selected (by the header layer selector 316) to handle a request to transmit a packet, that request may traverse the selected header layer 310 without changing to another header layer 310 during processing. For example, if the first header level 310A is selected by the header level selector 316, then the first input queue 318A sends the transfer request to the first output queue 322A via the first request crossbar 332A, and the first output queue 322A sends the transfer grant to the first input queue 318A via the first permit crossbar 334A. When the second header level 310B is selected by the header level selector 316, the second input queue 318B sends the transfer request to the second output queue 322B via the second request crossbar 332B, and the second output queue 322B sends the transfer grant to the second input queue 318B via the second grant crossbar 334B.The header level selector 316 may select a header level 310 based on one or more criteria. In some implementations, the requests are distributed to the header layers 310 to achieve load balancing. For example, the requests may be distributed in a round robin fashion. In some implementations, the requests for all packets of a particular frame type may be distributed to a particular header layer 310. For example, one header layer 310 may be used for all Ethernet frame requests, while another header layer 310 may be used for all HPC network frame requests. In some implementations, a header layer 310 is selected based on one or more header(s) of a packet. For example, the same header layer 310 may be used for all packets of the same packet flow. One or more fields in the header(s) may be looked up to compute a flow hash that is an integer value. Examples of packet header fields that can be leased are the source MAC / IP address, the destination MAC / IP address, the flow label, etc. The header level selector 316 can then select a header level 310 based on the flow hash. Each packet of a packet flow may have the same flow hash, and in this way the same header layer 310 is selected for each packet of the packet flow. As described below, the use of the same header layer 310 for each packet of a packet flow helps the packets maintain the proper order. The criteria for selecting a header layer 310 for a packet request may depend on whether the packet is an ordered or a disordered packet. For example, out-of-order packet requests may be distributed to header layers 310 according to the round robin principle, while ordered packet requests may be distributed to header layers 310 based on the information contained in the headers of the packets. Other variants are conceivable. For example, the header layer 310 may be selected based on the content of the packet itself (which may be evaluated using deep packet inspection), alone or in addition to one of the above criteria.A packet flow may be forwarded from the input port 306 to the output port 308. A packet flow is an ordered sequence of packets. When forwarding a packet flow, a single header layer 310 is used to control the forwarding of each packet of the packet flow. The single header layer 310 forwards the packets of the packet flow using multiple data crossbar 336. The order of packets in a packet flow should be maintained when forwarding the packet flow so that transmitter 326 reads the packets of the packet flow in the desired order from output buffer 324. However, if the packets of a packet flow are forwarded using multiple data crossbar 336, the packets of the packet flow may arrive at the output port 308 in a disordered order due to congestion of the switching fabric 304.The same header layer 310 may be used to handle each request to transmit a packet of a packet flow. In this manner, all transmission requests for a packet flow may pass through a selected header layer 310 without changing to a different header layer 310 during processing. In handling ordered packet requests, the header layer 310 is selected based on the information contained in the headers of the packets. In particular, a flow hash is calculated for the packets of the packet flow (as described above). The packets of the packet flow may all have the same flow hash. Header level selector 316 distributes requests having the same flow hash to the same header level 310. Thus, the transmission of all packets of a packet flow is controlled by the same header layer 310.In some implementations, the output queue 322 assigns a sequence number to the selected header layer 310 to each packet of a packet flow based on the order of the packets in the packet flow. In particular, each output queue 322 assigns sequence numbers to the transmission requests for packets of a packet flow in the order of the input of the requests. Each output queue 322 may have a sequencer (or counter) for each input port 306, and every time the output queue 322 grants a transmission request, it uses the value of the sequencer as a sequence number and increments the sequencer. When an output queue 322 grants a transmission request, the transmission grant sent to the input queue 318 of the selected header layer 310 contains the sequence number for the packet of that request. The sequence number for the packet is appended to the packet as it is transmitted from input buffer 314 to output buffer 324. Both the packet and the sequence number for the packet are stored in output buffer 324. Transmitter 326 reads the packets from output buffer 324 in the order of their sequence numbers. For example, if output buffer 324 contains both a first and a second packet of a packet flow, the packet having the lower sequence number may be read from output buffer 324 prior to the packet having the higher sequence number. In this manner, transmitter 326 reads the packets of the packet flow in the desired order from output buffer 324, even if the packets are not transferred in the proper order into output buffer 324.In handling a disordered packet request, the output queue 322 of the selected header layer 310 may indicate to the input queue 318 of the selected header layer 310 that the packet is a disordered packet (e.g., no sequenced packet). For example, the output queue 322 may assign a dummy sequence number to the packet. The dummy sequence number may be a predetermined value indicating that the packet is not part of a flow. If a packet in output buffer 324 has the dummy sequence number, transmitter 326 may read that packet from output buffer 324 at any time.FIG. 4 is a diagram of a packet forwarding method 400, in accordance with some implementations. Packet forwarding method 400 is described in connection with FIG. 3. The packet forwarding method 400 may be performed by the network switch 300 during forwarding of a packet from an input port 306 to an output port 308.The network switch 300 performs a step 402 of selecting a header level, the header level controlling the transfer of a packet from an input buffer to an output buffer, the header level comprising an input queue and an output queue. In particular, the selected header layer 310 controls the transfer of a packet from an input buffer 314 to an output buffer 324. The selected header layer 310 includes an input queue 318 and an output queue 322. The packet is stored by receiver 312 in input buffer 314. The header level selector 316 selects the selected header level 310 based on the above criteria, such as the frame type and / or header of the packet. As part of the header level selection, the header level selector 316 may calculate a flow hash for the packet by hashing fields in a header of the packet. The fields may be any of the aforementioned fields.The network switch 300 performs a step 404 of sending a transmission request from the header-level input queue to the header-level output queue. Specifically, a transfer request is sent from the input queue 318 of the selected header layer 310 to the output queue 322 of the selected header layer 310. The transmission request may include a description of the packet, e.g., information from the header of the packet.The network switch 300 performs a step 406 of receiving a transmission grant from the header-level output queue at the header-level input queue, the transmission grant comprising a sequence number for the packet. Specifically, a transmission grant is received from the output queue 322 of the selected header layer 310 at the input queue 318 of the selected header layer 310. The output queue 322 sends the transmission permission to the input queue 318 in response to the permission of the transmission request. If the transmission request includes a description of the packet, the output queue 322 may grant the transmission request based on the description of the packet. The transmission grant may include a sequence number for the packet. The sequence number may be assigned as described above. If the packet is a disordered packet (e.g., not part of a packet flow), the sequence number may be a dummy sequence number. If the packet is an ordered packet (e.g., part of a packet flow), the sequence number may be assigned to the packet based on the order of the packet in the packet flow. In the transmission request for the packet, it may be specified whether it is an ordered packet or an un ordered packet. The output queue 322 may assign the sequence number based on the order in which it receives requests. Of course, since the requests are sent to the output queue 322 in order in the packet flow, the sequence numbers may be assigned to the packets in this order.The network switch 300 performs an optional step of selecting a data crossbar 336 from the plurality of data crossbar 336 of the switching fabric 304. The selected data crossbar 336 may be selected based on a load of the selected data crossbar 336, as described above.The network switch 300 performs a step 408 of transmitting the packet and sequence number from the input buffer to the output buffer. Specifically, the packet and sequence number of the packet are transmitted from the input buffer 314 to the output buffer 324. The packet and its sequence number may be transmitted over the previously selected data crossbar 336.The network switch 300 performs a step 410 of reading the packet from the output buffer based on the sequence number. Specifically, the packet is read from the output buffer 324 based on the sequence number of the packet. Transmitter 326 reads the packet from output buffer 324. If the packet is part of a packet flow, the packets of the packet flow are read from the output buffer 324 in the order of their sequence numbers. If the packet is a disordered packet, transmitter 326 may read the packet whenever it wants.FIG. 5 is a diagram of a level selection method 500, according to some implementations. The plane selection method 500 will be described in conjunction with FIG. 3. The plane selection method 500 may be performed by the network switch 300 (in particular, the header plane selector 316) to control the forwarding of a received packet.The network switch 300 performs a step 502 of receiving a transmission request for a packet from the receiver 312. The receiver 312 may send the transmission request to the header level selector 316 when the receiver 312 stores the received packet in the input buffer 314.The network switch 300 performs a step 504 of selecting a header layer 310 of the header layers 310 based on one or more header(s) of the packet. For example, the header layer 310 may be selected by calculating a flow hash for the packet by hashing fields in the packet's header(s) and then selecting the header layer 310 based on the flow hash. The packet's header fields that are being leased may originate from the Ethernet header, IP header, TCP / UDP header, combinations thereof, and the like. If the packet is an encapsulated packet, the gapped header fields may originate from both the capsule header and the inner header of the encapsulated packet. The header layer 310 may also be selected according to other criteria mentioned above, e.g. according to the frame type of the packet.The network switch 300 executes a step 506 of queuing the transmission request into the input queue 318 of the selected header layer 310. The transfer request may then be sent to the output queue 322 of the selected header layer 310. A transmit grant may then be returned to the input queue 318 of the selected header layer 310, which may then transmit the packet to the output buffer 324 via one of the data crossbar 336 as described above.The techniques of the present disclosure may provide advantages. By using multiple data crossbar 336 for packet transmission, multiple packets may be transmitted in parallel across data crossbar 336, which may allow for higher performance compared to using a single data crossbar. Thus, multiple packets may be transferred from input buffer 314 to output buffer 324 per clock cycle. The line rate of the network switch 300 can thus also be maintained in forwarding small packets, including ordered and disordered packets. By using multiple header layers 310, the parallel transmission of packets over the data crossbar 336 can be independently controlled. The packet flow / flows may / may be distributed among multiple data crossbar 336, at packet level, to route the packet flow / flows.In the foregoing examples, the network switch 300 is described as being equipped with two data crossbar 336 and two header layers 310. However, it should be understood that the network switch 300 may have any desired number of data crossbar 336 and any desired number of header layers 310. In particular, the network switch 300 may have more crossbar 336 and / or more header layers 310 than described above, which may allow for the transmission of even more packets per clock cycle. Moreover, the switching techniques described herein may also be applied to other types of networks, such as wireless networks.FIG. 6 is a block diagram of a network switch 600 as employed in some cases. The network switch 600 is an example of the network switch 200 described above in FIG. 2. The network switch 600 may include a processor 602 and a memory 604. The memory 604 may be a non-transitory computer readable medium storing programming for execution by the processor. In this implementation, one or more modules within the network switch 600 may be partially or fully executed as software for performing the functions described herein. For example, the memory 604 may include instructions 612 for selecting a header level, the header level controlling transmission of a packet from an input buffer to an output buffer, the header level comprising an input queue and an output queue; instructions 614 for sending a transmission request from the input queue of the header level to the output queue of the header level; instructions 616 for receiving a transmission grant from the output queue of the header level at the input queue of the header level, the transmission grant comprising a sequence number for the packet; instructions 618 for transmitting the packet and the sequence number from the input buffer to the output buffer; and instructions 620 for reading the packet from the output buffer based on the sequence number.In an example implementation, an apparatus includes: an output buffer; a first data crossbar coupled to the output buffer; a second data crossbar coupled to the output buffer, the second data crossbar separated from the first data crossbar; an input buffer coupled to the first data crossbar and the second data crossbar; a first input queue configured to transfer a first packet from the input buffer to the output buffer via the first data crossbar; and a second input queue configured to transfer a second packet from the input buffer to the output buffer via the second data crossbar. In some implementations of the apparatus, the first packet is transmitted over the first data crossbar in parallel with the second packet transmitted over the second data crossbar. In some implementations, the apparatus further comprises: a header level selector configured to distribute a first transmission request for the first packet to the first input queue and a second transmission request for the second packet to the second input queue. In some implementations of the apparatus, the header level selector distributes the first transmission request and the second transmission request based on a first header of the first packet and on a second header of the second packet. In some implementations of the apparatus, the header level selector distributes the first transmission request and the second transmission request based on a first frame type of the first packet and a second frame type of the second packet. In some implementations of the apparatus, the first packet is part of a packet flow, and the first input queue is further configured to transfer each packet of the packet flow from the input buffer to the output buffer. In some implementations, the apparatus further comprises: a first output queue configured to receive a first transmission request for the first packet from the first input queue and transmit a first transmission grant to the first input queue; and a second output queue configured to receive a second transmission request for the second packet from the second input queue and transmit a second transmission grant to the second input queue. In some implementations, the apparatus further comprises: a first request crossbar coupled to the first input queue and the first output queue, the first input queue configured to send the first transfer request to the first output queue via the first request crossbar; a second request crossbar coupled to the second input queue and the second output queue, the second request crossbar separate from the first request crossbar, and the second input queue configured to send the second transfer request to the second output queue via the second request crossbar; a first grant crossbar connected to the first input queue and the first output queue, the first output queue configured to transmit the first transmission grant to the first input queue via the first grant crossbar; and a second grant crossbar connected to the second input queue and the second output queue, the second grant crossbar being separate from the first grant crossbar and the second output queue configured to transmit the second transmission grant to the second input queue via the second grant crossbar. In some implementations of the apparatus, the first transmission grant includes a first sequence number, the first output queue includes a first sequencer configured to generate the first sequence number, the second transmission grant includes a second sequence number, and the second output queue includes a second sequencer configured to generate the second sequence number.In an example implementation, an apparatus includes: a plurality of header layers, each of the header layers including an input queue, an output queue, a request crossbar connected to the input queue and the output queue, and a grant crossbar connected to the input queue and the output queue; a receiver and a header layer selector configured to: receive a transmission request for a packet from the receiver; select a header layer of the header layers based on a header of the packet; and queue the transmission request at the input queue of the header layer. In some implementations of the device, the input queue is configured to send the transfer request to the output queue via the request crossbar, and the output queue is configured to send a transfer grant to the input queue via the permit crossbar. In some implementations of the apparatus, the header level selector is configured to select the header level by calculating a flow hash for the packet by hashing fields in the header of the packet, wherein the header level is selected based on the flow hash.In an example implementation, a method comprises: selecting a header level, the header level controlling transmission of a packet from an input buffer to an output buffer, the header level including an input queue and an output queue; sending a transmission request from the input queue of the header level to the output queue of the header level; receiving a transmission grant from the output queue of the header level at the input queue of the header level, the transmission grant including a sequence number for the packet; transmitting the packet and the sequence number from the input buffer to the output buffer; and reading the packet from the output buffer based on the sequence number. In some implementations, the method further comprises: calculating a flow hash for the packet by hashing fields in a header of the packet, wherein the header level is selected based on the flow hash. In some implementations of the method, the header level is selected based on a frame type of the packet. In some implementations of the method, the transmission request includes a description of the packet, and the transmission request is granted by the output queue based on the description of the packet. In some implementations of the method, the packet is a disordered packet and the sequence number is a dummy sequence number. In some implementations of the method, the packet is part of a packet flow, the sequence number is assigned to the packet based on the order of the packet in the packet flow, and a plurality of data crossbar is used to transfer each packet of the packet flow from the input buffer to the output buffer. In some implementations, the method further comprises: selecting a data crossbar from a plurality of data crossbar, wherein transmitting the packet and sequence number from the input buffer to the output buffer comprises transmitting the packet and sequence number over the data crossbar. In some implementations of the method, the data crossbar is selected based on a load of the data crossbar. The method may be implemented in the form of instructions stored on a non-transitory computer readable medium.The foregoing describes features of several examples in order that those skilled in the art may better understand the aspects of the present disclosure. Various modifications and combinations of the illustrative examples, as well as other examples, will be apparent to those skilled in the art from the description. It is therefore intended that the appended claims cover all such modifications.

Claims

An apparatus comprising: an output buffer; a first data crossbar coupled to the output buffer; a second data crossbar coupled to the output buffer, the second data crossbar separated from the first data crossbar; an input buffer coupled to the first data crossbar and the second data crossbar; a first input queue configured to transfer a first packet from the input buffer to the output buffer via the first data crossbar; and a second input queue configured to transfer a second packet from the input buffer to the output buffer via the second data crossbar.The apparatus of claim 1, wherein the first packet is transmitted over the first data crossbar in parallel with the second packet transmitted over the second data crossbar.The apparatus of claim 1, further comprising: a header level selector configured to distribute a first transmission request for the first packet to the first input queue and a second transmission request for the second packet to the second input queue.The apparatus of claim 3, wherein the header level selector distributes the first transmission request and the second transmission request based on a first header of the first packet and on a second header of the second packet.The apparatus of claim 3, wherein the header level selector distributes the first transmission request and the second transmission request based on a first frame type of the first packet and a second frame type of the second packet.The apparatus of claim 1, wherein the first packet is part of a packet flow and the first input queue is further configured to transfer each packet of the packet flow from the input buffer to the output buffer.The apparatus of claim 1, further comprising: a first output queue configured to receive a first transmission request for the first packet from the first input queue and transmit a first transmission grant to the first input queue; and a second output queue configured to receive a second transmission request for the second packet from the second input queue and transmit a second transmission grant to the second input queue.The apparatus of claim 7, further comprising: a first request crossbar connected to the first input queue and the first output queue, the first input queue configured to send the first transfer request to the first output queue via the first request crossbar; a second request crossbar connected to the second input queue and the second output queue, the second request crossbar being separate from the first request crossbar and the second input queue configured to send the second transfer request to the second output queue via the second request crossbar; a first grant crossbar connected to the first input queue and the first output queue, the first output queue configured to send the first transfer grant to the first input queue via the first grant crossbar; a second grant crossbar coupled to the second input queue and the second output queue, wherein the second grant crossbar is separate from the first grant crossbar, and the second output queue is configured to send the second transmission grant to the second input queue via the second grant crossbar.The apparatus of claim 7, wherein the first transmission grant comprises a first sequence number, the first output queue comprises a first sequencer configured to generate the first sequence number, the second transmission grant comprises a second sequence number, and the second output queue comprises a second sequencer configured to generate the second sequence number.An apparatus comprising: a plurality of header layers, each of the header layers comprising an input queue, an output queue, a request crossbar connected to the input queue and the output queue, and a grant crossbar connected to the input queue and the output queue; a receiver; and a header layer selector configured to: receive a transmission request for a packet from the receiver; select a header layer from the header layers based on a header of the packet; and rank the transmission request into the input queue of the header layer.The apparatus of claim 10, wherein the input queue is configured to send the transfer request to the output queue via the request crossbar, and the output queue is configured to send a transfer grant to the input queue via the grant crossbar.The apparatus of claim 10, wherein the header level selector is configured to select the header level by calculating a flow hash for the packet by hashing fields in the header of the packet, wherein the header level is selected based on the flow hash.A method comprising: selecting a header level, the header level controlling transmission of a packet from an input buffer to an output buffer, the header level comprising an input queue and an output queue; sending a transmission request from the input queue of the header level to the output queue of the header level; receiving a transmission grant from the output queue of the header level at the input queue of the header level, the transmission grant comprising a sequence number for the packet; transmitting the packet and the sequence number from the input buffer to the output buffer; and reading the packet from the output buffer based on the sequence number.The method of claim 13, further comprising: calculating a flow hash for the packet by hashing fields in a header of the packet, wherein the header level is selected based on the flow hash.The method of claim 13, wherein the header level is selected based on a frame type of the packet.The method of claim 13, wherein the transmission request comprises a description of the packet, and the transmission request is granted by the output queue based on the description of the packet.The method of claim 13, wherein the packet is a disordered packet and the sequence number is a dummy sequence number.The method of claim 13, wherein the packet is part of a packet flow, the sequence number is assigned to the packet based on the order of the packet in the packet flow, and a plurality of data crossbar is used to transmit each packet of the packet flow from the input buffer to the output buffer.The method of claim 13, further comprising: selecting a data crossbar from a plurality of data crossbar, wherein transmitting the packet and sequence number from the input buffer to the output buffer comprises transmitting the packet and sequence number over the data crossbar.The method of claim 19, wherein the data crossbar is selected based on a load of the data crossbar.

Images